What’s the Matter With Mata v. Avianca, Inc

Let’s Take the Right Lessons

Earlier today, the New York Times and The Verge ran stories about an attorney who referenced nonexistent cases fabricated by ChatGPT. This news is poised to ignite discussions among legal tech enthusiasts, social media commentators, and mainstream news outlets. The interpretations of this situation will likely be as varied as a Rorschach test, ranging from snide comments against the use of generative AI in law, to attempts at absolving lawyers, and even apocalyptic predictions about AI. However, I believe we need to delve deeper, beyond surface-level assumptions and the partial information currently available.

Here is a high level summary of where things stand, as of today: A recent lawsuit filed by a man named Roberto Mata against the airline Avianca has ignited a profound discussion about the implications of AI-generated content and its use in legal proceedings. The current matter unfolded when Steven A. Schwartz, the lawyer representing Mata, relied on a generative AI model - ChatGPT - to prepare his court filing, citing numerous judicial decisions supposedly relevant to the case. However, it was discovered that these cases, along with the quotes and citations used, were entirely fabricated by ChatGPT. This resulted in a situation where a lawyer submitted a document full of invented cases and quotations to the court. According to the New York Times, Judge Castel (who presides over this case) said in an order that:

he had been presented with “an unprecedented circumstance,” a legal submission replete with “bogus judicial decisions, with bogus quotes and bogus internal citations.” He ordered a hearing for June 8 to discuss potential sanctions.

As I remarked on Twitter, my own immediate reaction when I first became aware of this situation echoed a common caution I express about using generative AI for legal work - “It can generate fabricated results, so it's imperative to critically evaluate and verify the information before using it.”

But the more I considered the facts as they are currently known, the more facts I wanted to know. Specifically, I’d love to see the entire prompt chain leading up to, including, and after the results that were filed with the court. It’s entirely possible the lesson here may not be simply that an unwitting lawyers mistakenly relied upon information that any ordinary person would have taken as fact.

In a blog post by Simon Wilison published earlier today on this matter, he notes:

I’ve been trying to come up with prompts to expand that false “Varghese v. China Southern Airlines Co., Ltd., 925 F.3d 1339 (11th Cir. 2019)” case into a full description, similar to the one in the screenshots in this document.

Even with ChatGPT 3.5 it’s surprisingly difficult to get it to do this without it throwing out obvious warnings.

I’m trying this today, May 27th. The research in question took place prior to March 1st. In the absence of detailed release notes, it’s hard to determine how ChatGPT might have behaved three months ago when faced with similar prompts.

So there’s another version of this story where that first set of citations was an innocent mistake, but the submission of those full documents (the set of screenshots from ChatGPT that were exposed purely through the presence of the OpenAI down arrow) was a deliberate attempt to cover for that mistake.

Wilison's insights highlight one plausible scenario. Still, it underscores the fact that there are numerous potential fact patterns that may yet come to light. We may be only scratching the surface of the story at this point, and as further details unfold, more wrinkles and complexities may emerge as well.

Ideally, the proceedings to follow will result in a full and public airing of the entire sequences of prompts leading to the results submitted to the court. Once the full circumstances come to light, it wouldn't be surprising if the key lessons are more about the frailties and foibles of certain individuals and only remotely about the competent and responsible use of generative AI in legal contexts, which was my initial reaction. Or perhaps there are other factors in play. Intriguingly, the facts may take us places we don’t expect. Let’s take care to have the lessons flow from the facts.