On 22 June 2023, Judge P. Kevin Castel of the US District Court for the Southern District of New York issued an opinion that would become a landmark in AI ethics. In Mata v. Avianca, he sanctioned two lawyers and their firm $5,000 for submitting a legal brief that cited six entirely fabricated court cases generated by ChatGPT. The lawyers had asked ChatGPT for relevant precedents, received plausible-sounding citations, and filed them without verification. The case established that AI-generated legal content requires the same verification as any other source, and the failure to verify constitutes sanctionable conduct.

What happened

Roberto Mata filed a personal injury lawsuit against Avianca Airlines in February 2022, alleging he was injured when a metal serving cart struck his knee during a flight. His lawyers at Levidow, Levidow & Oberman used ChatGPT to research relevant case law.

ChatGPT generated citations to cases that appeared legitimate: names, courts, dates, and legal reasoning that read like real judicial opinions. The cases included “Varghese v. China Southern Airlines,” “Shaboon v. Egyptair,” “Petersen v. Iran Air,” and others. None of them existed.

When Avianca’s lawyers could not locate the cited cases in legal databases, they notified the court. Judge Castel ordered Mata’s lawyers to produce copies. The lawyers asked ChatGPT if the cases were real; it assured them the cases “indeed exist” and “can be found in reputable legal databases such as LexisNexis and Westlaw.” They then submitted documents purporting to be the full opinions, which were also AI-generated fabrications.

In his June 2023 opinion, Judge Castel described one of the fabricated legal analyses as “gibberish.” He found that the lawyers had acted with “subjective bad faith” sufficient for sanctions under Federal Rule of Civil Procedure Rule 11. The $5,000 penalty was joint and several against both lawyers and the firm. The court also required them to notify each judge falsely named as an author of a fake opinion.

Why it mattered

Mata v. Avianca became the first widely reported case of lawyers being sanctioned for AI-hallucinated citations. It established several principles that courts and bar associations would build on:

Verification is non-delegable. The court made clear that using AI for research does not excuse lawyers from their professional duty to verify the accuracy of cited authority. ChatGPT’s confident assertions are not a substitute for checking the source.

Hallucination is not a defense. The lawyers argued they did not know ChatGPT would fabricate cases. The court found this irrelevant: the duty to verify exists regardless of the tool used.

The harm is real. Judge Castel enumerated the harms from fake citations: wasted opposing counsel time, wasted court resources, damage to the reputation of judges falsely named as authors, damage to parties attributed with fictional conduct, and erosion of trust in the legal system.

The escalation through 2026

Mata v. Avianca was not an isolated incident. A public database maintained by legal researcher Damien Charlotin tracked AI hallucination incidents in court filings. By early 2026, it documented over 1,200 cases worldwide where AI-hallucinated material appeared in legal filings, up from roughly 200 a year earlier. New cases were being added at a rate of five to six per day.

In the first quarter of 2026 alone, US courts imposed over $145,000 in sanctions for AI hallucinations in legal filings. A single Oregon case reached a record $110,000 penalty. The pattern was consistent: lawyers used AI tools for research, received fabricated citations, and filed them without verification.

Across roughly 40 million US court cases filed since January 2023, documented AI hallucination incidents numbered around 955 by mid-2026. The absolute number is small, but the reputational and professional consequences for those caught are severe: sanctions, malpractice exposure, and in some cases disciplinary proceedings.

The bar association response

In July 2024, the American Bar Association issued its first formal ethics opinion on the responsibilities of lawyers using generative AI. The 15-page opinion outlined how the Rules of Professional Conduct apply to AI use in legal practice. Key points:

  • Lawyers cannot reasonably rely on the accuracy of AI-generated content without independent verification.
  • The duty of competence requires understanding the capabilities and limitations of AI tools.
  • Supervision obligations apply to AI-assisted work by associates and staff.
  • Client confidentiality considerations apply when inputting case information into AI tools.

State bars followed with their own guidance. Many courts began requiring lawyers to certify whether AI was used in preparing filings and to confirm that any AI-generated content was verified.

Lessons for builders

The legal profession’s experience with AI hallucinations offers broader lessons:

High-stakes domains require verification layers. In any domain where errors have significant consequences, whether legal, medical, financial, or otherwise, AI output must be verified against authoritative sources before use.

User confidence does not equal accuracy. ChatGPT’s confident tone led lawyers to trust its output. Confidence calibration is a known weakness of large language models. Systems that surface uncertainty or require source verification reduce this risk.

Retrieval-augmented generation (RAG) helps but does not eliminate the problem. Legal research tools that retrieve from verified case databases and generate summaries reduce hallucination risk compared to pure generation. But even RAG systems can misquote, misattribute, or misrepresent sources, so human verification remains essential.

Professional responsibility adapts to new tools. Every profession that adopts AI will face its own version of the Mata question: when AI makes an error, who is responsible? The answer, consistently, is the human professional who deployed the tool without adequate verification.

Sources

Further reading