There is a particular species of institutional failure that does not announce itself with a bang but rather accumulates, quietly, in the footnotes of motions filed in Los Angeles Superior Court. What we are witnessing in the State Farm litigation is not a scandal in the sensational sense. It is something more banal and therefore more alarming: the routine colonization of a profession’s core epistemic function by a technology that cannot perform it, administered by people who could not be bothered to notice.
Fa’alagilagi Meni-Siliga, a U.S. Postal Service carrier, lost her Carson home to a fire in 2020. The rebuild never happened. She drained her savings, drained her retirement, declared bankruptcy, and lost the house to foreclosure. She sued State Farm. State Farm’s lawyers responded with motions citing, in the words of her attorney Eric Khodadian, “cases that do not exist, quotes that do not exist, and holdings that do not exist.” Seven fabricated case citations across eight filings. The lawyer who generated them, Jacquelene Robinson, used an AI tool called Irys, which she believed, incorrectly, was connected to Westlaw and performed automatic cite-checking. It did not. She did not verify. Lead counsel Kenneth Katel says he did not even know AI had been used, which is either a confession of negligence or a lie, and in either case disqualifying.
One hesitates to call this surprising. The State Bar of California has now filed disciplinary charges against three attorneys in separate matters for precisely this conduct. Omid Emile Khalifeh cited a nonexistent case and two irrelevant ones in a federal trademark filing, then told the court he had “independently verified” the citations, which was not true. Sepideh Ardestani submitted nonexistent citations in a wage-and-hour class action and blamed her handwritten notes, producing no documents to support that explanation. Steven Thomas Romeyn admitted he “did not verify every single citation prior to filing.” A custody dispute over an aging Labrador produced AI-fabricated precedents from both sides. A Paris-based researcher has catalogued 1,922 instances of legal hallucination worldwide, most in the United States. Last year a California attorney was fined $10,000 for a brief containing 21 invented quotes out of 23.
The structural diagnosis is straightforward, and I have grown tired of making it. Generative language models do not retrieve information. They produce statistically plausible sequences of text. When you ask one for a case citation, it does not search a database. It generates a string that resembles a case citation. Sometimes that string corresponds to a real case. Sometimes it does not. The model does not know the difference because there is no “knowing” happening. This is not a bug. It is the architecture.
What is remarkable is not that the technology fails. Technologies fail. What is remarkable is the eagerness with which a profession whose entire claim to social authority rests on verification, on the careful authentication of authority, has outsourced that function to a system that cannot perform it, and then expressed surprise when the results are fabricated. Stanford’s Daniel Ho and colleagues demonstrated last year that legal AI vendors have “overstated” their tools’ ability to guarantee hallucination-free citations. Forty-one percent of law firms and 47 percent of corporate legal departments now use generative AI. The number who can articulate what a large language model actually does, as opposed to what the marketing materials claim it does, is presumably rather smaller.
And then there is the matter of who bears the cost. Meni-Siliga lost her home. State Farm, a company with roughly $90 billion in revenue, deployed external counsel who could not be troubled to confirm that the cases they cited were real. When Khodadian presented his findings in court, Katel was, by his own admission, upset. Khodadian says Katel followed him down the hall “repeatedly demanding to see the list.” One imagines the scene: a lawyer who did not read his own filings, enraged at the man who did.
The State Bar’s chief trial counsel, George Cardona, issued a statement noting that “technology can assist legal practice, but it does not replace an attorney’s duty of competence, diligence, and honesty.” This is correct and also insufficient. The problem is not that attorneys need a reminder about their duties. The problem is that the economic incentives of the profession, particularly in insurance defense, reward volume over verification, and that the introduction of generative AI has supercharged those incentives while providing a ready-made excuse for their consequences. Ardestani received a 30-day suspension and ten hours of continuing legal education. The Eastern District of California called the time spent on her misconduct “a waste of limited judicial resources in a district that has labored under a longstanding caseload crisis.” Meanwhile, L.A. County judges have been given their own AI tool, called Learned Hand, to summarize motions and draft rulings. The named reference to Judge Learned Hand, one of the most rigorous jurists in American history, is almost too on the nose.
The discipline of law has always rested on a simple proposition: that the statements made in court can be checked against an external reality of statutes, precedents, and recorded decisions. When that checking stops, what remains is not law. It is rhetoric with a filing fee.