AI Hallucinations: a Costly Mistake for Lawyers

We’ve written extensively on how legal professionals can use AI responsibly—and, as yet one more courtroom choice is launched addressing legal professionals’ use of AI, it’s by no means been extra vital to know the dangers and limitations of AI use. 

Most lately, legal professionals from Morgan & Morgan—the most important private damage regulation agency in america—have been sanctioned for submitting courtroom filings that contained instances “hallucinated” by synthetic intelligence. 

Under, we’ll clarify what hallucinations are, evaluate the courtroom’s choice to sanction Morgan & Morgan’s legal professionals, and supply suggestions for mitigating threat when working with AI for authorized analysis.

Irrespective of the place you’re at in your authorized AI journey, Clio Duo, our proprietary AI know-how, will help you run your observe extra effectively whereas adhering to the best safety requirements. Study the way it works. 

What are AI hallucinations? 

AI hallucinations happen when a big language mannequin (LLM) generates false or deceptive data that, on its face, seems believable. Within the context of authorized analysis, because of this AI can—and does—produce case regulation, statutes, or authorized arguments that don’t exist. And, if legal professionals depend on AI-generated analysis with out independently verifying that the analysis is correct, they threat misinterpreting the regulation and dealing with critical skilled and moral penalties—just like the sanctions imposed on Morgan & Morgan’s legal professionals. 

What occurred with Morgan & Morgan’s AI utilization? 

Morgan & Morgan have been representing a plaintiff in a merchandise legal responsibility case. Whereas drafting the case motions (particularly, motions in limine), one of many legal professionals concerned used the agency’s in-house AI platform to find instances setting forth necessities for motions in limine. The AI platform generated citations—however, it seems, they weren’t actual instances. Additional, the drafting lawyer didn’t confirm the accuracy of the instances earlier than signing the filings together with two different legal professionals. 

How did the courtroom reply to the hallucinations? 

The courtroom issued a present trigger order on why the plaintiffs’ legal professionals shouldn’t be sanctioned or face disciplinary motion. The plaintiffs’ legal professionals admitted that the instances that they had cited didn’t exist and that that they had relied on AI with out taking steps to confirm the accuracy of the knowledge within the motions. 

On the time the order was launched, the plaintiffs’ legal professionals had already withdrawn their motions in limine, been “sincere and forthcoming” about their AI use, paid opposing counsel’s charges for defending the motions in limine, and carried out inner agency insurance policies to forestall such errors from occurring sooner or later. 

The courtroom famous its appreciation for the steps the plaintiffs’ legal professionals had taken, and beneficial that legal professionals in related conditions ought to observe the identical steps (at a minimal) to remediate conditions involving AI hallucinations previous to the issuance of sanctions sooner or later. 

What does the regulation say about sanctions and AI hallucinations? 

Below s. 11 of the Federal Guidelines of Civil Process, by presenting a pleading or movement to the courtroom (whether or not by signing, submitting, submitting, or advocating for it), a lawyer certifies that, to the most effective of their data, the authorized contentions within the pleading or movement are supported by current regulation or a nonfrivolous argument. Failing to adjust to this requirement can lead to sanctions. 

When figuring out whether or not sanctions are warranted below Rule 11, the courtroom will consider the state of affairs with “goal reasonableness.” In different phrases, if an affordable inquiry finds that the authorized contentions within the pleading or movement should not supported by current regulation, then a violation of Rule 11 has occurred. And, the place a violation of Rule 11 has occurred, the courtroom will impose an acceptable sanction. 

How the courtroom utilized the regulation to the case

The courtroom decided that the legal professionals’ conduct violated Rule 11. Notably, the legal professionals didn’t dispute that that they had cited AI-hallucinated instances within the temporary. Moreover, the courtroom acknowledged that signing a authorized doc signifies that the lawyer learn the doc and performed an affordable inquiry into the present regulation. Regardless of two of the three legal professionals not having been concerned within the drafting course of, their signatures (with failure to additional examine the content material of the motions) amounted to an improper delegation of their obligation below Rule 11. 

Having discovered that the legal professionals violated Rule 11, the courtroom imposed sanctions in opposition to all three legal professionals. The drafting lawyer was fined $3,000 and had his short-term admission revoked (he was licensed in one other state, and granted permission to observe in state for the case in query) whereas the opposite two legal professionals have been fined $1,000 every. 

Ultimate ideas on the Morgan & Morgan AI case

Finally, the courtroom stated it finest: “The important thing takeaway for attorneys is straightforward: Make an affordable inquiry into the regulation earlier than signing (or giving one other particular person permission to signal) a doc, as required by Rule 11.” Whereas the courtroom acknowledged that AI is a strong instrument for authorized professionals and may scale back the time spent on authorized analysis and drafting considerably, on the finish of the day, legal professionals nonetheless want to verify the accuracy of their work merchandise. 

Ideally, legal professionals is not going to run into related points with AI hallucinations sooner or later—however, when it occurs, it’s finest to take instant motion. The courtroom acknowledged as a lot, expressing its appreciation for the steps taken by the plaintiffs’ legal professionals and recommending that, at a minimal, legal professionals in related conditions ought to: 

Withdraw their motions;
Be sincere and forthcoming about their AI use;
Pay any charges incurred by opposing counsel in responding to the motions; and
Take inner steps to forestall such errors from occurring sooner or later. 

And don’t overlook: Authorized-specific AI instruments can play an vital position in serving to your agency undertake AI responsibly. For instance, Clio Duo, our proprietary AI know-how, will help improve effectivity whereas safeguarding shopper knowledge. Guide your free demo in the present day! 

We printed this weblog put up in February 2025. Final up to date: February 28, 2025.

Categorized in:
Expertise