Law & AI stuff

#16 Top law got caught, zone got flood, and plaintiff got coached

Big Law, bad cites

Bankruptcies are messy. It’s not surprising: save for the occasional financial shenanigans, one does not enter bankruptcy eagerly or well-prepared: the process typically starts at a point when things are bad or about to get worse. But bankruptcy is also messy because, as a legal field, it’s interested in everything a company does or owns, and many of these things are not necessarily “lawyer-ready”, especially since things can move fast when a company goes under.

And so, a large part of bankruptcy law can be viewed as a way to bring order to this messiness so as to corral the process towards an ordered outcome. Now, the way lawyers typically order things, is that they lay down rules and procedures to be followed. Many rules, in fact: in the USA, the canonical .pdf for the bankruptcy rules runs to 166 pages, more than the 140 pages of the federal rules of civil procedure, which they come to supplement.

But there is a tension here: this is a complicated field that calls for very specialised lawyers, and often teams of them to handle the more complicated cases and order the mess. Lawyers that are, then, expensive. At the same time, the clients are not necessarily “lawyer-ready”, especially in the one way that matters: they can’t always pay you, or pay you well (they are bankrupt !). The temptation is therefore strong to take shortcuts, and in 2026, this means relying on AI.

Earlier this week, as Bloomberg reported:

One of Wall Street’s prominent law firms, Sullivan & Cromwell, wrote to a bankruptcy judge to apologize for a court motion that included inaccurate citations generated by artificial intelligence, according to a filing in the US Bankruptcy Court for the Southern District of New York.

While the hallucination database is still growing strong, the media attention has tempered a bit in the last few months, deservedly: bashing lawyers is great fun, but gets repetitive after a bit. Yet this episode rekindled the interest significantly, probably because we are talking about “Big Law”, and not your sole practitioner that mistakenly thought ChatGPT could serve as a junior associate. The schadenfreude found a new outlet, with many online comments putting the mishap in contrast to the average billing rate of S&C’s lawyers.

Two things are interesting from this episode.

First, as the preceding developments should make clear, this is the ideal case for a blunder on the part of a sophisticated law firm: a complex and sensitive case, subject to time-pressure, with - what I imagine - many different people working on the brief in question. Most cases of AI misuse in the database come down to an individual mistake, something nobody had time to review. But complicated cases are vulnerable to this as well: the more people involved, the more sophisticated the pipeline to produce a brief, the higher the chances that something will slip through unnoticed. The pipeline is only as strong as its weakest human link.

Second, to some extent the media attention is unfair: Sullivan & Cromwell handled the accident as well as they could have done, and in fact with greater grace than most parties in the database. They came forward, through a letter by a senior lawyer, that acknowledged and owned the error. They went the extra mile and reviewed other filings, and disclosed further (non-AI) mistakes. In particular, they refrained from blaming the intern or employee, describing it - in ways that there is no ground to doubt - on the failure to apply existing rules. Errors happen, their letter’s tone suggest, and who can disagree?

But this might not matter: while the court is yet to comment on the episode, at least one other party later took the opportunity to request an adjournment of an upcoming hearing, in the name of the “integrity of the proceedings”. There is no mess that can’t be profited from.

Flooding the zone

A theme around here has been that the rise of AI will put a lot of stress on the legal system. This is the “gym membership” model of the law: it “works” only because people don’t use it to the full extent of their rights. Or, put another way, if every potential case snaked its way through the courts, the latter would be bloated and could not deal with the demand, resulting in injustice. But if AI means that every litigant has an attorney in their pocket, the barriers to that demand are expected to fall dramatically.

These are predictions, but there starts to be data bearing them out. Recently, for instance, the President of the Australian Fair Work Commission pointed out that this jurisdiction - specialised in labour law, and before which people frequently represent themselves - saw a 70% surge in the number of cases, such that “the commission was failing to meet its targets to resolve cases for the first time in many years”.

Meanwhile, a recent paper by Anand Shah and Joshua Levy ran the numbers for the USA, and found that

First, the number of pro se cases—or self-represented cases—is increasing dramatically, rising from a long-term steady-state average of 11% to 16.8% in FY2025. This increase is concentrated in case types characterized by formulaic document production and absent from more complex, attorney-intensive categories. Second, we argue these cases are placing larger burden on federal district courts. Pro se cases are not terminating faster, and this combined with the increased case numbers suggests more cases for judges to process. Moreover, intra-case activity is up, with the total volume of docket entries per court generated by pro se cases in their first 180 days up 158% from pre-AI means to 2025.

[…]

Using a random sample of 1,600 complaints drawn from an 8-year period (2019-2026), we find that a large and growing share of complaints are flagging positive for AI-generated text, from essentially zero in the pre-AI period to more than 18% in 2026.

Nice stuff. And there are two ways you can look at this data.

The first is, as they did it themselves in the paper’s title, to stress it as a progress for access to justice: if pro se cases rise in cases that are rather formulaic - e.g., “civil rights complaints, consumer credit disputes, foreclosure proceedings” -, and if the rate of “wins” stays constant, this matches a story of individuals asserting their rights in fields that, until then, were under-serviced by the legal profession despite being viable. Moreover, the fact that AI-generated text can be identified in only 18% of filings - with all the caveats that AI-detection warrants, of course - is coherent with the idea that AI merely helps people identify their rights and act upon them, rather than full delegation to the LLM.

But the second way to look at the result is to focus on the substantial rise in the number of cases, and to wonder what this will do for the justice system as a whole. Shah and Levy find that, for now, the average case duration remains within the same band as in the past, suggesting that courts are managing to cope with the increased flow of pro se cases. But this might not hold forever, and at some point the backlog will compound and hit the limits of a system whose supply side is unusually rigid: judgeships grow slowly, judges can hardly be “scaled”, and federal judges are, for now, not supposed to use AI to draft opinions. If average resolution time stays the same, then there will be cause to wonder if this is not at the cost of a decrease in quality.

Plus, there are other, more pervasive effects of a rise in cases: the post-AI equilibrium may be one in which both sides stress the docket with more submissions, absorbing ever more judicial attention. And this increased input will have asymmetrical impact for defendants: institutional parties, in particular, might struggle to cope with the onslaught of claims.

The second reading does not negate the idea of a greater access to justice, but it helps qualify it when seen in action: when legal demand is less restrained than it used to be, supply has to adapt. And business practice teaches us that there are two ways to go at it: scale capacity, or reintroduce scarcity. Watch out for the latter in the form of new, increased frictions.

Ventriloquism, now digital

Beyond ordering the mess of real life to make it “law-ready”, one thing procedural rules encode are the two main media to “practice” law: text and speech. Most legal systems keep room for both, in recognition of their distinct merits.

Text, through briefs and submissions, is the medium for complex legal reasoning, the development of legal argument, and the exhaustive marshalling of sources. It’s also a coordination device: the court might not always be in session or ready to hear you, so putting your case down allows the decision-maker to turn to it in her own time and on her own terms. Speech, meanwhile, is typically reserved for the important human issues in a case: the credibility of a witness, the testing of reactions in real time, or the greater stakes of a case as spelled out in a ringing closing statement.

And because these two media do different things, procedure has long treated them differently. We tolerate, and indeed expect, a great deal of intermediation in writing: lawyers draft, redraft, polish, and decide what and when to brief. Speech, by contrast, is often where the system insists most strongly on immediacy, spontaneity, and the presence of the human being supposedly speaking. The assumption being that truth lies in direct human speech.

A few weeks ago, we saw an incident in the UK in which a witness used Smart Glasses to get testimony advice in real time. While the court mused that an AI was involved, the reality was more mundane - though still very new - as the witness had simply been on a call with a human coach.

Still, that was bound to happen, and we did not have to wait long. In an employment-discrimination case involving US airline Delta, the parties appeared before the court over a discovery dispute, and then this happened (via):

During this conference, Delta’s counsel raised two primary issues with the Court. […] Second, Delta’s counsel noted that “Jones appeared to be reading materials from her screen while responding to questions, and when asked about that, [Jones] admitted that she had an [artificial intelligence] platform open on her device, which was . . . ChatGPT.” […] Delta’s counsel then asked Jones “whether [Jones] was feeding information into [ChatGPT] as the deposition was progressing” and Jones refused to answer, “citing attorney/client privilege as the reason for refusing to answer” the question despite confirming that she was not being represented by counsel.

[…]

Jones said that she was using ChatGPT to assist her in knowing how to “proceed legally as a pro se representation, the same as if there was in person an attorney in which they would object to different things.”

The court eventually forbade her from using ChatGPT in her deposition.

You can see where the court’s concern comes from: there is fear that the human is not only using AI as a crutch, but simply engaged in some form of ventriloquism, with the AI speaking through her. Still, how is this different - and why should this deserve a distinct treatment - from humans writing a brief with an LLM, which is currently allowed ?

One possible answer comes back to the distinction between text and speech: using ChatGPT during a deposition is objectionable not because the output is generated by AI, but because it cuts against the principle of live testimony, meant to test a party’s own knowledge and candour under pressure.

But that distinction is being stressed, since AI is collapsing the distinction between text and speech: speech-to-text and text-to-speech, we are told, are the new frontier of AI use, and there is a good case to be made that this will be the main way for people to engage with LLMs going forward - a Siri that finally works. At which point it’s possible that people’s “true voice”, the one they identify with, will be the one mediated, enhanced by AI.

When that happens, the question for judges will go beyond whether to ban a piece of software from a procedural stage. They will have to confront the procedural assumption that unmediated speech is where truth lives. And, in the process, decide what it actually means to speak for oneself.