AI & Law Stuff

#25 Instant pleadings, ordinary meaning, and mastering the file

No time for AI

“Justice delayed is justice denied”, generations of law students have learned, without realising that the opposite extreme – instantaneous justice – is rarely acceptable either.

This is because time, and its management, has a complicated relationship with litigation. Different timeframes are colliding – party time, procedural time, adjudicator’s time, and the normal world clock – in ways that are sometimes fruitful, sometimes detrimental, but always in tension. The timetable should leave enough room for the parties to participate fully in the case, but not so much that evidence would go stale or missing. It should offer opportunities for settlement, but not at the cost of the dispute festering across generations. It should allow sequencing and reciprocity, without offering avenues for undue chicanery. It should help manage dockets, but be mindful of due process.

Many legal institutions, from procedural rules to the concepts of limitation and prescription, remedies such as interim relief, and notions such as ex tunc and ex nunc are premised on a lawsuit inscribing itself in some timeframe. As well, parties know that time is a variable they can play with: cue the classic “Italian torpedo” tactic of filing a lawsuit in a notoriously slow forum to benefit from lis pendens. And since the supply of justice cannot meet the demand, procedural delays serve a rationing function - a friction we already discussed.

As the opening saying indicates, most of the difficulty in history – and still today in many jurisdictions – is with delay. In some cases, you have to wait in front of the Doorkeeper for years, “jetzt aber nicht”, or expect your children or their children to see a conclusion. Hence a perennial concern (or the evergreen jeremiads) about “reasonable time” in justice; Wikipedia informs me that the Babylonian Talmud already professed that “When justice sleeps, justice is canceled.” Hands are wrung and schemes are hatched in many chancelleries across the world to speed access to justice, often to no avail.

But now AI is capable of profoundly changing this relationship to time, in ways that are often ambiguous. As mentioned, instantaneous justice is rarely desirable, at least from the side of the losing party. A common objection to robot judges is indeed to insist on the importance of human reflection upon a matter, and reflection requires time.

These issues are becoming increasingly important. A good example is provided by a recent reconsideration judgment from the UK Employment Tribunal in Abraham v Hound Technology (via). A rather bewildered Judge Tynan noted that:

Within minutes of the hearing concluding the Claimant submitted an application for reconsideration, accompanied by a four-page statement of grounds in support of the application. Approximately 10 minutes later the Claimant submitted amended grounds in support of the application, to which had been added a further, fifth ground in support of the application.

After the respondent raised the (rather obvious) question of how the application had been produced so quickly, the claimant confirmed that she had used AI to smooth her documents, in particular given her dyslexia, but argued that the content was still her own. Judge Tynan declined to investigate further, but observed that the application partly misunderstood his reasons and that the claimant “might perhaps have awaited the written reasons before deciding upon her next steps”. The reconsideration request was then refused.

That reticence to go further is not surprising: procedural rules are mostly concerned with time ceilings, not floors, for the reason that the latter have rarely been necessary. As a result, the law has an enormous doctrinal vocabulary for parties who are too slow (limitation, laches, default, abandonment, want of prosecution) and essentially nothing for parties who are too fast.

In the Abraham case, Judge Tynan observed that “one of our primary concerns as judges is to ensure that AI generated documents reflect rather than suggest a party’s case and evidence” (my emphasis). The point made is that the system assumes that a litigant is the author of its own legal grounds, and that those were not adopted opportunistically at the suggestion of a wily advisor.

Yet, that distinction has never held water: lawyers routinely supply parties with arguments they would never have formulated themselves; in many systems where appeal is as of right, kitchen-sink approaches are patiently tolerated. The equilibrium held because the system assumed that cases and arguments had passed through some human bottleneck of comprehension, selection and adoption.

This is, in other words, the “proof of work” nature of legal submissions: if you took the time and effort to put them together, we could assume they were worth adjudicating. A long, structured submission or judgment ordinarily signals sustained consideration, selection and effort. When the same artefact can materialise in minutes, the signal breaks - and, maybe, the system with it.

The LLM Would Like a Word

One of the first encounters of the legal profession with Large Language Models was the dog-sitting-in-a-car criminal case that somehow landed with the District of Columbia Court of Appeals: in Ross v. United States, the key question was whether leaving a German shepherd in a car when the temperature climbed had qualified as animal cruelty. And in ascertaining what “common sense” held in this instance,¹ the majority and dissent fought over the relevance of ChatGPT answers.

The appeal is rather obvious, and intuitive: trained on countless human writings, and thus likely privy to a large part of human thought (the expressible part, at least), LLMs seem perfectly suited to land on what an average human would think of a given situation. There is, in fact, a large literature showing that, properly prompted, models can be used to model humans psychologically and replicate most of what humans would do in given situations; AI-based simulations are a booming business. Which is why one possible use of LLMs in particular is to help ascertain common sense or the communis opinio populi.

Closely associated with “common sense”, and even more potent in the legal domain is the use of such language models to make sense of words and elicit their “ordinary meaning”. That standard is ubiquitous in legal systems, and harks back to the common-sensical notion that simplicity is unique, while complexities are limitless: one can always read more in a word or text, given sufficient imagination, and the buck has to stop somewhere.

This was the subject of an earlier appeal court opinion, from Judge Newsom in Snell v. United Specialty Insurance Company. Noting – as one must eventually acknowledge – that “ordinary meaning” is not always that easy to derive, and that to do so based on dictionaries is often nothing more than sophisticated cherry-picking, Judge Newsom suggested a possible “heresy”: using LLMs to help in this endeavour. As he noted:

ordinary-meaning interpretation aims to capture how normal people use language in their everyday lives—and the bulk of the LLMs’ training data seem to reflect exactly that.

Others have disagreed with this stance, and the best charge against this approach can be found in a long article by James Grimmelman, Benjamin L.W. Sobel and David Stein in the Harvard Journal on Legislation, entitled “Generative Misinterpretation”. The point made is that, even if one put aside the brittleness and unreliability of LLMs, a distinction needs to be made between the process of legal interpretation and the artefacts (i.e., a decision) resulting from it. But beyond this, they point out that the acceptability of LLMs as oracles of common sense will not come easily, comparing the use of LLMs in legal interpretation to two still-controversial methods, corpus linguistics and trademark surveys.

Coincidentally, one of the latest decisions in this vein arose in a trademark context. The EU Intellectual Property Office recently opted to cancel a trademark – UGRUZINA – on the basis that it was purely descriptive (it means “at a Georgian’s” in Polish). In defending, the right-holder argued that, according to ChatGPT, the trademark at stake had no standard meaning in a “widely recognized language” (nevermind the 40+ million Polish speakers). Which was, in other words, a use of ChatGPT as a trademark survey, designed to prove not the meaning of a word, but that such a meaning did not exist.

That the Office ignored that argument is interesting: ultimately, beyond the (fascinating) theoretical and epistemological debates about what LLMs can and cannot do, this type of argument is just evidence like any other, evidence that can be weighed and discarded if needed. Not to go full legal realist here, but common sense and ordinary meaning have long been Procrustean notions deployed in line with the decision-maker’s wishes.

These notions recently found a new tool to get there, and it will be accepted or discarded, depending on whether it gets the decision-maker where they were already going. One risk, however, is that the democratisation of this kind of cherry-picking eventually kills it: if “ordinary meaning” is simply a question of precise prompting, the notion may become meaningless.

Agent understanding

Lawyers usually take it as a point of pride that their profession is old. You have got rituals, oaths, and professional organisations counting the many passing years since their foundation. You learn Latin, or Shakespeare, and you see lawyers doing lawyerly things, and intuit that the moral arc of history bends towards more norms and more lawyers, the vindication of right over might, etc.

But that seniority also means that the law may be unfit for a world that has changed around it. I suspect this is true in at least one critical way: while the volume of legal material, arguments, norms, and matters has grown substantially over the past few decades (and keeps growing, of course), the profession still to a large extent relies on the figure of the lone jurist who “masters the file”, or who “knows the law” on the tip of their fingers. Likewise, whereas specialisation is ever-more important to keep track of anything, cultural representations of lawyers still focus on the heroic generalist, able to help you with your divorce on Monday, handle your bankruptcy the next day, and get you out of jail before the weekend (tough week).

Still, behind the appearances and norms, the underlying practice has evolved and met that challenge largely through ever greater degrees of delegation: partners have counsel, who have senior associates, who staff junior associates, who cross their fingers that summer interns won’t load paralegals with the work, just as judges and arbitrators trust their clerks and assistants not to botch a first (and often final) draft. Someone remains responsible for the output at the end (thankfully), but that someone needs to ensure that the chain of delegation is robust and not prone to bring unfortunate surprises.

Now, to work properly that delegation should follow, if not specific rules (which some bars offer), then at least some broad principles in terms of scope (some things cannot be delegated) and in terms of the respective roles of delegator and delegee. One such broad principle echoes the original principle of staying “master of the file”, that is, being able to understand what is going on, what the key elements of a legal matter or question are, and defend whatever legal position your team – that is, you – have reached on this point.

Crucially, such understanding does not require knowing everything, or even doing it by yourself. Certainly, doing everything yourself is often the best way to understand and be “master of the file” – no shortcut, and the hard work of putting words to a page and brawling with an argument is a sure way to get what is going on. But it is not always practicable, and comes with its own set of challenges (i.e., your time is finite). It’s only a means to the end of understanding the matter at hand, and there are other ways to elicit that understanding.

You can simply – and many lawyers have for years acted that way – be prepped on the main by your delegees: you run a risk of being unaware of some key details, but that is part of a cost-benefit calculus taking into account your limited time and attention.

Now, the rise of AI agents, particularly in coding, casts a newly important light on how and what to delegate. In this vein, a recent lecture by Geoffrey Litt suggests that “understanding is the bottleneck” in agentic coding (X thread here). Not merely understanding in order to check or verify: LLMs are becoming ever better at checking their code. But understanding in order to participate in whatever is being done. Because, as Geoffrey puts it, referring to a coding agent’s task as a loop:

It’s never just one loop! A project is many, many loops with the agent.
And the understanding you have of the system is part of your ability to come up with the next idea to evolve it.
You need a rich set of concepts in your mind to think creatively and fluently about how to move something forward. If you’re lacking that fluency, your ability to participate in the project is meaningfully limited.

In the same vein, a lawyer that signs on something, or even verify it, but cannot generate the next question or strategic move is no longer master of anything. The labor of editing a junior’s draft is precisely where a partner or judge builds the residual semantic memory needed to handle a hostile hypothetical during oral argument.

From there, Geoffrey suggests several ways to enhance understanding from agents to the human(s) working with them,² and what fascinates me is that law developed understanding-transmission techniques centuries before agentic coding needed them; entire legal genres exist precisely to move understanding up and down a delegation chain, and to create the cognitive scaffolding that helps make delegation manageable:

• Explainer docs: any legal brief, or a bench memo that pre-digests a record. This puts a finger on the fact that much legal material consists in hypertext: a cover email explaining the tracked changes, a memo setting out the stakes in a brief.

• Quizzes: the partner grilling the associate before the hearing, but also oral argument itself, which is structurally a quiz the court administers to verify that counsel understands their own brief.

• Micro-worlds/simulations: the hypothetical. Law schools and appellate judges have been running adversarial simulations on legal positions (“counsel, suppose the truck were blue”) for about as long as the common law has existed.

• Shared spaces: the file, the record, the dossier, as the context where further questions can be answered.

Legal documents are rarely about the world – they are about other documents, and designed in ways that are meant to foster that understanding between the people involved. The danger is that they start working as substitutes for understanding, thereby depriving their user of what they need to decide what the next output should be.

1

Although, of note, Judge Deahl in a footnote discounted the idea that ChatGPT could represent “common sense”.

2

Understanding goes both ways: see all those AGENTS.md and other skills files, the system prompts, the carefully curated contexts memos written by humans for their delegees. In this respect, an unappreciated difficulty of AI, which may contribute to the slowness of its adoption, is that managers have long relied on juniors who, mark-ups after mark-ups, eventually intuited what the manager likes and dislikes. That won’t work with AI agents, for whom you need to spell out these preferences explicitly.