AI & Law Stuff
# 21 Subjectivity, the virtue of Slop, and non-AI lawyerly mishaps
Name on the Page
Why do lawyers, or anyone for that matter, take such care over their writing ?
A naive answer is that writing is meant to convey information: across people (when one writes for someone else), across time (e.g., when taking notes), or within oneself (writing to think and reason).
But this answer fails to account for a large part of the act of writing, which is that there is (though not always) a reader on the other side. And that reader is rarely there chiefly, let alone solely, for the information that’s being conveyed. Rather, the reader expects to link a text to its author.
Sasha Chapin, some years ago, penned a nice piece pointing out - among other excellent points - that no one reads books purely for their content; books, he held:
[…] aren’t information transfer devices, they are subjectivity-merging devices.
When you read a book, you aren’t just accessing a series of propositions. You are becoming immersed in a worldview, through the rhythms of a given prose style, the facts selected and omitted, and the author’s chosen self-disclosures. Reading is hypnosis. Just like hypnotists lead you into a trance in which a simple suggestion can become forceful, skillful writers, by absorbing you in their pages, give you a perspective, from which certain selected facts take on greater relevance.
Likewise, legal writing is not authorless writing. Separating drafting from signing, and delegating writing in general, meets a limit in the fact that the person signing is not random: it’s the person ready to take responsibility, a responsibility rooted in their authorship, and authoritativeness (no surprise these concepts share the same root). Any half-competent student could write an expert opinion on the law, and on the cheap too - and yet legal experts command impressive prices because their name is on the paper.
This is something that can be easily obscured when there is so much text flowing around us, and when AI makes it so easy to read intelligent-looking takes on expert subjects.
And so, this recent study finding that law professors prefer AI writing on expert topics over their colleagues’ - and, amusingly, flag their colleagues’ answers as harmful to students more than three times as often as the AI’s - is both interesting and wholly immaterial.
It is interesting even though it matches and echoes many similar studies, in countless different fields. We know that people prefer poetry, conversation, therapy, medical advice, agony aunts, even internet memes from AI, at least when they don’t know an LLM is behind it. Hell, even my own research in the legal field demonstrated that AI can compete at a very high level in moot court competitions, largely because their outputs are better-written and formatted than your average law sutdent’s. In other words, there are few areas of interpersonal encounter,1 where people won’t, at least in the lab and against a veil of ignorance, prefer AI over human outputs.
But, I suspect, this makes little difference, at least when it comes to legal advice. Note that, following good empirical practice, the authors of study stripped the name of their authors from the human texts. The law professors were asked which answer they preferred, not which colleague they would stake their name beside or reward with a nod. Yet this is often what matters: knowing that one peer is staking their reputation on a position, is making a judgment call, or offering a warranty.
We are looking for their subjectivity, because that subjectivity is what can persuade us or keep us in line. Strip that out, leave only the information, and maybe the AI has an edge - but few are the contexts where information is all we need.
Flooding the zone
A common theme around here is that the cheapness of text portends the demise of any system that relied on text-generation as proof-of-work or proof-of-stake. The “gym membership” model of the law, I called it: it works only because very few people actually assert their rights, but the same is true of quasi-adjudicative processes, such as HR or grievance mechanisms.
This has long gone beyond anecdotes and judicial gripes;2 recent graphs, widely shared over social media, highlight the extent of what’s happening in distinct fields.
Put otherwise, AI allows people to put an increasing number of things out there, but it’s unclear whether these things are worth putting out to begin with. Which could be the abstract definition of “slop”.
Relatedly, Jed Meers has a forthcoming piece in the Journal of Law and Society that relies on a survey of 500+ UK civil servants to find:
the use of AI tools by the public is increasingly affecting the front-line of the administrative state. As one participant put it, ‘we’re getting flooded with emails and letters that are clearly just people chucking their stuff into ChatGPT and hitting send.’
With some excellent anecdotes:3
A local official describing the tonal whiplash : “their last email was all ‘pls help my bins ain’t been emptied in 3 weeks FFS’ and now it’s a five-paragraph essay with perfect punctuation.”
An asylum caseworker whose scheduled removal failed because 100 pages of AI-generated, weak and irrelevant submissions landed at 10pm the night before, and the decision-maker couldn’t read them in the hours available. Slop as a successful denial-of-service attack on a deadline-bound process.
From this, Meers identifies “three kinds of disruption caused by AI slop raised in the dataset: disruption to signals from correspondence, to trust between officials and the public, and to systems designed for a pre-AI slop world.” The paper points out that the systems being challenged by AI will need to be redesigned to manage this onslaught, though Meers, to his credit, adds that the drivers of public AI use need addressing too, and even allows that slop may widen access to justice.
One frequent comment in the same vein - one I made several times - is that this new equilibrium will call for additional frictions. But it’s worth questioning what goal the frictions are meant to achieve: if it’s only for an existing system to persist without deep-rooted reforms, or even a look at the system’s underlying reasons, they should be deplored. In some areas of the administrative state, barriers have always been there to constrain demand; but it’s always awkward to point out that we need to save access to some goods by restricting it to some extent.
And indeed, in a draft essay I have yet to complete, I suggest this is one potential reason why “slop is good, actually”: cheap content at scale does not create the rationing of the old systems, but makes that rationing explicit, revealing that many things once labelled as free have, in fact, borne a price of some sort.4
And in doing so, slop sheds light on many hypocrisies modern states like to maintain. The hypocrisy that a service or benefit is universal (or even necessary in the first place) but hidden behind forms and small print; the hypocrisy that we read and learn and benefit from our peers’ written output (on social media overtaken by slop); the hypocrisy that public consultations are read or serve a purpose; or the hypocrisy that the right to be heard ever implied that someone was listening.
Some of these fictions might be utile; but we’ll soon find out which ones we can or should keep up.
Zealous advocacy
A key motif in the hallucinations database, I keep repeating, is that it surfaces some regrettable practices in the lawyers at stake. This is a profession that is reliably pressed by time and resources to settle for “good enough”, with varying notions of what this looks like.
In the litigation sphere in particular, the incentives to be aggressive are abundant - and sometimes stretch towards crossing some lines. These incentives are liable to scale together with the stakes of a given case, be it only because multiplying the people involved can lead to some de-responsibilisation, or a temporary effet de corps that leads one to accept or turn a blind eye to practices that one would not engage in otherwise.
And so, hallucinations and the misuse of AI, when caught, are, ultimately, often a mere symptom of some kind of pre-existing organisational dysfunction, possibly coupled with recklessness and a certain attitude towards competence by counsel.
From time to time, some stories tell us that there are other such symptoms, beyond the misuse of AI. Bloomberg Law recently reported on a rather edifying story :
Quinn Emanuel’s expert obtained non-public clinical-trial data months before the study results were publicly released, describing the information as confidential and embargoed.
Rather than seriously investigating how the expert had obtained the data, the team converted the material into a supplemental expert report.
Upon opposing counsel’s challenging the report as untimely, the court agreed to reopen discovery and vacate the trial date.
Internal communications (obtained through opposing counsel’s dogged insistence, excellent lawyering here) later established what had happened and who got the data when.
The firm then relied on thin distinctions: the expert had not shared the “abstract” of the relevant study itself, only the data from it; nobody had technically lied; and any ambiguity could be attributed to phrasing rather than candour.
The court found this deliberately misleading; counsel were fined nearly 3 million USD (you read that right), and ordered to complete some CLE on law and ethics.
Now, this is not an indictment of the modern practice of law; the vast majority of lawyers seek to act responsibly, a course of action that’s not always obvious. And even the behaviour in the example above could, taking a broad view, be excused by some circumstances, or at least be understandable.5 One should always be careful before drawing conclusions, or moralising too cheaply from the comfort of hindsight.
Because that’s the point: large parts of the legal domain are error-prone by design. The materials are too numerous, the facts too messy, the incentives too sharp, the deadlines too close, the other side too imaginative, and the distinction between “zealous advocacy” and “lack of candour” often revealed after the fact. Add AI to that mix, and you do not create the problem from scratch; you add a new failure mode to a system already full of them.
And so, that’s my bet: years from now, we will still have sanctions, CLE orders, insurer exclusions, and embarrassing headlines. One opportunity of AI is that it prompts everyone to review whether their practices are robust, and change accordingly. But I won’t hold my breath.
In a recent French administrative ruling, a self-represented litigant was fined a couple hundred euros for filing a brief that was not only filled with hallucinations but also - horresco referens - over 300 pages long, and in support of a doomed argument. You can almost hear the sigh.
As well as this great quote from anthropologist Matthew S. Hull: someone “who is an agent but not an author, who causes things to happen without writing (or being written about), is a kind of witch from the bureaucratic point of view.” Excellent stuff.
The classic joke: in the US, the doctor asks for your insurance; in the UK, for your patience; in Canada, whether you have considered assisted dying.
This has a close mirror in software development and some similar takeaways.
My company has decided to largely forego hand-written code and focus completely on agentic coding agents for now. We can now generate far more code than we understand, yet we are still responsible for it - the same shape as signing a brief you didn't write.
Law has Rule 11. The signature is a formal, named act where vouching is the assumption of accountability, and it's why every "the AI hallucinated the citation" defense has failed - you signed, you own it. Software has no equivalent. We can write tests to validate what the code does, but a test only checks what we already thought to check — and the understanding we've lost is exactly the edge cases we no longer think to test. Consequences spread across a team, sometimes several, until they diffuse up to the company itself.
The danger isn't volume. It's that commissioning thins felt ownership everywhere while accountable ownership doesn't move an inch. You can stay 100% accountable for code you've lost the capacity to understand - which is the one thing a signature exists to prevent.