AI & Law Stuff
#12 Coaching, Interfaces, and Noise
“abra kadabra” ChatGPT
In popular imagination, advocacy is not the main determinative arc of a trial: no, films, series and books often place the culmination of an adversarial trial at one critical episode: witness cross-examination.
Intuitively, these moments when someone is called to testify and to be challenged on that testimony feel like the point everything could shift, a perfect culmination of everything that came earlier. Hence the TV trope of a witness confessing only when they hit the stand, or revealing key details that turn the trial around.
The legal profession has partly kept pace with this, and making sure that your witness is an asset (or at least, does not become a liability) can sometimes be a large endeavour in itself. That endeavour, in turn, is governed by specific rules: since this testimony could be so important, you want to make sure it remains the witness’ own voice, their way of (mis)putting things, and not serve as a mere puppet for the lawyers running the show.
And so, in general, a key distinction is often made between preparing a witness, and coaching them. The former is meant to be limited to the process of testifying, how to react to some lines of questioning, as opposed to coaching which would be about what to answer. The classic formulation is that you can help a witness tell their story more effectively, but you cannot give them a different story to tell. Different jurisdictions draw that line differently - with the UK for instance known to frown on any kind of preparation, while the US is relatively more permissive - because everyone can immediately recognise that this line is very blurry.
Anyhow, this is 2026, and obviously the coach is ChatGPT … Meta’s Ray-Ban glasses ? Both ? LawyersWeekly reports:
London’s High Court has thrown out a man’s case after it emerged he wore smart glasses connected to his mobile phone for assistance during cross-examination, leading the judge to declare all his evidence “unreliable and untruthful”.
[…]
[Witness] denied using the smart glasses to receive answers during his testimony, insisting they were not connected to his mobile phone and claiming that the voice heard during proceedings was generated by ChatGPT.
The court heard that, before taking the witness box, [witness] made six calls to a contact saved in his phone as “abra kadabra”, which he claimed was a taxi driver, repeatedly insisting that the calls were merely to update the driver on his schedule.
[…]
“In my judgment, from what occurred in court, it is clear that call was made, connected to his smart glasses and continued during his evidence until his mobile phone was removed from him,” Judge Raquel Agnello KC stated.
There are several fascinating aspects of this story, but I want to focus on the fact that the witness, when caught doing something that is obviously forbidden, thought that a good defence was to invoke AI.
This makes sense: at least to me, the story of someone using AI to help them through a rough patch is straightforwardly more sympathetic. People are not equal when it comes to public speaking, and it’s hard to begrudge someone reaching out to AI to find the words that, unfairly, a more articulate witness might have come up with themselves.
In the same vein, readers may remember how, a few months ago, an elderly plaintiff used an AI avatar to plead a case in his stead. While the judges in that case cut short that attempt, I am not sure preventing people from using such crutches is very wise: this means we are satisfied with the baseline inequality of individuals in terms of self-expression.
And then, we saw a few weeks ago that the UK’s Civil Justice Council proposes to make a distinction between AI-generated text by lawyers (permissible, since a human attorney remains responsible) and by witnesses (prohibited, since we want the witness’s own voice). I pointed out at the time that this glides over a polite fiction: in many fields, witnesses don’t write their statements to begin with, or at least they are heavily assisted - we might say “coached” - in doing so.
There is a difference between a lawyer shaping a written statement in advance (which is accepted, if only because that seems hard to police), and someone feeding answers in real time during cross-examination. The first is preparation; the second is ventriloquism. But ventriloquism assumes someone else’s words are being substituted for yours. If the AI is simply helping you articulate what you already think, then what (or who) exactly are we protecting by forbidding it ? The authenticity of the witness’ voice, or the advantage that accrues to those who already have one ?
Don’t get between me and the bot
Many law firms and legal departments are currently struggling with how to deploy AI in practice.
One way to look at it is through the “how not to” lens: most organisations want to move as far away from the “anyone using ChatGPT willy-nilly” scenario. Management in general dislikes variance, improvisation, invisible habits, undocumented know-how, and employee discretion. Hence the role for tools, subscriptions, policies, and some plain old SaaS - even when the raw LLM would be more competent. These allow for a level of control of usage, as well as observability.
In other words, what a lot of law firms are aligning on recently is the idea of building or licensing a secure interface, connecting it to the right models, adding access controls, maybe attaching internal knowledge bases, defining use cases, and creating a common layer through which AI use becomes manageable.
This has obvious advantages: AI deployment promises many things that management wishes for: visibility, consistency, standardisation, auditability, reuse. It offers to make legal work more legible to the institution itself.
This is not unique to the legal field, of course, and this desire for interfaces matches offers of adequate tools from the tech side of the equation. One such offering is LiteLLM, which describes itself as “an open-source library that gives you a single, unified interface to call 100+ LLMs”.
Now, you might have heard that LiteLLM was recently found to be compromised, with malware stealing your secrets and credentials. Oops. This goes back to an issue identified two weeks ago: what you do to make your data and knowledge legible and leverageable also helps with making it extractable and hackable.1
But we can take this further: the alternative to centralised legibility, what a lot of managers have always wanted to do away with (for good reasons from their standpoint !) is a firm’s distributed opacity: many, different minds, many, different habits, much tacit knowledge, little standardisation, plenty of inconsistency. But also - and this is often under-appreciated - some sort of pluralism, local adaptation, friction, and, sometimes, wisdom.
It turns out, then, that opacity may be inefficient, sure, but it can also be resilient. The partner’s instinct, the advocate’s feel for a judge, the paralegal’s odd memory or sense that something is off, the fact that different teams do not think in exactly the same way - all of that is a pain to deal with, a struggle to manage, but it might also help avoid falling into a trap. The efficiencies gained by developing AI through a centralised layer can easily flatten legal knowledge, and make errors scalable, not only work.
The point, then, is not that standardisation is bad. It is that institutions tend to notice its gains before they notice its costs.
And that’s another way the LiteLLM story is interesting, in how it was identified: by an unrelated engineer whose laptop started freezing. Despite not being an expert in this field, Claude allowed him to move from identifying and broadcasting the issue in a little over an hour.2 This showcases both the vulnerability of AI tools and interfaces, and their role in defending against these vulnerabilities, a frame that I predict will surface again and again as we learn to master these tools.
The query, however, is: what will be the equivalent of a laptop freezing in legal arguments increasingly mediated by AI ? Who or what will have the incentive to go beyond the interface ?
The promise of middleware is that everyone will finally work the same way. The danger is that, one day, they will all be wrong the same way too.
Glaze and confusion
Before AI was an issue, the legal profession (or at least, my corner of it) was preoccupied with the issue of bias, both the human kind and (increasingly) the machine one. Then, thanks to the hit book by Daniel Kahneman, Olivier Sibony and Cass Sunstein, Noise (or variance) acquired well-deserved salience in these debates.3
These concerns have not abated, and I was recently on a panel where, among other things, the question arose how bias and noise will evolve in light of AI ?
A common way to introduce the subject is to distinguish bias from noise in terms of directionality. Bias has a direction, at least metaphorically: we know or expect a pro-X adjudicator to favour X and disfavour non-X. Whereas noise, or variance, lacks directionality: the issue is that decisions are all over the place, and you cannot efficiently predict what will happen.
Directionality is important in terms of remedies. You want to fight bias with a counter-bias, which you can readily identify as the opposite of the original bias. And you want to tailor your presentation of facts/the law in ways that do not anchor some bias (i.e., hiring lawyers without telling them on what side they’ll be). Moreover, while bias is usually deplored, it can also be leveraged or used to one’s advantage. (And more generally: we usually call what we don’t like “bias”, and what we think is good “heuristics” - humans are biased in multiple ways, for good and efficient reasons.)
Variance’s lack of directionality, meanwhile, makes such adaptation harder, and most remedies pertain to a form of increased information load and deeper verification abilities: greater collegiality, for instance, or structured decision-making. The idea is to increase the opportunity to “get it right”, or at least closer to an existing average, and to move away from the absolute discretion of one random decision-maker.
Now, two of the main issues with AI nowadays, as we have discussed, are sycophancy and hallucinations. It strikes me that, to some extent, they map very well onto that bias/noise distinction.
Sycophancy is directional: it’s a bias towards you, towards pleasing you. As such, it can amplify existing biases, and needs to be countered by a directional remedy: deliberate prompts to prevent it, second-guessing the LLM answer, changing the context or the approach to tease various answers.
Hallucinations are random-ish: while also meant to please whoever asked for an AI output, the LLM’s probabilistic workings will give you unexpected results. The way to prevent them is, again, with greater information load and verification abilities.
There are other effects,4 and while the analogy is useful, it is not perfect. In particular, human variance in adjudication can sometimes have a productive side: it may reveal that a norm fits facts poorly, that a category is unstable, or that a legal question deserves to be reopened. Hallucinations, by contrast, are essentially fruitless. Though they may help identify bad lawyers, they do not signal a tension in the law so much as a breakdown in the link between output and world.
In other words, the machine age has not moved us beyond bias and noise, but risks industrialising both. Both used to be the idiosyncrasies of decision-makers; with AI, they become properties of infrastructure. That may call for new approaches.
Not to mention the potential for isomorphic mimicry.
This is reminiscent of the XZ utils backdoor, whose story was recently brilliantly retold by Veritasium.
I have written about variance, identifying it in international arbitration, but also making the point that we might want it at the system level (if not at the single case level).
In particular, the flattening we just discussed, or the Artificial HiveMind aspects, might reduce noise at the level of legal advice.