AI & Law Stuff
#24 Mock-ups, Footnotes, and Insurers
Ready to vibebrief ?
How do you compete against other lawyers ?
Say, they have a client you’d like to work for (and earn from), what are your options ? There is, of course, the basic social approach of wining and dining, the person-based courtship that may at least get your foot in the door. But then, if that does not suffice, how can you prove your merits and compare them with those of your esteemed professional colleagues ?
Several things stand in the way. For one, you will rarely be privy to the exact kind of things the client wants from their lawyers: matters are confidential, privileged even, and there is no readily available comparator to measure against. Moreover, and as we discussed a few weeks ago, legal competence is hardly legible: knowing a good from a bad legal product requires skills, and in most cases will never be tested until things go wrong.
This leaves costs and price competition, but in an inefficient, intuitu personae-based market, that may not drive you very far. In fact, it’s even possible that lower costs are the wrong signal to send: competent advice has a value, and cutting prices may simply devalue your work and assist your competitors: if they are so expensive, they must be worth it. “Reassuringly expensive” thinks the client hiring a white-shoe firm to impress the other side.
Still, one thing you can do is to alert a potential client that your competitor is, precisely, not providing this value for money, and that whatever they are doing could be cheaply replicated, freeing resources for deeper work. Because if there is one thing clients hate, it’s being scammed and paying too much for shoddy work delegated to juniors; hence the many fights over how much detail timesheets should reveal about who did what.
In this respect, many lawyers might eventually take their cue from the software industry; the Financial Times recently reported on what the competition looks like there, courtesy of generative AI:
Private equity investors are turning to AI-generated replicas of software to assess whether acquisition targets have a competitive advantage, as generative AI threatens to upend the industry.
Bain & Company, one of the world’s leading advisers on dealmaking, is “vibecoding” — using prompts and AI to write code — to rapidly recreate pieces of target companies’ software.
The mock-ups let potential buyers test how difficult the technology would be to reproduce as the cost of building software is rapidly falling, and also understand how the product could evolve.
Likewise, you could see law firms or other players coming to you with a perfectly-formatted, deeply-researched-looking memo cooked by AI on the one case you just lost, for you to compare with whatever wet brick your legal team had put together in vain. Or you may receive an AI-generated assessment of the settlement your counsel recommended, suggesting you committed the worst possible sin, leaving money on the table. The mock-up need not be superior, or even correct, but only plausible enough to question the incumbent lawyer’s account of why the work was difficult, expensive, or unsuccessful.
The limits highlighted above remain potent, however, in that you don’t have access to the client’s private considerations, the why and how of the position they adopted, or even their preferences in terms of legal posture.
But this is why the proper parallel might not be law firms relying on AI to accelerate that competition-by-comparison. Indeed, when teaching on the future business model of law firms, it is common to note that actual competition might not come from these quarters, but from the clients themselves. Also in the FT, Brian Tang of Hong Kong U. notes:
“It’s one of the first times that the same technologies [are] being offered both to law firms and their clients,” Tang says. “Imagine what’s happening. The [law firm’s] client now says, ‘Hey, I have Harvey too, I can do this myself. So, what am I using you for?’ How does the relationship change?”
One way it has changed, I predict, is that now your work will be compared to whatever ChatGPT can spit out in half a minute.
Too many notes
When teaching natural language processing (NLP), there is an easy crowd-pleaser: stylometry, the art (or rather, the science) of attributing writings to a particular human, or of deriving some characteristics (age, gender, origins) from a few hundred words. The methods have been there for some years now, made a bang when (un)settling the attribution of the Federalist Papers (or, more recently, when trying to unmask Bitcoin’s creator), and even entered the legal field in some respect.1 Yet, most people still remain oblivious to the reality that, under the right conditions, a few hundred words in their hand (it also works with code) is able to assign (or predict with reasonable certainty) an identity.
I haven’t seen so far, however, anyone suggesting that how one handles footnotes is as predictive of personality and identity as main text, but I would not be surprised if that is the case. Footnotes, and what we put in there, vary a lot across people, and there are distinct religions in this respect, from the pure atheist (example below) to various shades of agnosticism (dry sourcing only), all the way to the real zealots of footnotes as a way to convey their wettest musings. Done well enough, proper footnoting can be appreciated as any other craft.2
And that’s because footnotes play a lot of roles in writing, from acknowledging sources to providing asides, allowing for degrees of hedging and the construction of a hierarchy among arguments, a showcase of erudition or of an artistic temperament. All these roles and many others are recounted in a wonderful short book by Anthony Grafton on the history of the footnote,3 explaining that they are part of the epistemic apparatus of scholarship, as a way to deal with authorities and to prove/disprove arguments.4
This being mentioned because of a recent bout of debate about the adequacy of this eminently valuable practice5 of footnoting legal thoughts. Reuters reports:
Chief U.S. District Judge James Boasberg of Washington, D.C.’s federal court, who drew national attention this year for clashes with the Trump administration over deportation flights to El Salvador, told me he has not used “a single footnote for many years.”
Last week, after striking a 45-page filing because it included 18 footnotes, Boasberg agreed to an interview to discuss his antipathy toward the legal profession’s favorite below-the-line habit.
There is no need to go into the (uninspiring) reasons that particular judge does not like footnotes (mostly that they smuggle extra pages past the page limits, and that anything worth saying should be in main text), but, this being a blog about law and AI, I’d suggest a further reason to enjoy footnotes is that, for now at least, those are more likely to be a properly human approach to text composition, as opposed to dry briefs rambling word after word, or token after token.
And indeed, LLMs are (so far, or by default) no great footnoters. They can do simple asides — hence the famous em-dashes that often give AI writing away — and they can increasingly source ideas and citations in ways that, little by little, should manage to minimise hallucinations in due time (or at least a lot of brain power is being spent on this particular issue).
But unless specifically prompted to do it, and even then, AI models do not enclose their thoughts or considerations in footnotes or important parentheticals. This is a hunch, and I have no data to back this, although I could adduce some tempting technical tenets (the token-predictability lessens the chance of true asides in a given text, training data was likely stripped of footnotes, especially from .pdfs, etc.), but more generally the point is that LLMs are not great at performing the judgment that a footnote encodes.
Since this is key: knowing when to footnote and what to put into it is, perhaps even more than the main text, a subjective act of will. A decision that a proposition needs support but not interruption; that a caveat must be preserved but hidden from (immediate) view; that a counter-authority should be disclosed without derailing the argument; or that a genealogical and pedantic aside deserves its place on the page.
Such editorial decisions remain, for now, beyond the reach of the models. Whether they are beyond the reach of most lawyers is a separate question.
Name on the Page
One virtue of lawyers, I wrote, is that they can serve as a shoulder to cry on. You build a relationship of trust with your favourite legal professional, and that trust means that you rely on them to steer you towards the good legal outcomes, avoid the bad ones, and eventually explain why you were cooked anyway (but how they spared you the worst).
In other words, lawyers play the role of some kind of insurance policy, putting your mind to rest when it comes to legal troubles. “The lawyers will take care of it” does not only denote delegation in a given situation, but also a cope and a hope.
That intuition sheds light on many aspects of the legal market, from its lack of competition (any name big enough will do) to why clients over-buy seniority on matters that plainly don’t need it. Heck, the whole concept of a “comfort letter” from a law firm gives away the game.
Some say it more or less explicitly. In a well-made piece about how the profession is reinventing itself because of AI, Bloomberg Law puts the spotlight on what is becoming the bugbear of most lawyers nowadays: receiving AI outputs to sign off on. But while the practice is often deplored, it fits what has long been the unspoken assumption that lawyers are there to sanction something already decided.
Clients are even hiring law firms just to check work done by AI, says Alexander Behrens, who focuses on AI issues at Allen & Overy Shearman and heads the firm’s financial regulatory practice in Germany. That, he says, puts lawyers in a similar position to insurance companies.
But then, should law get closer in shape to the insurance industry ? In some ways the parallel already exists, but in others there is still some ground to cover:
Underwriting. Lawyers price risk every time they decide how diligent they should be in working a matter. However, too often they price it by feel, and bill it by the hour (re-adjusted in line with feel), not as a premium against a modelled probability of loss calculated by true actuaries.
The pooling of risk. Insurance as an industry requires risks to be independent: not every boat sinks on the same journey. Lawyer-insurance could stay informal for the same reason, each matter could go wrong in its own way. To some extent, AI breaks the independence: the same model, the same blind spot, the same argument collapse, runs through every brief at once, which is a single risk wearing hundreds of legal memos.
Reinsurance. Correlated catastrophe risk is precisely what no single balance sheet can hold, which is how you get reinsurers, someone to stand behind the firm that stands behind the AI. Malpractice cover already exists and play that role for lawyers, of course, but might get greater salience in a world where assurance takes centre stage.
The names. As put by Jordan Furlong, in a scenario where “accountability” becomes the key advantage of lawyers, one may expect “an eventual migration of competitive advantage from lawyering excellence to institutional credibility — and to the strength of the associated brand”.6
Regulation. Lloyd’s began as underwriters scribbling their names under a risk on a shared slip at a coffee house, and formalised only once the volume and the stakes demanded capital and rules behind the signatures. With legal AI, volume and stakes are growing, and soon the name on the page may need a balance sheet behind it.7
But run the analogy too far, and it may land on a risk: one possible direction for how law firms will reinvent themselves, indeed, is that the product is bearing risks rather than analysis. If that is the case, rational law firms will seek to optimise the premium, not the accuracy. Let’s hope AI gets the latter right, then.
For instance, in alleging that a given arbitrator had not written an award, as happened in a famous case about a 50 billion USD award alleged to have been the work product of the tribunal secretary.
Think what you will of Infinite Jest, but the chapter-long footnotes are memorable.
And amongst the other savant treatments of the subject I am rather partial to the (recently Pantheonised) French historian Marc Bloch calling proper sourcing in footnotes a “morale de l’intelligence”, in his Apologie pour l’histoire ou métier d’historien (1941), available here.
You can guess my stance on that debate.
The link also lists certain (good) reasons to doubt, or at least limit, the “lawyers as insurance” story.
A further parallel can be found in how both Lloyd’s and the Inns of court started as trusts.