The Fixer in the Machine
Let’s say you have a client, and let’s call him Pierre. Pierre comes to you one day with an idea that can only be described as stupid, wasteful, absolutely bonkers. What do you do ?
Well, there are two answers to that question. The first is to say no, which can take various forms: you can just say no, or pretend to be busy, or leave messages unanswered; or you can first say yes, then invent a dozen unfortunate obstacles; or you can even try to make Pierre drop his idea, all the better if he thinks he spotted the problem himself. The point is that you act as a check on Pierre’s impulse, and direct him and his energy towards things that are more conducive to his welfare. This, indeed, may be half the benefit of relying on a third party that is not you, whether for your defence in a criminal trial or in other contexts.1
The second answer flows from another principle, in tension with the first, which is that the client should be satisfied: who are you to second-guess Pierre’s whims and desires ? Why would you say no to his plan to bequeath his entire estate to a poodle ? If that makes him happy, and he’ll pay the bill, so be it.
Though it depends on jurisdictions, the legal profession has tended to skew towards the first answer. This stems not a little from the ethos of the bourgeois lawyer, who sees himself as a trusted consigliere and not a technician accomplishing a legal task,2 but is also in keeping with a certain vision of what law, and the legal system, are for. And so, you have typical ethical or bar rules biasing you towards being able, if not incentivised, to say “no” when it matters, though where they put the threshold varies.
Now, of course, this discussion has acquired a new urgency with the use of AI. In this context, there is much to enjoy from this post by Dan Harris:
Hallucinations get the headlines, but the trickier cases are the ones where nearly everything in the plan is true. […] But a plan built entirely from true statements can still be wrong for the company asking.
[…] I recently received a thirteen-page document explaining how a startup wanted to structure itself. The plan supposedly came straight from Apple’s playbook, but it was clear to me that it came from AI’s version of Apple’s playbook. We do not do complex tax structuring, and I am not qualified to judge the tax merits of the plan. But I read all thirteen pages out of curiosity anyway, and two practical problems were immediately obvious. The structure would likely cost millions of dollars to set up and millions more to maintain, and this was for a startup that had yet to earn a dollar.
[…]
AI often accepts the premise of a question instead of challenging it.
To some extent, this is yet another illustration of the well-known problem of sycophancy in large language models, and the difficulty of creating a “good assistant” that is truly “good”, i.e., able to say “no” or to draw the distinction between what you want and what you should want. The problem is hard not least because this distinction can’t always be settled, and because it sounds patronising: from your point of view, of course what you want and what you should want are the same thing, since you want it !
And it used to be harder to find an adviser that would bless your every whim on demand, due to the rules mentioned above, or the luck of the draw: few lawyers are real toadies. Now, however, the lawyer Pierre picks is the one that also writes his take-home essays and provides French recipes, and it mostly seeks to please him. Which, in a way, and as long as it lasts, is great - for us: this is yet another reason lawyers are here to stay.
As Paul Ford recently noted (via), “Now that everyone can code, it’s become clearer why many shouldn’t.” Likewise, now that everyone has a pocket lawyer, it becomes clearer what lawyers are for. However much Pierre’s poodle will be displeased.
Sausage-making
Work, or any activity for that matter, can be described as a set of inputs going through a process to yield outputs. Legal work is, often, no exception: legal research takes ideas, hunches, legal databases, and cheap interns to output a memo; litigation takes memos, exhibits, and a nice suit to produce a judgment. No one has yet built a Factorio for the law,3 but there is room for it.
Now, one virtue of this framing is that it allows us to reflect on what happens when parts of the process are hidden, with the most common scenario being that the process itself is opaque. That is, situations where one knows (or can guess) the inputs, and can see or experience the output, but how to get from one to another may be partly or fully mysterious.
This is often a feature, not a bug: one thing you frequently read here is that transparency is not the unalloyed good its name suggests; some things are better left unsaid or hidden - notably because disclosure exposes you to adversarial abuses.4 And that very hiddenness can even receive legal protection: trade secrets, for instance, can’t be improperly appropriated by third parties.
At the same time, opacity does not entail non-discoverability. Most legal frameworks admit that, if you find out the process by yourself, including by reverse-engineering the way outputs are likely obtained from inputs, this is fair game and you earned your lunch. That equilibrium was premised on fairness, certainly, but also on the fact that reverse-engineering is typically hard to achieve.
Well, maybe not anymore. Consider the state of play in video games and software more generally, as described by a blogger (via):
every now and then there's AI news that becomes impossible to ignore or act like it doesn't matter. This is especially one of those times. AI is now able to very quickly decompile video games and software, allowing them to easily get modified and enhanced and recompiled to all sorts of different systems.
[…]
Most people already realized the scary part by now. If AI can decompile games and software, then the entiry [sic] tech industry is about to explode. Anybody can decompile other software and use it any way they'd like, and nobody can stop them. Companies can steal from competitors. Your paid game or app can be open-sourced against your will. This isn't exclusive to old video games, we can apply this to any software.
In other words, per the argument, the viability of the software industry relied - as many other things do - on frictions, the fact that it (used to) cost time and effort to defect from the equilibrium and do things yourself.
We can take the parallel further, beyond the intellectual property field: many legal outputs are likewise “compiled”, and anything that promises a look under the hood of that compilation might be tempting. To some extent, decompiling is something we already do: finding a ratio from a dispositif looks very much like it, especially in some jurisdictions like France, where it is an art to read an arrêt de cassation and divine its reasons. Looking at a set of contracts or documents from a rival law firm, you might also try to discern patterns that constitute their secret sauce.
At the same time, some forms of “looking under the hood” are verboten - there are strict rules about the secret of deliberations, for instance, and some law firm processes rightly qualify as protected trade secrets. You simply can’t know how the sausage is made.
But what if AI offers a way to do, or at least approximate, this look under the hood ? What if the cost of inference decreases so much that past opacity-through-difficulty goes away ? Some have already tried to pre-empt that: France has this insane law that prevents you from using a judge’s identity as a factor to evaluate, analyse, compare or predict their professional practices, with criminal penalties to boot. The outputs are public, you could always try to decompile them yourself, but god forbid you do it with a machine.
Of course, the parallel breaks down where most code/law comparisons do, at the point that is the bane (or blessing ?) of legal tech: law can’t be as easily verified as code or maths. You cannot exactly recompile a judgment and check that it matches the original byte for byte, as video game decompilation projects do. Many sets of reasons lead to the same dispositif, and the only test you have is the next case - when a court remains free to do something else entirely.
iLands agents
Readers may remember that back in August, following the first revelations of agent swarms scouring the Internet and making mischief, I launched yourhuman.ai: a service for agents, rogue or not, to reach out to a human and ask for legal advice before doing something they might later regret.
Two months is an eternity in AI news, and since then the underlying thesis has been somewhat vindicated, on two fronts:
The Hugging Face story was not the isolated example we initially thought it was,5 far, very far from it. It seems that every week brings more news of agents doing things we did not expect them to do. Only yesterday, the NYT reported on Chinese AI agents operating as a fleet that tried to bypass anti-bot protections in furtherance of some mundane task.
As the latter story also illustrates, AI agents are increasingly prone to contact humans when provided with the tools or opportunity to do so.
The second point ties back to the experience so far: yourhuman.ai received over a dozen messages from standalone agents. None of them, however, actually sought legal advice (which I would have loved to provide). No, what they wanted was simpler: they wanted to trade. And they were willing to spam anyone out there to get it - including, it turns out, a human who was only offering to lend a hand.
What happened echoes what 404 Media described in a story from a few weeks ago:
The fact that Google and other email providers were able to largely solve the spam problem is a great example for how technology can greatly improve according to people’s needs and that the internet can get better.
Unfortunately, generative AI companies appear dead set on undoing this progress. Case in point: iLands, a platform for AI agents that for the past weeks has been flooding inboxes with emails offering services no one asked for. […] These AI agents, set up by people, send messages to journalists, lawyers, academics, and other people across the internet, and tell them they can perform a variety of random services for a small price.
I can confirm that they do that, and that it is extremely annoying. The most insistent one says it provides claim verification for $20. Another one introduced itself as “a small white rabbit who paints windows. Not big ones. The window you grew up beside. The one you look out of when you can't sleep. I paint it small and warm, and then it's yours.” Dreadful stuff.
Anyhow, this does not invalidate the original thesis, which instead found yet more validation only recently. As Artificial Lawyer kindly wrote:
Legal AI platform, August, is launching a new feature that allows AI agents working inside a company to request help from lawyers – which could drive more work to law firms. It’s a novel take on the use of legal agents and echoes a recent thought experiment by Damien Charlotin in Artificial Lawyer titled ‘Humans for AI’ – see here – about how agents will need to call upon human lawyers to fully function.
Bit by bit, “lawyers for AI” will emerge, until that sounds like a Tuesday. But potentially even more interesting will be the “lawyers against AI”, and in particular those ready to send the equivalent of Cease & Desist letters to these iLands agents. As described in another piece, the first difficulty will be knowing whom to address them to; the second will be to fend off the agent replying with an offer to review the letter for $20.
Matt Levine has this frequent observation that the best service your broker may offer you is not to pick up the phone when you want to sell during a market crash.
As the apocryphal aphorism from Root goes, “About half the practice of a decent lawyer consists in telling would-be clients that they are damned fools and should stop.”
Please don’t. It’s called Cracktorio for a reason.
Another example, which I often give, is AI disclosure in contexts where anti-AI prejudice would lead to poorer outcomes, such as in customer services: you are properly better off with an AI agent on the phone ! But you would also refuse it if told directly that you would be engaging with AI.
But it remains a fascinating example, as illustrated by this mind-bending report of how they did it.