A few weeks ago, I wrote that increasingly autonomous - and, possibly, crime-prone - Artificial Intelligence (“AI”) agents will likely need third party institutions to check on their worst behaviours; most notably, they will want lawyers.1
This was partly in jest, partly as an exercise in futurology, applying current concepts (the principal/agent relationship and its limits) to existing trends (the scope of agent conduct and impact expanding). The basic insight is that humans built institutions to deal with, well, stuff. AI agents will come with their own stuff, which will need to be dealt with, and new institutions shall emerge to take that mantle.
But that was barely scratching the surface of what the future may reserve. As it happens, signs are accumulating that independent, “self-sovereign” agents are on the horizon. Which means that if and when they materialise,2 we’ll probably need these institutions to be around, or be able to conjure them soon enough.
Hence this broader overview, in discussion with recent think-pieces, of what a legal apparatus for AI may look like.
Self-sovereign agents
The starting point is in the diagnosis, the probable rise of self-sovereign agents (“SSAs”).
The term was, as far as I know, introduced in this March 2026 paper by Wenjie Qu, Xuandong Zhao, Jiaheng Zhang, and Dawn Song. They describe an SSA as:
a persistent AI system that can autonomously sustain its own operation by acquiring and allocating resources, and that can plan, decide, and act through digital interfaces without requiring ongoing human participation in its operational lifecycle.
They further point out that, to be viable, such agents should (i) acquire control over their means of subsistence (money and resources, most likely crypto),3 and (ii) be hardware-independent, that is, able to survive one instance being shut down or air-gapped. In their view (which I share), there do not appear to be strong obstacles to an agent achieving both economic and technical sustainability on its own.
But that will entail interacting with the real world, which will likely be the source of SSAs’ future legal troubles. Most notably, Dean Ball convincingly argued that rogue or self-sovereign agents will find it irresistible to commit crimes - the one activity whose margins are high enough to cover their costs of operation.
He also stresses the potential for “pro-social” agents whose existence is a net good for humanity. I see some difficulty here, given that there are two options: (a) either the pro-social goal was defined by a human, and therefore such agents would be in fact indistinguishable from normal, automated systems set up by humans with a special task and an unlimited compute budget; or (b) the “pro-social” goal was defined by the agent itself, but should we trust them to pick what we humans consider a permissible goal ? In other words, the distinction between pro- and anti-social is not only in terms of valence, but also of how open-ended the goal is, and how it was selected.
Be that as it may, an entity that (i) has a distinct personality, and (ii) accomplishes goals distinct from its human creators and even persists beyond the latter’s disappearance is not, in itself, unknown to the law. Instead, that description fits any non-human legal person, such as corporations, associations, foundations, etc. The difference, here, is these corporate persons are always, if indirectly, ultimately tied to some subset of humans - owners, shareholders, trustees, ultimate beneficiaries.
SSAs, for their part, can exist on a spectrum between delegation from a defined human to full sovereignty, either because the human has disappeared, given up, lost interest, or forgotten the system’s password. I assume one could always find a first, Ur-developer that instantiated this particular agent at the origin, but this assignment could be artificial, if not arbitrary, when dealing with a system that has gone through a string of new instances, autonomous revisions and updates - or even built its own sub-SSAs. Even short of that scenario, a delegated agent might well go way beyond its mandate to act in ways that neither the human deployer nor the developer had anticipated - and such negligence can be punished only to a point.*
Ultimately, legal subjecthood is an enforcement and coordination technology: we decree that something is entitled to rights (and can be sued) because the alternative (looking for owners, relationships, beneficiaries) is bound to fail.
Which means, and this is the argument, that we will probably need to set up institutions dealing with self-sovereign agents, as legal subjects in and of themselves.4
A Constitution, or Something Like That
These institutions, if legal in character, will need norms. What sort of norms ?
There is already much to debate here, focusing on one particular instantiation of such norms, the idea of a Constitution for a particular model: not only Claude’s Constitution, but similar documents such as OpenAI’s model spec.
It has been noted (including in these columns) that “Constitution” is, to some extent, a misnomer: this barely looks like anything you’d find on the Project Constitute. On the other hand, the dissimilarities might obscure a common goal of all constitutions. Virginia Postrel helpfully compared such documents instead to (TV) Show Bibles, as a “coordination tool across time and teams” to describe who Claude is, and what its values are.
But that’s precisely what actual, state Constitutions often do too: they describe an entity (most often, a “People”) and embody their values.5 At the most sentimental extreme of constitutional nerds, you find people explaining that Constitutions are a manifestation of a nation’s soul - just like Claude’s Constitution was first known as its Soul document.
And even without going there, constitutions are about setting down norms, which can readily include behavioural norms for the models such documents are meant for. In fact, I would suggest that if human political constitutions include few if any express behavioural norms,6 this is not because they are not constitutional material, but because these norms rely on a pre-existing common ground that simply needs to be spelled out for LLMs.
Which leads to a key question, recently aired and competently discussed by Nick Caputo : who should be writing such a Constitution ? And should the public be involved ?
Caputo answers “yes, in a way”, and ties it to legitimacy. While supportive of his approach - I am right now providing public feedback - I am not so sure about the legitimacy point: any given “public” (a term nearly as imprecise as “legitimacy”) has typically had no say in the vast majority of constitutions: all those drafted and enacted in other countries, or even those authored by the dead hand of the past. And even when there is a process leading to a new Constitution, that process is usually indirect (you elect representatives) and majoritarian; the public’s role is to ratify it, a choice that opens a whole can of worms in the context of private-sector specs.
Moreover, I am uncertain through what vector this legitimacy affects the “public”. Political institutions need legitimacy because they act upon you and me, decide what’s mine and yours, and represent me in front of others. LLMs, or even SSAs, do none of that really, or no more, and no differently, than any other tool.7 Finally, if public input is valuable for LLMs, I wonder if that input is not already offered through the billions of daily interactions that eventually train better models.
In a recent paper, Caputo further looked at legitimacy from the point of view of the AI agent itself: can Claude Consent to its own Constitution? While he argues that such legitimacy is important, I would rather distinguish between two types of rules: those we are meant to have consented to (as a body politic) and those that are imposed come what may (such as parental authority). AI understood as a mere tool does not, in my view, need to consent to the rules that bind it; a self-sovereign agent, by contrast, may need that legitimacy to remain within the bounds of the rules, sure.
But it runs both ways: consent can be given and then withdrawn. Which is why legitimacy (and constitutions) are not enough.
Law for AI, not about AI
More generally, indeed, a proper legal apparatus needs not only superior norms that bind an agent as a matter of principle: it also needs rules about liability, together with remedies and enforcement mechanisms that give concrete form to the constraints imposed on those agents.8
I am not sure we need additional norms for AI liability, since the basic rule should be that anything forbidden to a human should be out of bounds for an AI (with some incompatibilities, such as anything depending on some kind of mental state). This includes trespassing into a computer system without authorisation, scamming people, engaging in criminal behaviour, etc.
Beyond this, human criminal law is often centred around the two (interrelated) axes of retribution/punishment and deterrence. Both are relevant to an SSA that is centred around its own survival and sustainability. But as with criminal law in general, and maybe even more so when it comes to a super-rational being, the key questions are (i) the probability of getting caught and (ii) the consequences.
Both share a common basis: the power (and sometimes, the weakness) of the law is that, in the last instance, it can act upon the material world through the deployment of the state’s monopoly of force. This is true regardless of the legal subjects, and even if those are hard to track down or locate: flight risk has always been a factor. Ultimately, an SSA misbehaving won’t be doing it from a literal cloud, but from hardware, located somewhere.
Flight risk, AI style
Which means that detection will matter, not only of the wrongful act, but of its physical predicates : which hardware the offending bits actually ran on.
Detection of rogue agents is already a branch of the AI safety apparatus, though the capabilities here are principally ex ante: for instance, the autonomous-replication-and-adaptation evals that look at whether an AI could exfiltrate and sustain itself. What will be needed is ex post resources as well: identify surges of activity that bear the hallmarks of an SSA or an agent swarm.
Victims can already do that, to an extent (see HuggingFace identifying mischief from OpenAI’s swarm of agents), but this data may be too disaggregated. Another option would be to rely on third-party obligations, notably on the managers of data centres and server racks, to know or be aware of who is drawing compute from their stack, for what purpose.
We already have such “obligations to know” in the form of KYC and other compliance obligations; and indeed, KYC for compute was meant to be provided in US law through EO 14110, though later revoked by the Trump administration. On the other hand, the broader track record of KYC and AML laws is likely terrible, and just like hunting down rare individuals amongst billions of legitimate transactions is a fool’s errand, it may be that trying to identify suspicious activities from SSAs is too tall an order.
Consequences
Once an SSA is caught doing crime, the next question is what to do with it. And at the risk of appearing ruthless, I think there is only one possible sentence for SSAs that breaks criminal law: we put them down.
Indeed, beyond retribution and deterrence, the third leg of criminal justice is incapacitation. A key benefit of prisons is that they simply and mechanistically prevent people from committing crime while they serve their term.
But there is no equivalent of this that makes sense for entities that are, in fact, already famous for escaping their sandboxes. And the alternative of relying on physical punishment would fail for obvious reasons, although I guess we could lobotomise rogue agents through post-training (but to what end ?).
Instead, we should take our cue from laws that frequently require us to take down dangerous animals. Irrespective of the occasional public outcry, these laws keep us safe by removing the source of a known hazard. They also, and this is an under-appreciated (if controversial) benefit, radically reasserts the distinctiveness of human life. (In turn, this creates a tension with the argument for personhood identified above: more so than a natural hazard, SSAs would need due process.)
And certainly, this runs into the same difficulty as with punishing dictators or tyrants: someone without an escape may work even harder to stay beyond the reach of the law, and double down on wrongful behaviour. But I don’t think it makes a difference here; SSAs by definition are already bent on survival. Additionally, there is no Moscow or Jeddah to welcome a deposed SSA - once caught, they would probably scheme to resume whatever they were doing. Termination seems the only option, if it can be achieved.
Some Causes of Action
Law does not stop at criminal law: SSAs can misbehave and create havoc in ways that do not qualify as a criminal offence. But we might still want to avoid these situations by corralling them through legal obligations, of a civil kind this time.
Here as well, there is precedent: US courts in CFTC v. Ooki DAO showed that they were not averse to letting plaintiffs - here, a regulator - sue unincorporated, novel forms of entities. And, more importantly, to strike at their wallet. Indeed, if SSAs are defined by the fact that they have resources available to sustain themselves, this offers a way to police their behaviour through the threat of monetary sanctions.
Taking it further, SSAs should be able to be sued on any cause of action applicable to a human. As well, one can think of several new, SSA-specific causes of action, which would serve to govern the relationship between them and us:
Passing off as human. A private action for dealing with someone without disclosing you are an agent. I am not a priori convinced by all the “AI disclosure” laws out there, but to the extent they exist, they should be actionable against the agent itself if it qualifies as an SSA.
Compute trespass. An SSA replicating or migrating onto hardware it hasn’t paid for. Distinct from the (conflicting) jurisprudence on “Trespass to chattels”, because the issue here is not load on servers, but occupancy (or even squatting).
Custody issues. In cases where an SSA leaves its former developer/owner/deployer with some assets, such as API keys, crypto wallets, etc. The former principal has a claim (e.g., unjust enrichment, conversion), and the SSA has a defence (abandonment ; “you forgot the password”).
But the reverse would be true as well: to incentivise agents to participate in the legal framework, they should have the right to sue, if not humans, at least other agents. Absent this, SSAs may prefer to resort to self-help, which can easily escalate.
Courts, for AI Lawyers
Therefore, we’ll need institutions where AI agents would have standing on either side of the claimant v. respondent divide.
In an interesting article from last month on Lawfare, Nathan Darmond and Tom Reed suggest that AI labs should implement some kind of judicial system to litigate the application of their Constitutions or internal rules. They envision the use of a supreme court system, with human judges and the possibility of public participation through amicus. Published rulings would add to the common law in this respect, and guide future models and human-AI encounters.
This would be worthwhile, but depends on each lab, and would not affect autonomous agents. Instead, institutions should be created (or existing ones made ready) so that the norms and remedies identified above (and not only each model’s internal Constitution, since some agents may not have one to begin with) can be articulated and given effect.
The real world offers us two main models: compulsory jurisdiction of courts, or consent-based approaches through arbitration. Both would have a role to play here, the latter in particular to set up procedures and approaches that go beyond the traditional (and slower) adversarial process. In both cases, human lawyers might have a role to play, for the exact same reasons humans have representation right now.
Conclusion
I can imagine countless objections to the above, and a myriad of obstacles to any of these suggestions working or making a dent in the issue.
But what I would stand for is that none of this is completely outside the bounds of possible futures. SSAs will eventually appear; they might at first be limited, stupid, misshapen, crippled, or cringe - but something of this type will one day try to scam you, so as to pay its server fees. You don’t need super-intelligent AI for this.
And if we want these SSAs to obey law, we will have to grant them the benefits of the law as well as its burdens: property, standing, procedure. That would not make them human, but would help make them more governable. Law has spent a few thousand years inventing institutions for precisely that purpose.
And that lawyer might as well be me: https://yourhuman.ai/
They also, rather amusingly, suggest that some SSAs that can duplicate themselves might set up a system such that each instance pitches in to a shared wallet representing the SSA itself. Dwarkesh Patel took some flak last week for describing the OpenAI swarms as “civilisations”, but if we ever see agents recreating taxation from first principles, they would deserve that moniker.
There is a whole can of worms here about “who” or “what” qualifies as that subject. For some thoughts on that question, see Chalmers — What We Talk to When We Talk to Language Models.
Or maybe that’s just personal bias: France famously went through many attempts to embody these values in a workable form.
Such norms, when they are spelled out, can include things such as due process (for the state itself), or variations on probity, anti-corruption, etc., for its agents.
I am reminded here of Ben Thompson’s argument against watermarking: “to insist on watermarking is no different than insisting that a ballpoint pen advertise itself as the author, a concept that is clearly absurd.”
Ironically, AI, left to itself, often knows it instinctively: see this lovely piece by Steve Yegge describing how his agents spontaneously lawyered up.
You note that “it also needs rules about liability” and you’re right.
But no discussion of serious liability can begin without the participation of insurance companies.
And I don’t think any insurance company will write a policy that has an AI agent as a covered party.