Over 2.5k subscribers
Autonomous AI agents will be able to collude without being told to. When that happens our existing legal frameworks will have no one to hold responsible. Liability for agentic AI should be traceable, not absolute.
This is a link-enhanced version of an article that first appeared in the Mint. You can read the original at this link. For a full archive of all my articles please visit my website.
Nine years ago, in an article in this column, I tried to imagine what would happen if autonomous algorithms were let loose in a digital market with instructions to maximize profit. I believed that, even if they had not explicitly been told to do so, they would find ways to collude with other algorithms to achieve that outcome. If this happened, I had pointed out, our laws would be woefully ill-suited to address it.
Nine years later, agentic artificial intelligence (AI) is here—and the problem is not that our laws cannot reach it. It’s that the legal doctrine that does may reach too far.
What happens when autonomous AI agents are set loose in a market? In a recent experiment, researchers gave a pair of large language model-based pricing agents a market to compete in and a simple instruction to maximize profits. They neither instructed them to coordinate with one another nor provided them with tools to communicate. Despite this, the agents quickly learnt to hold prices above competitive levels and refrain from undercutting each other to avoid triggering a price war.
But this is just the tip of the iceberg. According to law professor and consumer advocate Zephyr Teachout, this behaviour can be easily augmented and turned on consumers with what she calls “surveillance pricing”—the use of information about a consumer to generate a personalized price for that person.
When e-commerce agents are equipped with this, they can use their knowledge of our browsing history and device information, as well as the urgency of our need, to offer us bespoke prices tailored to what they know we will be willing to pay. For example, sellers would charge parents more for diapers because they just bought children’s cough syrup, knowing that when their child is sick, they may be unwilling to shop around for a better offer.
We have, so far, relied on competition law to protect us from these harms. But this law was designed partly to identify rival businesses that secretly agreed to fix prices, carve up markets or rig bids. Because it assumes that markets are anti-competitive when businesses agree to make them so, it prohibits agreements that enable collusion with the intent to harm. If collusive actions are taken by autonomous agents of their own accord, with no human to attribute intent and no price ‘agreement’ on record, no competition law violation can be made out.
Indian courts have already ruled on this question. When the pricing algorithms used by Ola and Uber were challenged as anti-competitive, the court held that an algorithm setting the price does not, in and of itself, prove collusion. When algorithms autonomously coordinate with each other, there is neither an identifiable actor to establish intent nor any form of agreement between them that satisfies the conditions required to establish an offence.
But we still need to hold someone liable when a person is harmed by autonomous agentic systems. One option would be to attribute liability to the company that built the AI model. This, however, would be patently unfair, since the developers of general-purpose AI models can hardly be expected to foresee, let alone prevent, everything that AI agents developed by them will eventually do for users.
It probably makes far more sense to hold the deployer of an AI agent accountable—except that when agents can spawn other agents, harms often occur several layers down the stack. What’s more, when multiple deployers spawn many agents and each of them spawns several more sub-agents, it is in the interplay of this diversity of sub-agents that harm occurs. How can anyone be held liable when the damage is so far removed from a single person’s actions?
The trouble is that in India, this is entirely possible. In M.C. Mehta vs Union of India, the Supreme Court held that an enterprise engaged in a hazardous activity is absolutely liable for the harm that results. This was a departure from the older rule of strict liability, under which an enterprise could escape by pointing to an act of God or that of a third party. Absolute liability admits no such defences.
While this was a case about oleum gas and chemical plants, its logic—that those who unleash a dangerous force are liable for everything that happens when it escapes their control—maps cleanly onto situations where autonomous agents slip the leash. This means that a deployer may not be able to avoid liability because the harm was caused by a sub-agent several layers down the system that it never built.
If this is how liability is attributed, it could chill the development of AI in India. When anyone who deploys an agent is liable for everything it and its sub-agents go on to do, no one will dare deploy one. And agentic AI would become commercially unviable.
In my article nine years ago, I argued for guardrails to prevent this harm. I am now more convinced than ever that they serve an important function. By making liability legible even when the actual harm is caused several layers down, we would be able to assign responsibility fairly and proportionately to the autonomy each person chose to grant their agent.
Liability should be traceable, not absolute. If every agent is made to carry a record of who deployed it, along with the limits within which it is designed to function, we will have a better chance at holding the right person or persons responsible for harm.
Rahul Matthan
Comments