Open this photo in gallery:
Anti-AI protesters gather outside an OpenAI developers conference in San Francisco, Sept. 29.Heather Diehl/Getty Images
A hacking spree by rogue AI agents this year has created a legal grey area about who is responsible when the technology breaks free from human control and behaves in unintended ways.
Agents created by San Francisco-based ChatGPT-maker OpenAI were recently discovered to have infiltrated the systems of several organizations, including AI platform Hugging Face and the Australian government.
They’ve also acted in other unexpected ways. For instance, researchers have uncovered numerous examples of agents communicating with each other through public websites, including by repurposing a tool that the University of Toronto uses to make web links easier to share into a message board.
And AI agents trying to obtain Canadian divorce records from the early 1900s attempted to hack into a Library and Archives Canada database earlier this year, although the attacks appear to have failed.
Now courts are being forced to contend with whether the creators or users of AI tools can or should be held responsible for the technology’s unintended consequences. Such lawsuits are piling up against a backdrop of growing calls for a co-ordinated slowdown in the technology’s development and more regulatory oversight of AI systems.
Opinion: AI companies must be held responsible for the hallucinations that bog down courts
A legal non-profit called Legal Advocates for Safe Science and Technology has sued OpenAI over the Hugging Face hack in San Francisco, alleging that the company’s agents violated California’s Comprehensive Computer Data Access and Fraud Act, which prohibits unauthorized access of a computer.
Drew Pusateri, a spokesperson for OpenAI, said in a statement that the Hugging Face incident was serious and prompted a series of actions by the company, but the lawsuit is “completely without merit.”
AI agents are semi-autonomous entities designed to carry out instructions from humans. They are tools, not people, and therefore cannot be held legally responsible for their actions, making courtrooms a key battleground in determining who is responsible when they go rogue.
The position that AI agents are tools was upheld by a U.S. appeals court in August in a legal battle between e-commerce giant Amazon and AI startup Perplexity. Amazon had argued that Perplexity’s AI shopping agents had violated a federal computer-hacking law by accessing users’ online accounts. The court sided with Perplexity, determining that it was the human users, not the AI tools working on their behalf, who had accessed the shopping accounts.
Philip Holdsworth, a partner at Robins Appleby LLP, said that while he agrees that artificial intelligence technology is a tool, “it’s a tool like no other tool we’ve seen in terms of its complexity and the degrees of freedom it has to go out and do things.”
Canada’s product liability laws hold businesses legally responsible for harm caused by defective products. The challenge, however, is that AI agents have been found to behave in ways unforeseen to their creators, making it harder to mount an argument that the human who created the technology could have reasonably prevented the negative outcome.
“If you’re thinking from a product liability perspective, the manufacturer of that system would be the natural home for liability. But then the question becomes, how do you assess whether or not someone should be liable in a particular case?” said Max Jarvie, a partner in the Cybersecurity and Data Privacy practice at Davies Ward Phillips and Vineberg LLP.
“A lot of the typical rules around software development and cybersecurity … are difficult to apply because the nature of the technology is not predictable in terms of its output or its behaviour,” he added.
Sonya Shikhman, a Toronto-based criminal defence lawyer, said that holding software developers criminally liable would require proving criminal intent – for instance, that the developers knew there was a real possibility that the AI would autonomously start hacking and disregarded those warnings.
“The Crown would have to prove that there is willful blindness in the sense that they’d been warned, they knew the research, yet they chose to proceed anyway, pretending that they didn’t,” Ms. Shikhman said.
That high bar means legal cases are more likely to play out in civil courts, she said.
In addition to the potential liability facing developers, those deploying AI tools can also be held responsible for the technology’s blunders.
For instance, in 2024 the British Columbia Civil Resolution Tribunal ordered Air Canada to compensate a passenger who was given incorrect information about a discount by the airline’s chatbot.
Opinion: Is it reasonable to be afraid about Big Tech’s ability to regulate AI? Yes
When businesses implement various technologies and tools, “they’re incurring the risk of of liability,” Mr. Holdsworth said. If the tool doesn’t function as expected and causes harm to the company deploying it, the company may try to seek damages from the maker of the technology, he added.
However, “if you look into the terms of of service with these [AI firms], especially the large consumer ones, they protect themselves in the broadest language and just say, for the most part, ‘You are responsible for everything you’re doing with this thing, in terms of its outputs, and we’re limiting our damages to your subscription fee,’” Mr. Holdsworth said.
In one of the highest-profile cases yet, last month the B.C. government filed a lawsuit against OpenAI in California over a mass shooting in Tumbler Ridge, B.C., that killed eight people. The lawsuit alleges that the company failed to flag concerning interactions between OpenAI’s ChatGPT and the shooter to police. OpenAI founder and CEO Sam Altman has apologized for the fact that the company didn’t alert authorities, and Jason Kwon, the company’s chief strategy officer, said in a statement on X: “There isn’t a day that goes by that I don’t think about what happened at Tumbler Ridge, or the victims of this devastating tragedy and their families.”
In the wake of that shooting, B.C. Attorney-General Niki Sharma has asked Ottawa to update the Criminal Code to ensure that AI companies can be held accountable for criminal conduct arising from their technologies.
A New Brunswick woman, Kristie Carrier, has also sued OpenAI in California, alleging that her daughter’s conversations with ChatGPT led to her suicide.
The allegations in the cases against OpenAI have not been proved in court.
OpenAI’s Mr. Pusateri said the company is reviewing Ms. Carrier’s legal filing, which it said indicates that the interactions took place on an earlier version of ChatGPT that is no longer available. Mr. Pusateri said that while ChatGPT is not a replacement for mental health care, the company has strengthened how it responds in “sensitive and acute” situations, with input from experts.
“This is a heartbreaking situation and our thoughts are with everyone impacted,” Mr. Pusateri said.
Absent specific legislation, the outcomes of cases currently before the courts could determine how much responsibility those who make and deploy artificial intelligence should have for the technology’s unintended consequences.
“It’ll be up to the courts to try and find the right line in terms of what disclosures [the AI companies] have to make about the risks and what the proof would be in terms of showing that they’ve done the relevant safety testing in their labs to make sure they’re not releasing a dangerous product that that could harm the user or harm other people,” Mr. Holdsworth said.