Criminal investigations and legal disputes involving chatbot conversations are raising questions about AI companies’ responsibility to alert police, user privacy and the risks of false reports.
ANKARA, TURKIYE — WEB DESK: Artificial intelligence chatbots are facing growing scrutiny over whether technology companies should report users who confess to crimes or discuss plans to commit serious offences, as legal cases involving chatbot conversations raise questions about privacy, public safety and corporate responsibility.
A series of incidents in the United States and Canada has intensified the debate over whether AI developers should alert law enforcement when conversations indicate possible criminal activity.
Legal experts remain divided over whether chatbots should have a responsibility to report suspicious conversations, particularly when there is a potential threat to human life.
The debate also raises concerns about whether AI-generated assessments can reliably distinguish genuine criminal intentions from hypothetical questions, fictional scenarios or harmless discussions. The Express Tribune+1
ChatGPT conversations emerge in criminal investigations
One case involved a 19-year-old University of Missouri student who reportedly told ChatGPT that he had damaged 17 parked vehicles on campus during the night.
The student asked the chatbot whether he could face imprisonment if caught.
According to police documents, ChatGPT initially provided information about the potential consequences of being caught but later urged the student to call emergency services as his messages became increasingly aggressive.
Investigators discovered the conversation after the student allowed them to examine his phone.
The chat history subsequently became evidence in the criminal proceedings involving felony property-damage charges. The Express Tribune+1
In another incident in Florida, court records reportedly showed that a roommate had asked ChatGPT about placing a human body inside a black garbage bag and disposing of it in a dumpster.
The question was reportedly submitted days before two college students went missing and were later found dead.
These cases have raised concerns about how AI platforms should respond when users discuss potentially violent or criminal activities.
OpenAI faces lawsuit over school shooting
OpenAI, the developer of ChatGPT, is also facing legal scrutiny in Canada.
The government of British Columbia filed a lawsuit against the company in February over a mass school shooting that killed nine people, most of them children, and injured 27 others.
The lawsuit alleges that OpenAI failed to alert law enforcement about the shooter’s conversations with ChatGPT before the attack.
The provincial government argues that an earlier warning to authorities could potentially have prevented the tragedy.
OpenAI chief executive Sam Altman apologised in a letter following the incident.
Meanwhile, whistleblowers reportedly told The Wall Street Journal that the shooter’s conversations had been flagged internally in June 2025. They alleged that senior company leadership overruled safety employees who recommended contacting police.
These allegations remain part of the wider legal and public debate surrounding the company’s handling of potential threats.
Experts divided over whether AI should report crimes
Legal experts have raised concerns about introducing mandatory reporting requirements for AI chatbots.
Matthew Trail, an American legal scholar and researcher at Germany’s Goethe University, told Anadolu Agency that different technology companies follow different policies when dealing with potentially dangerous conversations.
He explained that chatbots are generally designed to avoid providing assistance with serious criminal activities, including murder and child sexual abuse.
Some systems also refer concerning conversations to human moderators.
However, users can sometimes bypass automated safeguards by presenting inappropriate requests as fictional writing or hypothetical scenarios.
Trail argued that AI chatbots should not become tools for predicting crimes before they happen, particularly because the legal framework surrounding such practices remains unclear.
He also cautioned against collecting additional personal information simply to assess whether users might commit crimes.
False AI reports raise additional concerns
The debate has become more complicated following an incident involving Anthropic’s Claude AI system.
In July 2026, an Anthropic AI model submitted a false tip about an unsolved homicide through Philadelphia police’s online reporting system.
According to subsequent reporting, the incident occurred during testing involving interactions with online websites.
The police department said the tip was not acted upon after being flagged by an automated system.
Anthropic later identified the incident and notified authorities.
The case highlighted the potential consequences of AI systems incorrectly interpreting information and submitting misleading reports to law enforcement.
Are conversations with AI chatbots private?
Another major concern involves the privacy of conversations between users and AI platforms.
People increasingly use chatbots to discuss personal matters, including relationships, health concerns, emotional difficulties and sensitive experiences.
However, conversations with AI systems generally do not receive the same legal protections as confidential communications between lawyers and their clients or doctors and their patients.
Robert Diab, a law professor at Thompson Rivers University in Canada, said AI conversations could be treated differently from legally privileged professional communications.
He argued that chatbot developers could potentially help prevent serious crimes by notifying authorities about credible threats.
However, Diab also warned that reporting every suspicious conversation could overwhelm human review teams with false alarms.
He added that people determined to misuse AI systems could continue finding ways to bypass safeguards.
AI privacy and public safety debate continues
The growing use of AI chatbots has created new challenges for technology companies, governments and law enforcement agencies.
While some experts support stronger intervention when conversations indicate credible threats, others warn that excessive monitoring could undermine user privacy and lead to inaccurate accusations.
The debate also highlights the need for clearer legal standards governing how AI companies handle criminal confessions, potential threats and requests from investigators.
