AI Reported Woman to Police After Violent Diary Entries

A woman in Florida has been arrested after threatening to shoot at a local police station in conversations with an AI service she described as a digital diary. In the discussions, she also reportedly claimed to have acquired a new weapon. The AI service’s safety system responded to the threats, which were then reviewed and reported to the police.

According to the police investigation, a user identifying herself as Carli wrote in Anthropic’s service on September 26 that she was going to shoot at the Lee County police station.

The next day, the same user allegedly wrote that she had obtained a new weapon. The AI service’s safety systems detected these statements and triggered a process where the content was forwarded for human review.

According to the police report that forms the basis of the investigation, Anthropic uses various safety functions to detect keywords and content that may pose a threat.

Because the statements were deemed serious, the case was escalated to the company’s manual review team. The information was then passed on to the police.

Police were able to identify the woman as Carli Michelle Heller and went to her home in Bonita Springs. She was arrested on suspicion of making written violent threats.

“You can’t expect to be anonymous”

Sheriff Carmine Marceno has stated that Heller defended herself by saying she uses AI much like a “diary.” Marceno stresses, however, that people using AI services should not assume their conversations are entirely private or that they can remain anonymous.

“Artificial intelligence is a powerful tool, and just like any other technology, it can be misused. When someone uses AI to issue or facilitate a threat, we must take that threat seriously,” Marceno told WINK News.

The sheriff also says users should be aware that their activity can be traced back to them, even when using AI chats or making searches via such services. He emphasizes in particular that anyone using this technology cannot expect to be completely anonymous.

“AI can change the ways in which threats are made, but our commitment to protecting the residents of Lee County has not changed,” Marceno says.

AI can share information with authorities

Anthropic describes in its published user policy how the company’s so-called Safeguards Team uses systems to detect and monitor content to ensure the rules are followed.

The company’s policy, among other things, prohibits using its services to facilitate or encourage acts of violence, threats, or intimidation.

Furthermore, the company also states in its government requests policy that, under certain limited circumstances, user information may be shared with law enforcement authorities. This applies in particular to emergencies where there is a risk of death or serious bodily injury.

The case also raises questions about how people perceive their conversations with artificial intelligence. Ian Marlow, CEO of FITECH, says that the conversational style of interacting with AI can make users feel the technology is far more private than it actually is.

“Something that feels conversational doesn’t register as, ‘Oh, this is something being recorded,’ because they’re simply having a conversation,” Marlow says.

“AI is not your best friend”

Chrissann Ruehle, Assistant Professor in AI and Management at Florida Gulf Coast University, points out another possible explanation for why people turn to AI. According to her, it can feel easier to express oneself to a machine because the user doesn’t need to fear being judged or questioned.

At the same time, she warns against treating AI as if it were a private diary. Even if the conversation feels personal and confidential, it is still a technical service.

“AI is not your best friend. We must remember that this is a technology,” says Ruehle.

Anthropic has not yet commented on the case. Carli Michelle Heller has been charged with making a written violent threat and has a court hearing scheduled for November.