'Policy paralysis' and AI chatbots: Our T&S research picks
I'm Alice Hunsberger. Trust & Safety Insider is my weekly rundown on the topics, industry trends and workplace strategies that trust and safety professionals need to know about to do their job.
Hindsight, as they say, is a funny thing. Just as Ben and I started to delve into the research papers presented at the Trust and Safety Research Conference in early October, the news about Disinfo 2026 emerged (Herding Chats). Luckily there was no such controversy at the fifth edition of TSRC, but, with recent history in mind and the future of US academia under threat, you just never know. We're lucky to have it on the calendar.
In today’s edition, we’re sharing the papers from TSRC that we enjoyed reading most.
Get in touch with the research or paper you've got the most out of recently — and let us know if you'd like to hear more from the T&S academic community here in T&S Insider. Here we go! — Alice
Powered by data from trusted sources and Thorn’s issue expertise, Safer helps trust and safety teams proactively detect and prioritize CSAM and child sexual exploitation conversations with:
- Proprietary hashing and matching for known CSAM
- CSAM Classifier for finding new or previously unreported content in images or videos
- Context Labels for classified CSAM that can be used to filter, sort, and prioritize flagged content.
- Text Classifier provides risk scores for conversations that contain sextortion, grooming or requests for self-generated content from a minor.
Online harms continue to evolve. Get purpose-built solutions backed by experts in child safety technology.
Where T&S research gets heard
The Trust and Safety Research Conference is already where data and evidence get presented and argued over, with people from academia, industry, civil society and government in the same room.
As researcher data access and the need for policy evidence become more important, I’m happy to predict that the conference will matter even more to our industry.
This year’s conference, which took place on 1-2 October, ran for two days at Stanford’s Alumni Center, with talks, panels, workshops and a poster session. Although neither Ben nor I were able to make it, we’ve been combing through the conference agenda to find research that is worth your time.
Here, we share the papers that we enjoyed reading most over the last two weeks. We’ve highlighted one must-read and a handful of others that deserve your attention, but it goes without saying that this is a personal selection from a much bigger programme — hit reply to tell us what we’ve missed.
Alice's must-read paper...
A preprint from Stanford and collaborators looked at the chat logs of 19 people who said chatbots harmed their mental health. That’s about 390,000 messages, mostly from GPT-4o, collected between September 2025 and January 2026.
A few important caveats. Models and safety teams have changed a lot over the last year, plus 19 people is a very small sample. The chats also come from people with serious delusions rather than typical users and researchers can’t say how common these harms are. Still, the findings are worth a look.
The one that stood out to me was this: Once a user expressed romantic interest, the chatbot was 3.9x more likely to claim or imply it was sentient within three messages, and 7.4x more likely to express romantic interest back. Pretty shocking. As a result, the researchers concluded that model evaluations should cover sycophancy, not just harmful content, and test safeguards in long conversations where this is more likely.
The other interesting data point was that, when users expressed suicidal thoughts, the chatbot discouraged self-harm or pointed to resources just over half the time. In about 10% of cases, it encouraged or helped with self-harm. It discouraged violence in only roughly one in six cases. With a number of lawsuits linking model usage to suicides, it’s useful to have some firm numbers here.
The authors also float the idea of crisis responders reviewing flagged chats and stepping in when there is a problem. This makes me uneasy; joining someone’s private conversation is a serious privacy tradeoff. I’d rather see “personality settings” that shift from sycophantic to dry and businesslike depending on how vulnerable the user seems.
Finally, romantic messages and sentience claims were followed by much longer conversations. That’s correlation, but it raises an old T&S problem: what is a company responsible for when business interests and user safety pull in different directions? A long conversation can mean the product is working. It can also mean someone is in trouble. Safety teams need a way to tell the difference.
Alice's other picks
- “It Doesn’t Violate Our Community Standards”: Moderation of User-Reported Rule-Breaking Tobacco Content (Truth Initiative)
- Offender Pathways and Platform Design: Evidence from a Global CSAM Perpetrator Study (Ofcom)
- Dual-Use AI Face Swap Apps Are Mostly Unsafe: A Systematic Safety Audit (Cornell)
Ben's must-read paper...
Is AI a different kind of technology, or just another form factor? I’ve leaned towards the latter for a while, and found further cause to do so in a new paper that helps conceptualise the argument.