AI safety theatre, Shopify’s policy dodge, and the reality of moderation work
Hello and welcome to Everything in Moderation's Week in Review, your need-to-know news and analysis about platform policy, content moderation and internet regulation. It's written by me, Ben Whitelaw and supported by members like you.
This week, global leaders and thinkers gathered in Paris for what one EiM reader called "a very French summit" to talk AI governance. Safety, perhaps unsurprisingly with recent events, was not a major topic but there was several notable announcements that suggests there are people — and US-based platforms — with a shared perspective on what a safe internet looks like. More on those below.
Welcome to new EiM subscribers from The Fire, INETCO, KU Leuven, the Conscious Ad Network, Google, Duck Duck Go, Character.ai and elsewhere. If someone shared today's newsletter with you, sign up to get Week in Review in your inbox every Friday and T&S Insider by Alice Hunsberger every Monday.
Let's get into this week's stories — BW
Today's edition is in partnership with Checkstep, the all-in-one Trust & Safety Platform
Generative AI is transforming how content is created and shared, but it’s also creating new challenges for Trust & Safety. From deepfakes to misinformation and scams, the risks are evolving faster than ever.
We understand how overwhelming it can be to navigate these emerging threats while keeping your platform safe. That’s why we partnered with GenAI experts at Sightengine to create the GenAI Moderation Guide, a unified resource to help you tackle these challenges head-on.e
In the guide, you’ll find:
- A step-by-step approach to mapping and prioritising generative AI risks,
- Practical advice on crafting AI-resilient policies tailored to your platform’s needs,
- Real-world insights and templates to help you stay ahead of emerging threats.
Policies
New and emerging internet policy and online speech regulation
All eyes this week were on the AI Action Summit in Paris, where 1000+ tech and politics bods spent two days discussing how to balance AI innovation and security. Unlike previous summits over the last few years, and according to EiM readers that were there, safety was a much smaller part of the conversation. This is reflected commentary I’ve read this week:
- Analysis from Mark Scott over at DFRLab gets to the heart of the issue: governments are simplifying their approach to AI regulation as part of a shift towards competition and economic growth.
- A statement from Anthropic founder Dario Amodei (EiM #223), who was present in Paris, called for democratic societies to lead the development of AI and flagged the risk of biological and chemical warfare.
- Unlike the London and Korean editions, the US and the UK refused to sign the end-of-summit communique that called for AI to be developed as “safe, secure and trustworthy”.
Sounds like a fun networking event more than a turning point for AI governance. Drop me a line if you were there and have thoughts.
Last November saw thousands of documents about how large platforms moderate content made public under the Digital Services Act and there is another transparency report deadline looming in April: but, as David Sullivan writes for Lawfare this week, observers have largely “ignore[d] the substance of these reports” which is a “missed opportunity to improve the process” of regulatory transparency. The executive director of the Digital Trust & Safety Partnership calls for better guidance for future report rounds and engagement from a wider group safety-focused stakeholders “to improve the process”.
Implementation fatigue?: joining the dots on the two above stories, it appears that companies in Europe and the US may be reaching their limit with the EU’s approach to systemic risk assessments for online platforms. Perhaps if the DSA implementation was working seamlessly (and it’s hard to say that it is), there may be greater willingness to adopt a similar-approach for general-purpose AI models. But that looks unlikely.
Also in this section...
- Will crackdowns on big tech work? (London School of Economics)
- Message to US States: Don’t Forget the Fundamentals of Fighting Online Hate and Antisemitism (Tech Policy Press)
- Safeguarding the Digital Playground: New Regulations in Australia, Indonesia, and Singapore for Protecting Minors on Social Media (CMS Law)
Alice Hunsberger
Products
Features, functionality and technology shaping online speech
One of the big announcements at the AI Action Summit was a new cross-industry initiative that seeks to create a common set of safety tools that are open source and free to end users (at least initially). Backed by Roblox, Discord, OpenAI, and Google and others, the Robust Open Online Safety Tools (ROOST) initiative will “make essential infrastructure more transparent, accessible, and inclusive” and has raised $27m from platforms and funders to get itself through the first four years of operations. As The Verge reports, it will focus on three areas: building tools to address CSAM, classifiers to reduce harm and moderator consoles with wellbeing baked in.
Platformer’s Casey Newton called it “an initiative that might actually begin to reverse this slide” of platforms failing to take safety seriously. He also noted the absence of Meta, Reddit, Snap, which poses the question: will platforms actually contribute to ROOST or could this be a kind of safety-washing?
Disclaimer: one of the founding organisations behind ROOST financially supports the production of Ctrl-Alt-Speech but has no input into editorial decisions on the podcast or newsletter.
On the topic of flashy announcements between big platforms, I was interested to note the unveiling of Current AI, a public-private partnership, led by the French government and the likes of Google and Salesforce, to create large-scale projects in the public interest . Ten countries including Finland, Nigeria and Chile, and tech founders like Reid Hoffman (Linkedin), Clement Delangue (Hugging Face) and Artur Mensch (Mistral) have all given their backing to the project, which will focus on providing accessible data, open source AI and involving communities that will be affected by AI.
Wider view: I was struck by a quote from special envoy Anne Bouverot on Current AI's slickly designed website which suggests that the partnership is part of France’s shift away from safety and EU-led regulation:
“We need to do things in the affirmative sense not just think about regulation and things we want to forbid”
While there’s merit in that, T&S folks present at the summit suggested that it went too far the other way. AI, it seems, is the growth engine to which every country is hitching their wagon. Where we end up is anyone’s guess.
Platforms
Social networks and the application of content guidelines
This week’s latest platform policy drama involves Shopify. The e-commerce platform’s technology briefly powered swastika-branded shirts from on the website of rapper Ye (who you may know as Kanye West) until it was removed on Tuesday. The issue was not that the products were anti-semitic but that “this merchant did not engage in authentic commerce practices” and was a fraud risk.
In an internal memo reported by The Logic, general counsel Jess Hertz noted that the focus on usage policy allowed it to “remove as much subjectivity as possible” when making decisions about content moderation. The convoluted logic might have something to do with the company removing “hateful conduct” from its policy last summer and the fact that quarterly profits were up 31% year on year. Hopefully not linked?
Hate speech on X/Twitter didn’t just rocket immediately after Elon Musk took over, it continued increase according to new research from the University of California. Comparing English-language posts from ten months prior to Musk’s acquisition to eight months afterwards, researchers saw sustained increases in racist, transphobic and other hate speech as well a significant uptick in engagement with such speech. Oh and bots — despite Elon’s best efforts — may have increased too. Read the full paper.
Also in this section...
- Safer Internet Day 2025: Tackling abusive AI-generated content risks through education and empowerment (Microsoft)
People
Those impacting the future of online safety and moderation
Sonia Kgomo never wanted to be a Facebook moderator. Like thousands of others, she took the job because it paid better than retail. But within weeks, she was watching suicides, filtering gore, and training herself not to flinch.
Her op-ed in The Guardian is a gut-punch of a read. She describes the psychological toll of reviewing horrific content, the lack of support, and the crushing realisation while talking to her colleagues — who were also her friends — that her mental health was suffering. She has since gone on to be involved in the African Tech Workers Rising project.
Her story isn’t new — I’ve paid testament to many others here (EiM #159, Q&A with Billy Perrigo). As it stands, content moderation and data labelling continues to be a human-driven, trauma-inducing endeavour one that platforms continue to outsource to people with the fewest protections. Whether initiatives like the Data Workers Inquiry and the manifesto of a group of German moderators can change the future of the work done by Sonia remains to be seen.
Posts of note
Handpicked posts that caught my eye this week
- “We hope readers find this informative and practical as a way to help harmonize approaches across Europe when it comes to youth online safety.” – Amazon's Farah Lalani on a new report with Dr Sameer Hinduja on empowering youth safety efforts in Europe.
- “The path from the present moment to radically different futures isn’t clear. When everything feels inevitable and the next step isn’t clear, we need guides.” - Charles Johnson on the imminent launch of Untangled Guides, which frankly sounds great.
- “Kyle Langvardt and I have a new paper out in the Journal of Free Speech Law on last year's Supreme Court decision in Moody v. NetChoice and what it means for the future of First Amendment protections for platform content moderation decisions.” – Looking forward to digging into this paper from law associate professor Alan Rozenshtein.
Member discussion