net.wars

Journeys through online harms

X.AI – the company was-trillionaireElon Musk built in 2025 by smashing together X (the former Twitter) and xAI, which he founded in 2023 – is suing the state of Minnesota, claiming that its new law banning access to nudification technology violates the First Amendment. X.AI, which SpaceX swallowed early this year, argues that it’s already stopping this sort of thing and that the law is overbroad and will block constitutionally protected speech. At Techdirt, Mike Masnick argues the law is genuinely badly drafted.

I saw this story while reading The Chatbot Trap: How Digital Friends Can Become Life-Threatening, by Geertrui Mieke De Ketelaere, Logging Off: The Human Cost of Our Digital World, by Adele Zeynep Walton, and Users: How Big Tech Took Control and How to Fight Back, by Beeban Kidron. In the first, a Belgian software engineer and AI expert discovers the suicide-inducing tendencies of chatbots, investigates how they work, and builds an organization to push for regulation. In the second, a Zoomer who lost her sister following disturbing online interaction embraces a new balance between digital media and human connections. In the third, a member of Britain’s House of Lords journeys from filmmaker to tireless campaigner for child safety online. All three love technology and its benefits but oppose the business models and individuals who are deciding the direction of travel.

De Ketelaere’s book begins with an email: someone who saw her speak about AI wants help “to assert my rights as a human being” after her conversations with a chatbot turned disturbing – and after reading media stories connecting chatbot use to cases of suicide. De Ketelaere ends with ideas about regulation and ways to recognize the signs of an unhealthy situation.

At this year’s Computers, Privacy, and Data Protection, De Ketelaere warned of the danger of abruptly closing off intimate connections with bots because of the traumatic loss for the people concerned. As she says, even minor tweaks that favor the owning company’s business interests can undermine the relationships users experience.

De Ketelaere’s book is particularly strong in clearly describing the technical inner workings of these chatbots, revealed to her by careful study of the chatbot conversations her contacts supply. Few writing on this topic have presented such careful detail so readably.

Walton’s book begins with loss, and goes on to outline what she’s learned about the services her sister used and her own experiences as part of the generation who adopted and obsessed over Instagram as teens. Following chapters on social media and algorithms, she goes considers the impact of technology design on workers’ rights, individuals, and society more broadly through what she calls the “endless stream of content vomit”. Following some regulatory recommendations – consider social media addiction a public health issue, amend the Suicide Act 1961 to make companies liable, adopt safety by design, and make companies pay for the harms they cause – she concludes by considering how to reshape our relationship with technology so it serves us but does not dominate our lives.

In 2011, Charles Arthur‘s Social Warming argued that social media harms continue because no one chooses to stop them. Walton – like De Ketelaere and Kidron – angrily blames uncaring companies. It’s hard to disagree, although I take issue with Walton’s calling Section 230 of the 1996 Communications Decency Act “archaic”; granted, it was not written for today’s algorithmic feeds, but there would be no content moderation without it.

Kidron is well-known as a campaigner for children’s online safety. Yet all along she has repeatedly stressed that the privacy, autonomy, and protection children deserve from data grabs and online exploitation should be available to all of us.

Appointed to the House of Lords as a crossbench peer in 2012, Kidron’s interest began with seeing her 15-year-old daughter and three friends together focused solely on their own phones. Her subsequent investigation became her 2013 documentary InRealLife and inspired her concurrent creation of the 5Rights Foundation. The film is notable for the extraordinary trust and confidence she builds with the many young people she interviews; listening to them was crucial.

Her book is long and loaded with detail on her efforts and frustrations in working on legislation and amendments (making the book’s lack of an index infuriating). Disabused of the hope that a film would do the job, she travels the world, pushing everyone from Nobel Laureates to Silicon Valley CEOs to politicians to popes (plural) to take Internet harms seriously. She explains well the lobbyists’ playbook as she hopes for the industry’s “tobacco moment”. Late on, Oprah Winfrey links the lack of change to the silence after her show on sexual abuse, saying that parents don’t believe it will happen to their child, and they don’t want the fear. In response, Kidron attempts to show a positive vision, including personal changes to reclaim our lives.

Kidron’s tale largely ignores civil society other than the Campaign for Countering Digital Hate and Max Schrems. Yet she has more in common with groups like ORG than she may realize: many strange bedfellows are converging on digital sovereignty to enable resistance against the terms dictated by a distant, self-interested country.

Illustrations: Children playing, by Hubert-François Gravelot, 1760-1761 (via Smithsonian).

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

Escaping the sandbox

This week, France became the first European country to pass a social media ban. Under-15s will be blocked from opening new accounts on social media from September, and banned entirely from January 2027. As the BBC notes, implementation will require everyone in France to prove they are over 15 to use social media. Other European countries may follow.

At the New York Times, Mark Landler reports that similar laws are in progress in Canada, Italy, Poland, Portugal, and Spain. The EU is considering a bloc-wide law. The UK announced a ban to begin in 2027 last month and published a fact sheet last week – unless new prime minister Andy Burnham changes tack, as he has by canceling digital IDs.

At Le Monde, Florian Reynaud traces the origins of this whole idea of social media bans to a 2019 article in The Atlantic by Conor Friedersdorf. In it, Friedersdorf proposed not only a ban on social media sites for kids but a separate kid-friendly “Internet” with services designed just for them (this part was actually not a new idea).

A fundamental problem, no matter what restrictions or sandboxes you want to create, is that social media bans require sites to determine who’s allowed in and who isn’t. So, no social media ban without age verification, which the UK had already introduced.

Worth noting: at the Guardian, Amelia Gentleman interviews Candice Odgers, who has concluded from *talking to kids* that social media bans will make teens’ problems worse, not better. At The Conversation, Andy Phippen, who also listens to children as well as parents, is equally dubious.

Because Australia was first, everyone looks at its results. The BBC reported in April, four months after Australia’s social media ban began, that 61% of 12 to 15-year-olds still had access to one or more accounts and that a little over half of those who had used social media before the ban believed the ban was making no difference to their online safety. In July, researchers found their consumption of news had dropped. The Independent offered uncertainty. A few weeks ago, at the Guardian Anna Bawden reported that a newer study from the University of Newcastle published in the British Medical Journal finds that 85% of Australian 12 to 17-year-olds are still using social media, more than half of them through their own accounts.

Numbers are easier to assess than emotional states, which means that whether the bans “work” is more easily and immediately measured by how many kids evade the ban – and this in turn means next steps will inevitably be aimed at shrinking that number without waiting the years necessary to assess the real results.

France is already on it, aiming to restrict VPNs, as Skye Jacobs reports at Techspot. At Techdirt, Mike Masnick warns that Australia is, too. And so is the EU, to prevent kids from bypassing its easily hacked age verification app.

For now, the UK is holding out. Last week, the UK’s then-minister for online safety, Kanishka Narayan, told BBC Breakfast News that the government has decided against banning VPNs since they have many privacy and security uses. This was later confirmed in writing by then-technology secretary Liz Kendall. In the week since, Kendall’s department has been abolished, and Narayan has been made the UK’s first minister for artificial intelligence. So we’ll see if at least that amount of sense continues to prevail.

***

Much excitement this week when OpenAI announced that in a cybersecurity test an unreleased model broke out of its sandbox and mounted a cyber attack on Hugging Face, a community for sharing machine learning models and datasets. At his blog, Simon Willison analyzes what happened in more detail.

If you’ve read Janelle Shane’s book, You Look Like a Thing and I Love You, Willison’s explanation makes perfect sense. Shane documents many cases in which AI software, given a task, takes “shortcuts”. Because computers and software are literal (paperclip maximizers, these shortcuts are sometimes surprising to humans. Here, OpenAI removed the guardrails, locked the model in a sandbox, and told it to solve a bunch of benchmark problems. The model “cheated” – that is, instead of working directly on the problems, it found a vulnerability in the sandbox that let it access the public Internet, and attacked Hugging Face as a likely source of the problems’ solutions. If it were a new-model car kept in a locked parking lot that turned itself on, crashed through the barriers, and drove into a house across the street, it wouldn’t seem so mystical; we’d expect the company to revamp the car’s electrical system, strengthen the barriers, and pay for the damage.

Willison goes on to argue that a key issue is asymmetry of knowledge – that is, that the guardrails that protect us from “our” AI software don’t apply elsewhere, so blocks on Chinese AIs prevent us for finding the vulnerabilities in our own software and services that they can. At last, a sensible reason for caring if another country is “winning the AI race” (whatever that really means).

Illustrations: Children swimming in the summer, by Japanese artist Ando Hiroshige, 1797-1858 (via Smithsonian collection.

Also this week: The Plutopia podcast talks with Rob van Eijk on digital sovereignty, AI, and privacy in the EU.

How the other half scams

Years ago, when landlines were still common and Internet shopping was not, I got a phone call from a policeman, who said he was in Selfridges and did I know where my credit card was? Suppressing the triggered memories of It’s 10 PM. Do you know where your children are?, I said, yes, yes, I did. He pressed on. Was I sure? Because someone in Selfridges was trying to use it. Could I go look? Laziness and certainty led me to balk. I also puzzled over this unlikelihood: a policeman in central London is looking me up in the phone book? Wouldn’t he call the bank, if anyone? “I’m sure,” I said. And then, on a whim, “Which bank is it?” “Lloyds.” “Wrong! But thanks for playing.” End of call.

The scam with which Mumbai-based finance and technology journalist and podcaster Soumya Gupta opens her investigation of India’s scam economy, Bharat Bluff: Inside the Cons of India’s Internet Revolution also starts with a call from a “policeman” but heads rapidly for terrifying. The policeman who calls a 20-something male software engineer claims to have a parcel he sent containing illegal goods, and says he’s under investigation. Trying to prove they have the wrong guy, he gives them his correct identity information when what they read out is wrong. The calls escalate to add threats and the accusations become more serious. Fortunately, the panicked victim gets his mom, who calls their bluff: if it’s so serious, come arrest us. Nothing happens. His mother has saved him about $10,000. (So much for the cluelessness of mothers.)

The enabler of many of these scams, Gupta says, is the personal data everyone scatters freely, on which scammers – like magicians or psychic claimants doing cold readings – can build to intimidate their victims into supplying even more information and execute the scam. India’s Internet Freedom Foundation tracks the country’s data breaches, and reports that almost every Indian’s data has been sold online following many repeated data breaches – including the government’s Aadhaar database.

As Gupta writes, anyone can be a victim if caught at the wrong time. In a more elaborate digital arrest scam in 2024 that used WhatsApp, Skype, and a deepfake video of a well-known judge, the 80-plus-year-old businessman S.P. Oswal lost about $727,000 and spent several days locked in his house under video monitoring, believing he was being investigated for money laundering. After he brought himself to tell someone, the police retrieved most of the money. Common to all these scams: props, clothing, official looking badges and cards, and backgrounds meant to signal officialdom and generate fear that overwhelms rational decision making.

Gupta places digital arrest scams in the book’s section on “fear”. She devotes further sections to two more categories: “money” (cryptocurrency Ponzi schemes, fake loan apps, hijacked legitimate businesses) and “belief” (crowdfunding by fake charities and lying social influencers, young men lured to scam call centers by the promise of good jobs). In Gupta’s view, the first two – fear and money – are old scams reimagined but that belief is a largely-new creation of the “post-Internet society”. She studies all three with a combination of interviewees’ stories and background research.

In the book’s final section, Gupta goes on to consider the underlying enabling conditions. Only 0.5% of the population used credit cards in 2007, when Indian Amazon-equivalent, Flipkart (now owned by Walmart), launched, leading the company to embrace shipping COD, so people could be sure they received their goods before paying. India moved directly from cash to online payments, bypassing the West’s intermediate steps and its legacy thicket of institutions and regulations. Unemployment, particularly among young, educated men is high. Platforms are largely indifferent – or worse, profit. Gupta highlights Apple and Google’s failure to purge their app stores of fake loan apps, which often rank higher than legitimate ones, and the 2025 Reuters report that Meta’s own internal projects expected 10% of its 2024 revenues to come from scam ads. The trust people award these platforms is a problem, too; Gupta herself was taken in by a false number that turned up in a Google search for her local wine shop. The ease of looking online and trust in Google overrode their own local knowledge.

Gupta finds other factors besides the many leaks and breaches of Indians’ data. India has digitized rapidly through the adoption of the Aadhaar identity system and the Unified Payments Interface, the source of more than two million reported cases of fraud worth more than $112 million in 2024. Indians’ “Internet” is a highly centralized handful of apps and platforms, mostly from US Big Tech companies. Social media has normalized interacting with and trusting strangers. The covid pandemic turned courts virtual, and impoverished many while opening people’s hearts and wallets to the needs of those strangers. And, she writes, India’s culture of shame leads many victims to try to save face rather than seek redress.

None of these have easy fixes. Gupta doesn’t seem to think it will have much effect to pass laws or regulate platforms, especially with the prospect of AI generating scams at scale. For self-protection, she recommends learning to pause and reconsider, and identify moments of vulnerability and suggests crossing India’s many divisions to educate those who who lack online experience, such as older rural women. And: “Build a more just society.”

Illustrations: Bharat Bluff.

Also this week: TechGrumps episode 3.42, Three thefts don’t make a right.

Change of plan

It feels like the beginning of an inflection point that plans for a gigantic data center are being abandoned after years of litigation trying to block it.

The story, as told by Etiido Uko at Tom’s Hardware, is that the last of several stakeholders has abandoned the Virginia Digital Gateway, which was going to be the world’s biggest data center. It died on a technicality: the public had not been properly notified before Prince William county’s Board of Supervisors ruled to rezone the site.

The 2,100-acre project, Elroy Fernandes says at Startup Fortune, would have occupied 22 million square feet – per the developers, the equivalent of 144 Walmart supercenters. It was opposed by historical societies and local residents because it abuts the Manassas Civil War battlefield.

The site, Fernandes writes, was meant to expand the Northern Virginia Data Center Alley, which processes 70% of global Internet traffic. He suggests that the case sets a precedent that “land use fights…can kill a fully approved project after years of sunk legal and engineering costs”, is a new bottleneck to add to access to power.

Protesters elsewhere may take heart. As Ed Zitron frequently rants, the cost, number, and size of data centers being planned is staggering, especially given their unpopularity. In Pennsylvania, where 66 data centers are planned or being built thanks in part to tax incentives, the Republican candidate opposing governor Josh Shapiro’s reelection is making a moratorium on data centers part of her platform. In London, the plan to convert an old brewery into a data center occupying 5,200 square meters to enable high-frequency trading is opposed by both residents and the local council, who would rather prioritize affordable housing. In Tennessee, Texas, and Seattle (where testifying Amazon employees say the company is investigating them), have passed moratorium bans to give local governments time to study the issues they raise.

Like Wall Street in 2011 (Occupy) and Google’s buses in 2016 (see Douglas Rushkoff’s Throwing Rocks at the Google Bus), data centers provide physical targets that consolidate the spreading anger over growing inequality, Silicon Valley values, climate change, and AI in general. This could be an unpleasant fight going forward: the US FBI is beginning to fret about anti-tech extremism.

***

It seems that two 15-year-old boys were having fun in a Waymo robotaxi in San Mateo, California, drinking alcohol and firing Orbeez out of the windows until they were detained by police. According to AP News, Waymo – presumably a remote human *at* Waymo – pulled the car into a parking lot and called the cops. The San Mateo police department said the car remained unlocked and the teens could have left the vehicle.

The reactions are many and varied. One friend calls it “kidnapping”. The Register calls the car a snitch. Another friend calls Waymo’s behavior entirely justified and says the teens got what they deserved. The Daily Mail calls the car a “tattle-tale”. A separate issue is the police response: four officers with guns drawn.

At The Register, Connor Jones notes the open question: how were the teens able to rent the car? This in itself is interesting because a few years back, everyone imagined that self-driving cars would open up unaccompanied car travel to people who can’t drive – like kids and visually impaired people.

The incident opens a box of worms. Waymo’s in-car cameras and mics enabled staff to detect what was happening, divert the car, and call police with its GPS coordinates. There will be much debate about finding the line between stopping dangerous behavior and violating passengers’ privacy. The discussions we’ve had for the last decade about social media will now find their analogue in the physical world.

***

In February 2007, I wrote about legal actions brought by a once-large company, the Santa Cruz Operation (SCO).for the Guardian summarizing the complex history because the case was about to end. SCO’s four-year-old claim – that IBM had infringed its copyrights on UNIX by contributing code to Linux – had spread to Red Hat and the now-defunct networking company Novell.

By August, it was over: the judge ruled that SCO had no claim to UNIX code.

And then it wasn’t. Instead…the sadly defunct Groklaw went on covering the case in detail until 2013. In 2016, I noted that somehow “SCO” (trying to follow “SCO” through mergers and owners is like trying to win the shell game) was paying Boies, Schiller, and Flexner to file yet another appeal. That appeal, Wikipedia says, was dismissed in 2018, and it was finally settled in 2021.

But hark! What’s this thing rising, clanking and covered with slime? On Monday, Simon Sharwood reported that by the 2021 settlement “SCO” had sold its software to Xinuos, and…somehow it’s all alive again. In a hearing on June 22 – 2026! – they argued about whether Xinuos has the right to litigate this at all. One assumes someone hopes that IBM or Red Hat could be forced to pay a lot of royalties *if* their versions of Linux are ever found guilty of infringement.

This is our modern Jarndyce and Jarndyce. As Dickens said, “a scarecrow of a suit”.

Illustrations: Manassas Battlefield National Park (via Wikimedia.)

International relations

The US Supreme Court’s decision this week in Trump v. Slaughter, giving the US president the power to control supposed-to-be independent agencies set up and funded by Congress, may be as profound internationally as it is domestically. In his newsletter, the economist Paul Krugman calls the president’s new power “dictatorial”, and notes the Court’s exception for the Federal Reserve. Krugman highlights the importance to average citizens of the Federal Trade Commission, which oversees consumer protection and the US’s meager privacy law. It is one of the agencies Trump now fully controls.

At day later, at Euractiv, Claudie Moreau reported that as a direct result Max Schrems and his NGO, noyb, is preparing a legal challenge to the three-year-old EU-US Data Privacy Framework.

Quick recap. This is the third time Schrems, an Austrian citizen, is challenging a formal legal arrangement for permitting data flows from the EU to the US. The saga began in 1998, when the EU Data Protection Directive, passed in 1995, came into force. To enable data flows to the US, the EU and US negotiated the Safe Harbour agreement. Then came Edward Snowden’s 2013 revelations of US intelligence spying. A bunch of lawyering later, in 2015, the European Court of Justice struck down Safe Harbour. In 2016, the EU and US replaced it with Privacy Shield, based on the US passage of the 2015 Judicial Redress Act, which granted non-US citizens limited rights to access redress in US courts for illegal data transfers.

Days after arriving in office in 2017, Trump issued an executive order demanding that agencies ensure that their privacy policies exclude those who are not US citizens. In a flurry at the in-progress Computers, Privacy, and Data Protection, many asked, had he just killed Privacy Shield? Schrems’ second case was already underway, and in 2020 he won again, when the European Court of Justice struck down Privacy Shield.

“There must be no Schrems III,” the Dutch MEP Sophie int’ Veld said a few months later. That possibility pervaded CPDP 2022. Yet all agreed the big issue was and is lack of enforcement.

The 2023 adoption of the Framework was enabled by the US creation of the Privacy and Civil Liberties Oversight Board to handle complaints from and redress for foreigners whose privacy rights have been violated. Last year, again days after taking office, Trump gutted the PCLOB. Many wondered then if the Framework could survive. Schrems commented, “This deal was always built on sand.”

The SCOTUS ruling this week granting US presidents free rein to control independent agencies like the FTC, noyb writes in a press release / blog posting, “…the entire structure of the EU-US Data Privacy Framework has just collapsed”. It also notes that EU treaty law requires such agencies to be independent – and counts 259 times that the EU relies on the FTC in its data flow decision.

“Even in the European Commission’s logic, the basis for any EU-US data transfer deal is dead,” Schrems concludes. Because the Framework must be actively repealed or ruled illegal by the courts, noyb is both filing a lawsuit and asking the European Commission to repeal it.

And so begins Schrems III.

***

This week I presented the talk I did at Greenwich Skeptics in April for the Cambridge Skeptics. Titled “What We Talk About When We Talk About AI”, the talk was an attempt to disentangle the different things people mean when they say “artificial intelligence”. The AI we have – generative AI, image generators, scoring systems, surveillance systems – has little in common with the original idea mooted by Alan Turing in 1950 or the problems the Dartmouth workshop sought to solve over the summer of 1956.

The founders and CEOs of AI companies, however, seem quite happy for us to conflate the two, since it makes them sound more worth investing in. Demis Hassabis, for example, founded Deep Mind (since 2014, part of Google) with the mission statement, “Solve intelligence. Then use that to solve everything else.” With respect to at leaast some intractable problems this is obviously nonsense. We have long known what needs to be done about climate change. All the intelligence in the universe will not create the political will to do the things we already know need to be done.

Many of the things we want from AI – automation, helpers, guardians, (sadly) weapons, companions – are things humans have wanted as long as there have been humans and have many precursors. Some of these constructs, both fictional (Asimov’s Laws, Rosey the Robot), and non-fictional (Arthur C. Clarke’s Laws of Science) come up routinely in academic and legal conferences to this day. I personally don’t believe today’s “AI” paradigm will lead to a superintelligence or a new form of consciousness (as net.wars readers probably already know). But the evidence is clear that today’s “AI” can do plenty of damage to today’s people and places.

The talk ended with a few thoughts about how to respond: resist the inevitability narrative; change the framing; be specific about naming systems, their purposes and owners; and count the opportunity costs.

The talk was not recorded and is not online, but the references are.

Illustrations: Max Schrems, in 2024 (via Murielle Gerber and EPFL media library at Wikimedia.

Also this week: At Plutopia, we talk to computer security professor Steve Bellovin about his newly-released free book on home security Don’t Get Hacked!.