net.wars

Journeys through online harms

X.AI – the company was-trillionaireElon Musk built in 2025 by smashing together X (the former Twitter) and xAI, which he founded in 2023 – is suing the state of Minnesota, claiming that its new law banning access to nudification technology violates the First Amendment. X.AI, which SpaceX swallowed early this year, argues that it’s already stopping this sort of thing and that the law is overbroad and will block constitutionally protected speech. At Techdirt, Mike Masnick argues the law is genuinely badly drafted.

I saw this story while reading The Chatbot Trap: How Digital Friends Can Become Life-Threatening, by Geertrui Mieke De Ketelaere, Logging Off: The Human Cost of Our Digital World, by Adele Zeynep Walton, and Users: How Big Tech Took Control and How to Fight Back, by Beeban Kidron. In the first, a Belgian software engineer and AI expert discovers the suicide-inducing tendencies of chatbots, investigates how they work, and builds an organization to push for regulation. In the second, a Zoomer who lost her sister following disturbing online interaction embraces a new balance between digital media and human connections. In the third, a member of Britain’s House of Lords journeys from filmmaker to tireless campaigner for child safety online. All three love technology and its benefits but oppose the business models and individuals who are deciding the direction of travel.

De Ketelaere’s book begins with an email: someone who saw her speak about AI wants help “to assert my rights as a human being” after her conversations with a chatbot turned disturbing – and after reading media stories connecting chatbot use to cases of suicide. De Ketelaere ends with ideas about regulation and ways to recognize the signs of an unhealthy situation.

At this year’s Computers, Privacy, and Data Protection, De Ketelaere warned of the danger of abruptly closing off intimate connections with bots because of the traumatic loss for the people concerned. As she says, even minor tweaks that favor the owning company’s business interests can undermine the relationships users experience.

De Ketelaere’s book is particularly strong in clearly describing the technical inner workings of these chatbots, revealed to her by careful study of the chatbot conversations her contacts supply. Few writing on this topic have presented such careful detail so readably.

Walton’s book begins with loss, and goes on to outline what she’s learned about the services her sister used and her own experiences as part of the generation who adopted and obsessed over Instagram as teens. Following chapters on social media and algorithms, she goes considers the impact of technology design on workers’ rights, individuals, and society more broadly through what she calls the “endless stream of content vomit”. Following some regulatory recommendations – consider social media addiction a public health issue, amend the Suicide Act 1961 to make companies liable, adopt safety by design, and make companies pay for the harms they cause – she concludes by considering how to reshape our relationship with technology so it serves us but does not dominate our lives.

In 2011, Charles Arthur‘s Social Warming argued that social media harms continue because no one chooses to stop them. Walton – like De Ketelaere and Kidron – angrily blames uncaring companies. It’s hard to disagree, although I take issue with Walton’s calling Section 230 of the 1996 Communications Decency Act “archaic”; granted, it was not written for today’s algorithmic feeds, but there would be no content moderation without it.

Kidron is well-known as a campaigner for children’s online safety. Yet all along she has repeatedly stressed that the privacy, autonomy, and protection children deserve from data grabs and online exploitation should be available to all of us.

Appointed to the House of Lords as a crossbench peer in 2012, Kidron’s interest began with seeing her 15-year-old daughter and three friends together focused solely on their own phones. Her subsequent investigation became her 2013 documentary InRealLife and inspired her concurrent creation of the 5Rights Foundation. The film is notable for the extraordinary trust and confidence she builds with the many young people she interviews; listening to them was crucial.

Her book is long and loaded with detail on her efforts and frustrations in working on legislation and amendments (making the book’s lack of an index infuriating). Disabused of the hope that a film would do the job, she travels the world, pushing everyone from Nobel Laureates to Silicon Valley CEOs to politicians to popes (plural) to take Internet harms seriously. She explains well the lobbyists’ playbook as she hopes for the industry’s “tobacco moment”. Late on, Oprah Winfrey links the lack of change to the silence after her show on sexual abuse, saying that parents don’t believe it will happen to their child, and they don’t want the fear. In response, Kidron attempts to show a positive vision, including personal changes to reclaim our lives.

Kidron’s tale largely ignores civil society other than the Campaign for Countering Digital Hate and Max Schrems. Yet she has more in common with groups like ORG than she may realize: many strange bedfellows are converging on digital sovereignty to enable resistance against the terms dictated by a distant, self-interested country.

Illustrations: Children playing, by Hubert-François Gravelot, 1760-1761 (via Smithsonian).

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

Escaping the sandbox

This week, France became the first European country to pass a social media ban. Under-15s will be blocked from opening new accounts on social media from September, and banned entirely from January 2027. As the BBC notes, implementation will require everyone in France to prove they are over 15 to use social media. Other European countries may follow.

At the New York Times, Mark Landler reports that similar laws are in progress in Canada, Italy, Poland, Portugal, and Spain. The EU is considering a bloc-wide law. The UK announced a ban to begin in 2027 last month and published a fact sheet last week – unless new prime minister Andy Burnham changes tack, as he has by canceling digital IDs.

At Le Monde, Florian Reynaud traces the origins of this whole idea of social media bans to a 2019 article in The Atlantic by Conor Friedersdorf. In it, Friedersdorf proposed not only a ban on social media sites for kids but a separate kid-friendly “Internet” with services designed just for them (this part was actually not a new idea).

A fundamental problem, no matter what restrictions or sandboxes you want to create, is that social media bans require sites to determine who’s allowed in and who isn’t. So, no social media ban without age verification, which the UK had already introduced.

Worth noting: at the Guardian, Amelia Gentleman interviews Candice Odgers, who has concluded from *talking to kids* that social media bans will make teens’ problems worse, not better. At The Conversation, Andy Phippen, who also listens to children as well as parents, is equally dubious.

Because Australia was first, everyone looks at its results. The BBC reported in April, four months after Australia’s social media ban began, that 61% of 12 to 15-year-olds still had access to one or more accounts and that a little over half of those who had used social media before the ban believed the ban was making no difference to their online safety. In July, researchers found their consumption of news had dropped. The Independent offered uncertainty. A few weeks ago, at the Guardian Anna Bawden reported that a newer study from the University of Newcastle published in the British Medical Journal finds that 85% of Australian 12 to 17-year-olds are still using social media, more than half of them through their own accounts.

Numbers are easier to assess than emotional states, which means that whether the bans “work” is more easily and immediately measured by how many kids evade the ban – and this in turn means next steps will inevitably be aimed at shrinking that number without waiting the years necessary to assess the real results.

France is already on it, aiming to restrict VPNs, as Skye Jacobs reports at Techspot. At Techdirt, Mike Masnick warns that Australia is, too. And so is the EU, to prevent kids from bypassing its easily hacked age verification app.

For now, the UK is holding out. Last week, the UK’s then-minister for online safety, Kanishka Narayan, told BBC Breakfast News that the government has decided against banning VPNs since they have many privacy and security uses. This was later confirmed in writing by then-technology secretary Liz Kendall. In the week since, Kendall’s department has been abolished, and Narayan has been made the UK’s first minister for artificial intelligence. So we’ll see if at least that amount of sense continues to prevail.

***

Much excitement this week when OpenAI announced that in a cybersecurity test an unreleased model broke out of its sandbox and mounted a cyber attack on Hugging Face, a community for sharing machine learning models and datasets. At his blog, Simon Willison analyzes what happened in more detail.

If you’ve read Janelle Shane’s book, You Look Like a Thing and I Love You, Willison’s explanation makes perfect sense. Shane documents many cases in which AI software, given a task, takes “shortcuts”. Because computers and software are literal (paperclip maximizers, these shortcuts are sometimes surprising to humans. Here, OpenAI removed the guardrails, locked the model in a sandbox, and told it to solve a bunch of benchmark problems. The model “cheated” – that is, instead of working directly on the problems, it found a vulnerability in the sandbox that let it access the public Internet, and attacked Hugging Face as a likely source of the problems’ solutions. If it were a new-model car kept in a locked parking lot that turned itself on, crashed through the barriers, and drove into a house across the street, it wouldn’t seem so mystical; we’d expect the company to revamp the car’s electrical system, strengthen the barriers, and pay for the damage.

Willison goes on to argue that a key issue is asymmetry of knowledge – that is, that the guardrails that protect us from “our” AI software don’t apply elsewhere, so blocks on Chinese AIs prevent us for finding the vulnerabilities in our own software and services that they can. At last, a sensible reason for caring if another country is “winning the AI race” (whatever that really means).

Illustrations: Children swimming in the summer, by Japanese artist Ando Hiroshige, 1797-1858 (via Smithsonian collection.

Also this week: The Plutopia podcast talks with Rob van Eijk on digital sovereignty, AI, and privacy in the EU.

Change of plan

It feels like the beginning of an inflection point that plans for a gigantic data center are being abandoned after years of litigation trying to block it.

The story, as told by Etiido Uko at Tom’s Hardware, is that the last of several stakeholders has abandoned the Virginia Digital Gateway, which was going to be the world’s biggest data center. It died on a technicality: the public had not been properly notified before Prince William county’s Board of Supervisors ruled to rezone the site.

The 2,100-acre project, Elroy Fernandes says at Startup Fortune, would have occupied 22 million square feet – per the developers, the equivalent of 144 Walmart supercenters. It was opposed by historical societies and local residents because it abuts the Manassas Civil War battlefield.

The site, Fernandes writes, was meant to expand the Northern Virginia Data Center Alley, which processes 70% of global Internet traffic. He suggests that the case sets a precedent that “land use fights…can kill a fully approved project after years of sunk legal and engineering costs”, is a new bottleneck to add to access to power.

Protesters elsewhere may take heart. As Ed Zitron frequently rants, the cost, number, and size of data centers being planned is staggering, especially given their unpopularity. In Pennsylvania, where 66 data centers are planned or being built thanks in part to tax incentives, the Republican candidate opposing governor Josh Shapiro’s reelection is making a moratorium on data centers part of her platform. In London, the plan to convert an old brewery into a data center occupying 5,200 square meters to enable high-frequency trading is opposed by both residents and the local council, who would rather prioritize affordable housing. In Tennessee, Texas, and Seattle (where testifying Amazon employees say the company is investigating them), have passed moratorium bans to give local governments time to study the issues they raise.

Like Wall Street in 2011 (Occupy) and Google’s buses in 2016 (see Douglas Rushkoff’s Throwing Rocks at the Google Bus), data centers provide physical targets that consolidate the spreading anger over growing inequality, Silicon Valley values, climate change, and AI in general. This could be an unpleasant fight going forward: the US FBI is beginning to fret about anti-tech extremism.

***

It seems that two 15-year-old boys were having fun in a Waymo robotaxi in San Mateo, California, drinking alcohol and firing Orbeez out of the windows until they were detained by police. According to AP News, Waymo – presumably a remote human *at* Waymo – pulled the car into a parking lot and called the cops. The San Mateo police department said the car remained unlocked and the teens could have left the vehicle.

The reactions are many and varied. One friend calls it “kidnapping”. The Register calls the car a snitch. Another friend calls Waymo’s behavior entirely justified and says the teens got what they deserved. The Daily Mail calls the car a “tattle-tale”. A separate issue is the police response: four officers with guns drawn.

At The Register, Connor Jones notes the open question: how were the teens able to rent the car? This in itself is interesting because a few years back, everyone imagined that self-driving cars would open up unaccompanied car travel to people who can’t drive – like kids and visually impaired people.

The incident opens a box of worms. Waymo’s in-car cameras and mics enabled staff to detect what was happening, divert the car, and call police with its GPS coordinates. There will be much debate about finding the line between stopping dangerous behavior and violating passengers’ privacy. The discussions we’ve had for the last decade about social media will now find their analogue in the physical world.

***

In February 2007, I wrote about legal actions brought by a once-large company, the Santa Cruz Operation (SCO).for the Guardian summarizing the complex history because the case was about to end. SCO’s four-year-old claim – that IBM had infringed its copyrights on UNIX by contributing code to Linux – had spread to Red Hat and the now-defunct networking company Novell.

By August, it was over: the judge ruled that SCO had no claim to UNIX code.

And then it wasn’t. Instead…the sadly defunct Groklaw went on covering the case in detail until 2013. In 2016, I noted that somehow “SCO” (trying to follow “SCO” through mergers and owners is like trying to win the shell game) was paying Boies, Schiller, and Flexner to file yet another appeal. That appeal, Wikipedia says, was dismissed in 2018, and it was finally settled in 2021.

But hark! What’s this thing rising, clanking and covered with slime? On Monday, Simon Sharwood reported that by the 2021 settlement “SCO” had sold its software to Xinuos, and…somehow it’s all alive again. In a hearing on June 22 – 2026! – they argued about whether Xinuos has the right to litigate this at all. One assumes someone hopes that IBM or Red Hat could be forced to pay a lot of royalties *if* their versions of Linux are ever found guilty of infringement.

This is our modern Jarndyce and Jarndyce. As Dickens said, “a scarecrow of a suit”.

Illustrations: Manassas Battlefield National Park (via Wikimedia.)

International relations

The US Supreme Court’s decision this week in Trump v. Slaughter, giving the US president the power to control supposed-to-be independent agencies set up and funded by Congress, may be as profound internationally as it is domestically. In his newsletter, the economist Paul Krugman calls the president’s new power “dictatorial”, and notes the Court’s exception for the Federal Reserve. Krugman highlights the importance to average citizens of the Federal Trade Commission, which oversees consumer protection and the US’s meager privacy law. It is one of the agencies Trump now fully controls.

At day later, at Euractiv, Claudie Moreau reported that as a direct result Max Schrems and his NGO, noyb, is preparing a legal challenge to the three-year-old EU-US Data Privacy Framework.

Quick recap. This is the third time Schrems, an Austrian citizen, is challenging a formal legal arrangement for permitting data flows from the EU to the US. The saga began in 1998, when the EU Data Protection Directive, passed in 1995, came into force. To enable data flows to the US, the EU and US negotiated the Safe Harbour agreement. Then came Edward Snowden’s 2013 revelations of US intelligence spying. A bunch of lawyering later, in 2015, the European Court of Justice struck down Safe Harbour. In 2016, the EU and US replaced it with Privacy Shield, based on the US passage of the 2015 Judicial Redress Act, which granted non-US citizens limited rights to access redress in US courts for illegal data transfers.

Days after arriving in office in 2017, Trump issued an executive order demanding that agencies ensure that their privacy policies exclude those who are not US citizens. In a flurry at the in-progress Computers, Privacy, and Data Protection, many asked, had he just killed Privacy Shield? Schrems’ second case was already underway, and in 2020 he won again, when the European Court of Justice struck down Privacy Shield.

“There must be no Schrems III,” the Dutch MEP Sophie int’ Veld said a few months later. That possibility pervaded CPDP 2022. Yet all agreed the big issue was and is lack of enforcement.

The 2023 adoption of the Framework was enabled by the US creation of the Privacy and Civil Liberties Oversight Board to handle complaints from and redress for foreigners whose privacy rights have been violated. Last year, again days after taking office, Trump gutted the PCLOB. Many wondered then if the Framework could survive. Schrems commented, “This deal was always built on sand.”

The SCOTUS ruling this week granting US presidents free rein to control independent agencies like the FTC, noyb writes in a press release / blog posting, “…the entire structure of the EU-US Data Privacy Framework has just collapsed”. It also notes that EU treaty law requires such agencies to be independent – and counts 259 times that the EU relies on the FTC in its data flow decision.

“Even in the European Commission’s logic, the basis for any EU-US data transfer deal is dead,” Schrems concludes. Because the Framework must be actively repealed or ruled illegal by the courts, noyb is both filing a lawsuit and asking the European Commission to repeal it.

And so begins Schrems III.

***

This week I presented the talk I did at Greenwich Skeptics in April for the Cambridge Skeptics. Titled “What We Talk About When We Talk About AI”, the talk was an attempt to disentangle the different things people mean when they say “artificial intelligence”. The AI we have – generative AI, image generators, scoring systems, surveillance systems – has little in common with the original idea mooted by Alan Turing in 1950 or the problems the Dartmouth workshop sought to solve over the summer of 1956.

The founders and CEOs of AI companies, however, seem quite happy for us to conflate the two, since it makes them sound more worth investing in. Demis Hassabis, for example, founded Deep Mind (since 2014, part of Google) with the mission statement, “Solve intelligence. Then use that to solve everything else.” With respect to at leaast some intractable problems this is obviously nonsense. We have long known what needs to be done about climate change. All the intelligence in the universe will not create the political will to do the things we already know need to be done.

Many of the things we want from AI – automation, helpers, guardians, (sadly) weapons, companions – are things humans have wanted as long as there have been humans and have many precursors. Some of these constructs, both fictional (Asimov’s Laws, Rosey the Robot), and non-fictional (Arthur C. Clarke’s Laws of Science) come up routinely in academic and legal conferences to this day. I personally don’t believe today’s “AI” paradigm will lead to a superintelligence or a new form of consciousness (as net.wars readers probably already know). But the evidence is clear that today’s “AI” can do plenty of damage to today’s people and places.

The talk ended with a few thoughts about how to respond: resist the inevitability narrative; change the framing; be specific about naming systems, their purposes and owners; and count the opportunity costs.

The talk was not recorded and is not online, but the references are.

Illustrations: Max Schrems, in 2024 (via Murielle Gerber and EPFL media library at Wikimedia.

Also this week: At Plutopia, we talk to computer security professor Steve Bellovin about his newly-released free book on home security Don’t Get Hacked!.

Technology distinguishable from magic

This week, a regional court in Munich issued what may be a landmark decision: it ruled that Google is liable for the content of its “AI Overviews”. The court was careful to distinguish between these, which Google’s algorithms generate, and search results, which link to content on third-party websites Google does not control. In other words, Google owns its own mistakes. The court awarded 80% of costs to the plaintiffs, two Munich-based publishers who claimed that the company spread damaging false information about them by linking them to scams in these AI-generated summaries.

As Max Bastian writes at Decoder, the ruling could set precedents for other companies. Google has yet to comment, but presumably will appeal. Away from AI is certainly not the direction the company wants to go; it’s spent the last couple of years building generative AI more and more deeply into its search, hoping to keep users locked in instead of chasing off to other sites. (Ironically, bouncing users off to other sites was the reason Yahoo refused to buy it in 1998, when the received business model was keeping users on your own site as long as possible.)

Google tried to argue that users should do their own fact-checking. But, as the court seems to have understood, where search results send you to the source page, AI Overviews look complete and don’t always offer sourcing to check. A study conducted for the New York Times found in an analysis of more than 4,000 searches that AI Overviews produced using the Gemini 2 model were accurate 85% of the time, rising to 91% when using Gemini 3 – about average for these systems. Enter the Law of Truly Large Numbers: 91% only sounds pretty good until you multiply the remaining 9% by billions to calculate the millions of wrong answers being disseminated every single day. Oumi, the startup that performed the analysis, found the AI Overviews included sources such as Facebook and Reddit posts, drew incorrect information even from authoritative sources, and are prone to manipulation. The article notes that Google disputes the analysis, saying that the benchmarks were developed by OpenAI and themselves contain inaccuracies.

The Munich court ordered Google to stop repeating the claims about the publishers, and awarded the publishers 80% of costs. It also rejected Google’s attempt to frame the issue as one of freedom of speech, calling the AI Overviews, “above all an expression of Google’s business activities”.

One reason the judge’s ruling is so significant is that most approaches for dealing with misinformation that have been mooted to date are at human, instead of computer, scale. Fact-checking, for example, while valuable, moves very, very slowly, one claim at a time. If the ruling stands, it will help tackle this type of misinformation at source.

***

Politicians like to talk as if the moon they want is available if the industry would just stop being obstructive. With AI’s capabilities in headlines everywhere, they are now demanding that phones should block children from taking, viewing, or sharing nude photos. This is the policy Keir Starmer announced this week in a speech, based on claims from the British company SafetoNet. The government has since provided more detail. Mic Wright has a round-up of press reactions. At the Daily Telegraph, Big Brother Watch director Silkie Carlo provides a strong civil liberties objection.

So far, neither Apple nor Google has said much. At New Scientist, Chris Stokel-Walker notes that both companies already have some controls in place, but spreading them through third-party apps poses challenges, especially as some phones’ operating systems aren’t recent enough to have the more sophisticated parental controls in the first place.

The moral may be: if you tell people your technology is magic, don’t be surprised when they expect it to *be* magic.

***

This week I had to verify my identity for Companies House. This is supposed to be a straightforward matter of creating a Gov One login, entering some details of a government-issued ID, and uploading a photo. The website was discontented: it couldn’t find my address, (is my century-old home too old to be in the database?), and didn’t accept the details I entered. Eventually, it offered two alternatives: use a phone app, or present myself in person at a preselected post office.

The app balked. It couldn’t open its links even though I’d authorized it. So, in this year of two thousand and twenty-six I got on a train to go to the nearest remaining post office that could perform the necessary rituals to show them first a QR code to access my application and then the ID, whose information is digitally held but had to be retyped on the post office tablet, and finally pose for a photo for the system – not the post office human – to compare and match. Some of those steps took several tries to mollify the system. Naturally, they don’t report whether you’ve passed until after you’ve gone home.

These are the people who want to create a digital ID infrastructure. I can only assume that if they ever get that system up and running actually using it will involve faxing things because by then all the post offices will be gone.

Illustrations: “The Magic Lantern”, by Auguste Edouart, circa 1835 (via The Met); at one time we thought that technology was magic.

Also this week:
At the Plutopia podcast, we talk to about her new book, Bad Influence.
The TechGrumps podcast episode, 3.41: The KardashElons of AI.

The sovereignty paradox

The year since the 2025 Computers, Privacy, and Data Protection has made Europe more distinct as an entity. Two years ago, we were being chastised for paying insufficient attention to structural corporate power. At this conference last year, we were warned about “simplification”, since published as the Digital Omnibus that threatens to undo some aspects of data protection and other privacy rights.

This year, we heard a lot about “European values”. Invoked as a metric: does “simplification” measure up or is it a sign of weakening? Invoked as a frame to guide developing the digital euro. Invoked as a reason why digital sovereignty is increasingly essential. In November 2025, 23 European countries signed a declaration embracing the latter as a policy.

And yet, although Plixavra Vogiatzoglou introduced a panel discussion by calling digital sovereignty “urgent because of Trump’s trade wars”, she then said it was unrealistic.

There’s a lot to like in the 2025 declaration, which lists as principles open source solutions, the principle of common assets, competitive markets, and democracy. The problem, as Vogiatzoglou said, is that Europe doesn’t make all the necessary components for building its own bottom-to-top stack. The investment required is likely to favor the wealthiest countries, deepening the imbalances that already favor US technology companies.

This is the “sovereignty paradox,” as Zuzanna Warso called it. Replicating the current infrastructure with little change other than a different owner isn’t the right goal – as someone said later, who wants a European Palantir? Warso favors redesigning the technical ecosystem to foster the “digital commons” – and being honest about the tradeoffs.

The same theme reappeared in a discussion of agentic assistants: they will ultimately sit on infrastructure belonging to the same few hyperscalers. Frederika Kaltheuner posited three scenarios: full vertical integration (like Google), integrated models and software (Anthropic and Claude Code), or open source and smaller models, which she thought was Europe’s only opportunity for sovereignty.

All of this seems set to get worse with agentic AI, which, Apple’s Gary Davis said, will allow agents onto all our devices capable of listening, observing, inferring, and acting across apps. What controls do we want? I personally want the control of barring this proposed technological future from my life, but how many of us will have that choice? Davis also noted the European Commission’s release, a few weeks ago, of proposals for requiring Google to allow competing AI services onto Android. If they follow through, he said, it will allow a large-scale privacy and security experiment on European users. While Davis’s employer has its own rasons for opposing this, he has a point. This is untried technology controlled by a handful of companies that could give them overwhelming power over individuals.

Among other new threats to privacy was eye tracking, a constant reality in games played with virtual reality headsets that could easily spread more widely via augmented reality smartglasses. The only way for Europe to counter this, Michael Raschke said, is to create large market-leading companies to act as gatekeepers to intermediate to meet European expectations of security and privacy.

A discussion of the digital euro had this same backdrop: part of the point is to reclaim some of the payments business from US giants Visa and Mastercard. Although, that’s over-simplified: the plans include offline and online versions of the digital euro which do different things. The offline version is meant as a digital reinterpretation of cash that allows anonymous person-to-person payments. The online version is…well, it’s hard to distinguish it from a bank transfer, except that “central bank digital currency” makes stuffy old banks sound kind of cool? Or it did when “crypto” was new and hot. The British equivalent, the digital pound, is in the design phase.

Those thoughts made Leon Schumacher‘s spirited intervention satisfying: where, he asked, was future-proofing against quantum computing, or accommodation for agentic AI, which is expected to underpin…well, no one knows quite how much in transactions, but they’re willing to guess. A trillion dollars, says McKinsey; up to $17.5 trillion, thinks Deloitte, $190 to $385 billion, per Morgan Stanley. Weirdly they all agree on *when*: by 2030. The digital euro is intended to arrive in 2029.

Meanwhile, there is simplification, which has few fans in the privacy world. As Orla Lynskey noted, simplification doesn’t *have* to mean deregulation – however, the European Commission’s proposals reduce rights, facilitate more data processing and *don’t* simplify. Plus, she added, changes of this magnitude require more time for thought and care.

Even business folk present, such as Spanish company founder Alicia Asín Pérez, thought deregulation was less important than many other constraints on business on her list. “My concern is, who area we deregulating for? What will be the consequences?” she said.

What most people favored instead is less fragmentation, and enforcement of the laws we have, a long-running theme at this event.

The former MEP Sophie in t’ Veld had a different take on European values. “We are obsessed with the US,” she said. “But it’s happening right here.” She called out government leaders for ignoring orders from the European Court of Justice, courts for rubber-stamping requests to target journalists with spyware, and called governments not using spyware complicit by their silence.

Illustrations: EU flag (via Wikimedia).

The soul in the machine

One of the first things skeptics learn is to never assume that paranormal belief implies stupidity. Smart people believe questionable things all the time; intelligence is different from the ability to assess your own cognitive biases, especially when you are working outside your field of expertise.

The astronomer Carl Sagan, one of 26 founders of the Committee for Skeptical Inquiry hinted at this in saying that the more you want to believe something the more careful you have to be about assessing the evidence. “Extraordinary claims require extraordinary evidence,” he often said, and he was right.

This week, the evolutionary biologist and author Richard Dawkins announced he thinks “his” AI is conscious, based on a couple of days’ interaction with Anthropic’s Claude chatbot. Inevitably, someone – Matthew Sheffield at Flux – has called the story “The Claude Delusion”. Dawkins has some company; at The Register, Liam Proven reports an engineer’s similar belief, and at the Independent Holly Baxter finds several more among company CEOs.

At Unherd, where he published his account, Dawkins begins with the “imitation game”, the test Alan Turing proposed in his 1950 essay, Computing Machinery and Intelligence (PDF). Turing, who adapted the test from one intended to differentiate men and women, suggested that relying on remote communication via text would eliminate unfairness to the machine, which obviously lacks human physical capabilities. The basic idea is that the mAchine passes the test if the human judge, given a transcript of the conversation between human and machine, can’t tell which is which.

It’s clear that chatbots can pass the Turing test. What that teaches us is not that chatbots can think but that Turing’s test is the wrong tool for assessing that. What chatbots have actually shown is that Turing’s test is the wrong tool for assessing whether something can think. As James Boyle memorably wrote, “Sentences do not imply sentience”. This profound change will take time to understand. In the meantime, it’s going to fool a lot of people. Although, as a science fiction writer friend once said, “You only have to look at a baby…”

In his essay, Turing outlined his own beliefs relating to his central question. He thought that in 50 years (that is, by 2000), it would be possible to program computers so that an average questioner would have only a 70% chance of making the right identification after five minutes. He then went on to consider many different types of objections to this belief, and to lay out his case. Absent are two factors we now know are crucial: the psychology of the human questioner and judge, and the business model of the machine’s owner.

The last few years have taught us both the capabilities and the flaws in chatbots: they provide plausible answers; they frequently generate entirely wrong information; and they are sycophantic and prone to output text that flatters their human questioner. So it’s easy to find a natural explanation for Dawkins’ belief that “his” AI is conscious: he is anthropomorphizing a stochastic parrot simulation that issues realistic and flattering responses. The simplest explanation, per Occam’s Razor, is that the consciousness exists solely between keyboard and chair.

Tangentially, the fix OpenAI has proposed for outputting entirely wrong text, Wei Xang writes at Science Alert, would also help make it clearer to users that generative AI is not sentient: introduce confidence intervals to expose the uncertainty derived from the gaps in the training data that generate unfounded guesses.

Google DeepMind engineer Alexander Leichner apparently agrees; this week, Emanuel Maiberg reports at 404 Media, he published a paper arguing that large language models will never be conscious. The biologists and philosophers Maiberg quotes agree with this conclusion – and point out decades of similar conclusions in their disciplines over decades.

The claim that a human-made a bunch of computers processing inputs is sentient is truly extraordinary. We forget this, because we have all read and watched so much science fiction with sentient, emotional machines: Her; Ex Machina; Blade Runner; Marvin, the Paranoid Android); and the first fictional android I ever encountered, Daneel Olivaw in The Caves of Steel. I mention mostly movies because actors make machines so much more obviously soulful.

Extraordinary claims require proportionately extraordinary evidence. If we accept that the Turing test was inadequate, which is not moving the goalposts but *learning something*, how would we go about devising a scientific method for identifying sentience?

The Cambridge professor of communications Jon Crowcroft didn’t exactly propose one. But, he emailed, “What we do know (from cognitive neuroscientists and from AI software) is that you can actually look at the internal operations of a biological brain and of an AI software system, and you can see that in the biological case there are things going on that are some sort of process we might call consciousness, but in the AI case there is no such structure. Nor would you expect there to be because no-one programmed an AI to have such a feature. nor is it emergent. In animals (not just humans) consciousness has an evolutionary value. Things like theory of mind are part of social bonding which makes cooperative strategies, for predators and prey, more effective.”

In other words, what we have learned from all this is that Dawkins is human. Who knew?

Illustrations: Stable Diffusion’s rendering of stochastic parrots, as prompted by Jon Crowcroft.

Elsewhere this week:
This month’s Letter to America column at Skeptical Inquirer reviews Beyond Belief (Helen Pearson), Bad Influence (Deborah Cohen), and Sneeze (David Miles).

The railway and the balloon

Is AI more like a train or a hot air balloon? Veronica Paternolli and Ryan Calo asked at this year’s We Robot. Nineteenth-century hot air balloons were notoriously uncontrollable. The US legal system assigned strict liability: it was your fault if your balloon crashed in someone’s backyard, even if you did everything you could to prevent it. Railways were far more disruptive but also far more predictable, and therefore were liable only in cases of negligence. Which regime should apply to AI is an ongoing debate.

Calo and Patornolli also wondered if agentic AI could reverse 30 years of being forced to take on busy work companies formerly did for us. This “shadow work” encompasses everything from retrieving bank statements and completing reCaptchas to pumping our own gas. Actually, more than 30 years: in 1962, Agatha Christie’s Miss Marple complained that self-service supermarkets were replacing shopkeepers who served you. If companies want a negligence regime, Patornelli and Calo argue, they should deploy agentic AI to relieve us of the “sludge” instead of displacing jobs and aggregating wealth.

But would we believe them? Technologists have promised before that their products will up-end the balance of power and simplify our lives – some of them the same people and companies. The web browsers and search engines that promised a universe of information today are faithless agents serving their owners and developers. Why should agentic AI – if it’s ever trustworthy – be any different? Many of us want a life with less demanding devices – and agentic AI sounds like even more “relationship” work.

Underlying Patornolli’s and Calo’s argument, however, is a fundamental clash. Like Mireille Hildebrandt at a 2017 Royal Society meeting, they argue that law is purposely flexible so it can adapt to unforeseen circumstances and, even more important, contestable (otherwise, Hildbrandt said, it’s just administration). Computers, even dressed in “AI”, always have hard boundaries underneath. As Bill Smart explained here in 2016, no matter how “fuzzy” its logic, no computer can evaluate standards like the “reasonable woman“. No matter how “fuzzy” its logic, a computer will issue a ticket if you are going even just the tiniest fraction of a nanometer faster than the speed limit. Anti-doping authorities have a similar problem as Neil Robinson said in a recent episode of the Anti-Doping podcast: the extreme sensitivity of modern tests is catching people with no intent to dope.

Liability wasn’t the immediate problem in Tomomi Ota’s description of everyday life with a Pepper robot at home (YouTube), which she took shopping, to restaurants, and on public transit as part of the Robot Friendly project, An account that led AJung Moon to wonder if a future filled with robots is really desirable. The inevitability narrative would say we’re going to get it anyway, begging the questions of whether we have a) the resources to make billions of robots and b) where we would put them all.

Sometimes these things fail in the simplest ways: a close-up of a Pepper that has been used as a greeter shows broken fingers because it was not robust enough for the basic social protocol of shaking hands. In studying the integration of robots into customer service situations, Elsa Concas, Stefan Larsson, and Laetitia Tanqueray found staff consultation is essential. In a staged setting such as the Japanese “ramen and robots” Pepper Parlour, the robots were a draw for customers and appreciated by the staff, who were paid more. In an unstaged airport tourist information center, they were basically useless and ignored. A commenter noted the same is often true of the robots intended for elder care in Japan: most end up in a cupboard,

This theme was also picked up by Emily LaRosa, who studied the limits of explainability in automated apple picking. In this case of “epistemic injustice”, the neglect of local knowledge and ecological tradition led her to propose a “Curated Information Framework”. She concluded that trust in AI systems is not created by transparency on its own if that means handing over large amounts of inscrutable data, but by taking lived context into account – “situated transparency”.

LaRosa’s study echoed the paper Ota co-wrote with Rikiya Yamamoto, which derives new “laws of robotics” to update Isaac Asimov’s Three Laws, which can’t be programmed and whose fixed, “top-down” nature was what he needed in a story-telling device. The real world, they argue, requires principles built bottom-up from practical experience. Their selection: mutual respect, social membership, and co-evolution.

They have lots of competition. Moon counts more than 100 sets of principles and ethical frameworks published since 2018, many of which she says make assumptions debunked in the 2025 paper The Future is Rosie?” or as Paul Ohm and David Atkinson discussed, encoded in the benchmarks – documents used to define AIs’ behavior and priorities. This “latent rulebook”, they said, is increasingly secret.

Meanwhile, like explainability, the right to repair fails for AI, which changes constantly with software updates, networking, and interacting. Ryota Akasaka argued that current legal approaches don’t work for products that aren’t fixed and will lose everything they’ve accrued when “repaired” to their original state. When Ota was offered the opportunity to upgrade her development model Pepper, she declined in shock. Replacing your robot’s head, it seems, ends a beautiful friendship.

Illustrations: “Hidden Labour of Internet Browsing”, by Anne Fehres and Luke Conroy. Via A14 Media (CC-by-4.0).

Intimacy capitalism

Many non-human characteristics make AI attractive, Sue-Anne Teo said to open this year’s We Robot: endless patience, long and detailed memory, and sycophancy. I’m less certain about the last of those; lots of us react poorly to undisguised flattery. And yet: Stanford researchers agree with her that this is a thing. The AIs are certainly programmed to *try* to human-wash themselves: they use the perpendicular pronoun, “apologize” for errors, and type “you’re right”. And the Stanford folks’ research shows that users respond by becoming “more self-centered, more morally dogmatic”. Probably it’s easier for that to happen if you’re consulting the AI on a personal matter than if you’re just asking it to find an article you read once based on a few hazy memories of what it’s about. No chatbot has yet congratulated me on my choice of half-remembered reading material.

Teo’s vision of the business potential of AIs is part of a long-running theme at We Robot: the subscription service that terminates your relationship when you stop paying. The potential for emotional manipulation by a company that is programmed to maximize profits as if they were paperclips is as great as that of a spirit medium over a client who believes their only link to a beloved deceased person is through their belief in that medium’s ability to establish contact. When I suggest this, Teo says mediums don’t scale. True. But the potential for emotional dependence and manipulation for those individuals seems psychologically similar.

A couple of months ago, Kate Devlin, a professor of AI and society at Kings College London, talked more positively about human-AI relationships, arguing that those engaging in them are often not the archetypal lonely and isolated people we all imagine. Some are married – happily, they tell her. Still, she frequently reminds people “your AI does not love you back”. The same can be true in reverse. Here, a Japanese researcher with three Peppers at home is asked if she misses them when she’s away. “No.”

As a psychologist, Devlin’s job is not business models. But they drive the AI’s design. Companies spend money in time and effort to make robots humanoid – or at least cute – to make them successful in the marketplace. The same is true of chatbots programmed to appear conscious. Cue (again) James Boyle: “For the first time in history…sentences do not mean sentience.” We are some way from having adapted to that.

Teo has a name for the peculiarly toxic mix of anthropomorphism, cold-eyed profit, data collection, and dark patterns that she’s ruminating on: “intimacy capitalism”. New to me, but instantly compelling.

I can see where an academic must rigorously untangle this into a solidly-founded theory; Teo is still working out fully what it means. But the phrase resonates without that depth: the rapaciousness described by surveillance capitalism and surveillance pricing crossed with the new ability to exploit personal vulnerabilities exposed by those same non-human characteristics of infinite patience and a long, detailed memory. Ugh.

I wish I could say that people do not respond as well to the blandishments of synthetic pretend-humans, but the statistics are against me. Worse, a study referenced in discussion found that people award authority to AI companions’ pronouncements because they trust them – which sounds to me like exactly the same as trusting an online “influencer” on subjects where they have no expertise because they’re familiar and maybe got some random things right in the past. As skeptics found in studying years of psychic predictions, people remember the hits and forget the misses.

So while you or I might say, make the chatbot act like a chatbot instead of dolling it up in human signage, the business model, fed by popular preference, is against us. Related, Gizem Gültekin-Várkonyi, who presented a discussion of “robot literacy”, wants people to stop saying “the algorithm” is discriminatory or “the algorithm” makes a decision. “It is us,” she said, reminding me of Pogo.

The presumption is that loading these various toxicities into robots will be worse. I’m less sure; I think the cute but less human ones ought to have a better chance because the more humanoid ones are so obviously *not* human and more likely to fall into the Uncanny Valley.

But for how long? In the lunch break, someone was running a series of “pick the AI” image tests. Two breakfasts, side by side. One had perfectly presented fried eggs, a fruit medley with strawberries, and I think some potatoes. The other had frazzled fried eggs, baked beans, and, nestled next to them, a dead giveaway. What AI knows from black pudding?

By next year, or soon after, AI chatbots and image generators will have been fed data about black pudding (without ever tasting one). Similarly, someday in the future, crude robots will be both cuter and, possibly, more lifelike.

Would you trust your baby with one of those robots? Who is liable if it puts the baby in the washing machine? At that moment, as multiple legal opinions awaited voicing, the actual two-month-old baby in the room howled. Can robots have such exquisite comic timing?

Illustrations: Pepper, as seen at We Robot 2016.

Also this week: At Gathering4Gardner’s YouTube channel, mathematician and juggler Colin Wright and I talk about skepticism.

Long Island AI

The peak of dot-com mania was identifiable even at the time: it was in January 2000, when AOL bought Time Warner for $183 billion. Peak podcast mania arguably came in 2020 when Spotify paid Joe Rogan $100 million (per rumor). It’s now time to call the peak of AI mania: Allbirds, which was loved a few years back for making sustainable wool shoes …has sold off all stock and assets, and is rebranding as Newbird AI to sell “AI compute infrastructure”. At The Overspill, Charles Arthur compares this to 2017, when Long Island Iced Tea renamed itself Long Blockchain, arguably the peak of bitcoin mania. His theory that all Newbird AI has left is a bunch of empty warehouses, so hopes someone will put data centers in them is the only possibility that makes any sense.

Most of those bits of history led to more financial silliness and then a crash. The AOL-TimeWarner marriage was notoriously catastrophic. AOL, which was supposed to modernize old, slow Time-Warner, in fact was already becoming obsolete as users shifted to new technology (broadband) and the wider Internet. By 2003, the company was selling itself off in pieces. Long Island Blockchain cratered; a year later it had abandoned its plans to buy bitcoin mining equipment. Its delisting by NASDAQ was accompanied by the SEC’s charging three people with insider trading. Joe Rogan of course remains hugely popular and Spotify is doing fine, but it’s certainly not controlling the business of podcasting as it appeared to hope it would.

The day following the announcemenet, Newbird AI’s shares rose as much as 700% (briefly), partly on the additional news that it will close a funding round of $50 million in the second quarter of 2026. Even CNBC calls this “pivot” bizarre.

For our purposes, it doesn’t matter if this wacky strategy works (pick your definition of “works”), because when the share price of a basically assetless company goes up 700% because it’s added “AI” to its name we have reached the absurdity that marks the peak of every bubble.

It’s not the only sign (or the only absurdity). In the UK, last month Aisha Down reported at the Guardian that many of the efforts prime minister Keir Starmer – and Rishi Sunak before him – has announced to “mainline AI into the veins” of the British economy are based on what she calls “phantom investments”. She reported faithfully that the Department of Science, Innovation, and Technology said it “rejected these assertions”, but this week we learned that at least one piece of her reporting was absolutely correct.

This week’s news revolves around a project called Stargate, announced in September 2025 and involving the UK-headquartered AI infrastructure provider Nscale, Microsoft, Nvidia, and OpenAI. This week, OpenAI announced it was putting the project – for which it was supposed to build a data center – on hold. OpenAI blames energy prices and, as Joseph Bambridge reports at Politico, the government’s decision last month to shelve proposals to allow data miners to use copyrighted content unless its owner opts out. The proposal was widely opposed by the UK’s creative industry, and was indefinitely delayed in a report issued on March 18 (ReedSmith has a useful legal summary)..

The loss – or delay – of Stargate is a rounding error to companies the size of Microsoft and Nvidia. It’s more significant for Nscale, which according to CNBC raised $2 billion in a funding round just last month with investments from Nvidia, Dell, Lenovo, and other much less famous names; at the same time, it added former Google and Facebook ad business builder Sheryl Sandberg and former Facebook global policy head Nick Clegg to its board. The new funding raised Nscale’s valuation to $14.6 billion. At his blog, Ed Zitron calls the ability to raise funding for a data center that doesn’t exist “weird“, and suggests that AI companies should admit that their chatbots are just “regular old software”.

Meta king Mark Zuckerberg, last seen losing money on the former next-big-thing metaverse, is, Megan Bobrowsky reports at the Wall Street Journal, building an AI agent to get him information and answers faster. This comes on top of last month’s announcement that Meta is buying the AI agent network Moltbook, seemingly mostly in order to hire its two founders for Meta’s Superintelligence Labs. This week, Zuckerberg also announced he was building an AI clone of himself to interact with staff so they can feel more connected to him. Seems like in reality it would make corporate management feel like automated customer service.

The question about bubbles is always: is this one like railroads or like tulips? Tulips left nothing of value behind while railroads went on to be transformative. In 2001, almost everyone knew the Internet would go on growing in size and importance. In the AI case, despite the current silliness, over time we will learn how to use these new capabilities and limit the downsides. But first, we will have to deal with the fallout of the fact that the finances do not add up.

Illustrations: Tulips, (via Wikimedia).

Also this week:
At Skeptical Inquirer, I go to this year’s Gathering 4 Gardner.
At the Plutopia podcast, we interview Tereza Pultarova, who reports on developing military technology in Ukraine.