net.wars

Parting glances

So Britain will soon wave off yet another prime minister; they’ve been dropping like flies ever since the 2016 Brexit vote. At the Guardian, John Crace snarks that come Remembrance Sunday they’ll need more space around the Cenotaph for all the former prime ministers (nine, plus whoever’s next).

I had hoped for two things from the Labour government elected in 2024: 1) an end to the Conservative psychodrama, and 2) that as a former human rights lawyer. Starmer would modulate the government’s approach to surveillance, AI, and Internet regulation. Instead, he has continued seamlessly the line taken by the Conservative governments before him – and reinvented failed policies from the Labour government of the 1990s.

The last almost-two years have seen Starmer’s government plan digital IDs, pursue implementing the 2023 Online Safety Act including age verification, continue awarding contracts to Palantir, embrace AI for government services despite known flaws undermining fairness and accuracy, increase the hostility in the immigration system, tolerate the spread of police use of live facial recognition without public discussion, and continue adding restrictions on protest. With his premiership under threat, last week he announced the social media ban. It obviously didn’t save him, if that was the purpose. Now, we wait to see if it is taken up by whoever’s next.

At Index on Censorship, Sally Gimson says the ban, “bypasses the benefits of the Internet for children without actually tackling the risks”. She goes on to note that many teens, having grown up with this technology, are more privacy-conscious than their parents and less likely to spread scams than their grandparents.

The UK government’s relationship with technology needs a rethink, preferably away from former prime minister Tony Blair and his AI advocacy. We could start where Starmer could have: with rights-based values. And by recognizing that if you can’t solve tough social and economic problems by randomly throwing new technologies at them, you can’t solve them with random restrictions either.

While Starmer was resigning, Dan Robinson reports at The Register, the Department for Culture, Media, and Sport issued a green paper proposing to require social media and video platforms to promote trustworthy media over other sources. In “trustworthy”, the paper includes the UK’s traditional public service broadcasters, plus…well, who?

The green paper says the criteria will be “decided in an open and transparent manner with regard to protecting media freedom”. It then suggests that the definition of news publishers in the Online Safety Act provides a useful starting point: “both broadcasters and UK based entities whose primary purpose is publishing news-related material, created by different persons, subject to editorial control, a standards code, and with a mechanism for resolving complaints”.

Under that definition, it seems like the Daily Mail, which Wikipedia editors deem unreliable, is in, and personal blogs are out. Probably so are independent sites like LondonCentric, which is run by a single highly experienced journalist but pays freelances and does prize-winning investigative work. News podcasts can just wonder if they exist. In the reactions summarized at Nieman Lab, public service broadcasters love it, publishers like it if they’re included, and platforms say it’s unfair. Public comments are open.

Two things are interesting about the Wikipedia list. First, it is an ongoing collaborative effort. Second, the record of its construction shows that the contributors understand that an outlet may be trustworthy on some subjects and not others, deserve reevaluation over time, and vary according to topic and means of creation (see for example the comment about BusinessInsider’s AI-generated and syndicated content). Of course, Wikipedia is different. DCMS wants to improve the nation’s information diet. Wikipedia is merely trying to ensure that the sources cited for facts in the encyclopedia are reliable.

While I understand the dangers of a world choked with misinformation and disinformation, this definition sounds like an attempt to return to the old world of gatekeepers who, even if they couldn’t be trusted to keep the government’s secrets, were *familiar*. The Internet and other technologies have brought, for good and ill, the ability to publish and distribute to people who were previously unheard.

As the 2024 report from the Reuters Insitute says, news has no single trust problem (and therefore no single solution). Misinformation and disinformation are only a part.

For its report, Reuters convened 41 focus groups across Brazil, India, the UK, and the US, drawing participants from audiences who have been historically underserved, marginalized, or harmed by reporting. Previous work had already found generalized suspicion of all news media; the focus groups showed that these participants’r distrust was based in their history and the price was personal.

In last week’s 2026 report Reuters found that use of all media for news, even online news and social media, is declining, but at different rates – newspapers are fading faster than TV, for example. Reuters also found that in every country a “small but significant” minority say they use no news sources at all.

Starmer did some good things, as Jonathan Freedland writes at the Guardian, and none of this is what brought him down. But it’s an opportunity for the next guy.

Illustrations: Keir Starmer, passing the photographs of former prime ministers he is about to join (via Wikimedia.

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

Banned

Some policies are popular until people examine the details. This may be happening with social media bans like the one UK prime minister Keir Starmer announced this week. It will prohibit under-16s from using the main social media platforms or livestreaming, ban under-18s from using “romantic simulation” chatbots, and limit strangers’ ability to contact under-16s via direct messaging on gaming platforms. More detail will come in July; the government will also consider imposing a nightly curfew and requiring breaks in scrolling for under-18s. As so often, it’s possible to support the goals of policy proposals while disagreeing with the proposals themselves.

The BBC reports that 90% of the parents who responded to the recent consultation backed the ban (although the panel survey report is less clear-cut). Yet Ofcom’s May 2026 report shows more nuance: more than half of parents agree that the benefits of being online outweigh the risks. Narrowing that to social media use, 46% of parents still think the benefits outweigh the risks, dropping to 33% for parents of eight to 12-year-olds.

The move has been met with some skepticism from Cambridge psychologists, and even from some veteran child safety campaigners such as Jim Gamble.

The BBC reports that Ian Russell, the father of Molly Russell, a 14-year-old whose suicide in 2017 was attributed to viewing harmful content online, has called the ban a blunt instrument that will merely cause more problems. Russell argues for more thought, less haste.

At his blog, Lewis Goodall connects the haste to Starmer’s government’s precarity, which he thinks may doom the policy despite widespread concern about children online. Goodall, too, isn’t sure it’s the right policy. As we’ve also noted here before, this government is simultaneously pushing to lower the voting age to 16. At ConservativeHome, John Oxley points out the absurdity of banning these new voters from accessing social media to look up candidates’ policies. The teens the Guardian interviewed varied in their views.

It’s also true that today’s teens have less independence and fewer options for offline socializing than older generations did. As Alec Muffett writes on Bluesky, you cannot force 2020s children into 1980s childhoods because so much infrastructure is gone. When you take away online interaction, what’s left?

The Open Rights Group recaps 13 years of online child safety measures, beginning with ISP and mobile network filters in 2013 and ending with this week’s announcements. ORG argues that taking a systems view shows that these escalating online safety measures leave the underlying problem untouched: the feedback loop that ought to drive users away when they encounter awful content is broken.

Because Australia was the first country to implement a social media ban, it’s the model everyone looks at. There’s been a lot of discussion about whether the ban “works”, based on how well it’s keeping teens off social media. A survey of Australian parents found that two-thirds of teens still have social media accounts. Other research says that of those who’ve lost their accounts, half say the ban limits their access to news.

But is that what we should mean by “work”? By that standard, testing someone for allergies by eliminating specific foods would “work” if the person didn’t consume them. But what we want to know is whether the person is actually allergic to those foods, or, by analogy, whether the ban remediates the harms – depression, anxiety, and other mental health issues – that politicians claim to be worried about. A social media ban sounds simple; addressing climate change, the state of the economy, the cost of education, and the fear that there will be no jobs is hard.

ORG contends that enabling people to move between social networks at will, improving competition, and breaking up the platforms would do more to counter online harms than the present approach. This week also provided an example of how not to do this.

The newly launched, fully European social network “W Social” is based on the AT protocol that powers Bluesky, and limited to identified humans. Per Euronews, W Social is a privately owned Swedish startup whose investors are other European companies. In order to apply for an account, you must first provide to W Identity, a separate entity, your name, date of birth, phone number, address, passport, and photo. You then give permission to W Identity to give W Social your account number, date of birth, and passport country. You can only have one account.

Although Anna Zeiter, the Swiss CEO of W, which is a subsidiary of Sweden-based climate action platform We Don’t Have Time, is described as a “privacy expert”, the amount and sensitivity of the information being demanded is disproportionate, especially as there is barely any declining, especially among young people. What if they created a ban and nobody came?

Illustrations: Children swimming in the summer, by Japanese artist Ando Hiroshige, 1797-1858 (via Smithsonian collection.

Technology distinguishable from magic

This week, a regional court in Munich issued what may be a landmark decision: it ruled that Google is liable for the content of its “AI Overviews”. The court was careful to distinguish between these, which Google’s algorithms generate, and search results, which link to content on third-party websites Google does not control. In other words, Google owns its own mistakes. The court awarded 80% of costs to the plaintiffs, two Munich-based publishers who claimed that the company spread damaging false information about them by linking them to scams in these AI-generated summaries.

As Max Bastian writes at Decoder, the ruling could set precedents for other companies. Google has yet to comment, but presumably will appeal. Away from AI is certainly not the direction the company wants to go; it’s spent the last couple of years building generative AI more and more deeply into its search, hoping to keep users locked in instead of chasing off to other sites. (Ironically, bouncing users off to other sites was the reason Yahoo refused to buy it in 1998, when the received business model was keeping users on your own site as long as possible.)

Google tried to argue that users should do their own fact-checking. But, as the court seems to have understood, where search results send you to the source page, AI Overviews look complete and don’t always offer sourcing to check. A study conducted for the New York Times found in an analysis of more than 4,000 searches that AI Overviews produced using the Gemini 2 model were accurate 85% of the time, rising to 91% when using Gemini 3 – about average for these systems. Enter the Law of Truly Large Numbers: 91% only sounds pretty good until you multiply the remaining 9% by billions to calculate the millions of wrong answers being disseminated every single day. Oumi, the startup that performed the analysis, found the AI Overviews included sources such as Facebook and Reddit posts, drew incorrect information even from authoritative sources, and are prone to manipulation. The article notes that Google disputes the analysis, saying that the benchmarks were developed by OpenAI and themselves contain inaccuracies.

The Munich court ordered Google to stop repeating the claims about the publishers, and awarded the publishers 80% of costs. It also rejected Google’s attempt to frame the issue as one of freedom of speech, calling the AI Overviews, “above all an expression of Google’s business activities”.

One reason the judge’s ruling is so significant is that most approaches for dealing with misinformation that have been mooted to date are at human, instead of computer, scale. Fact-checking, for example, while valuable, moves very, very slowly, one claim at a time. If the ruling stands, it will help tackle this type of misinformation at source.

***

Politicians like to talk as if the moon they want is available if the industry would just stop being obstructive. With AI’s capabilities in headlines everywhere, they are now demanding that phones should block children from taking, viewing, or sharing nude photos. This is the policy Keir Starmer announced this week in a speech, based on claims from the British company SafetoNet. The government has since provided more detail. Mic Wright has a round-up of press reactions. At the Daily Telegraph, Big Brother Watch director Silkie Carlo provides a strong civil liberties objection.

So far, neither Apple nor Google has said much. At New Scientist, Chris Stokel-Walker notes that both companies already have some controls in place, but spreading them through third-party apps poses challenges, especially as some phones’ operating systems aren’t recent enough to have the more sophisticated parental controls in the first place.

The moral may be: if you tell people your technology is magic, don’t be surprised when they expect it to *be* magic.

***

This week I had to verify my identity for Companies House. This is supposed to be a straightforward matter of creating a Gov One login, entering some details of a government-issued ID, and uploading a photo. The website was discontented: it couldn’t find my address, (is my century-old home too old to be in the database?), and didn’t accept the details I entered. Eventually, it offered two alternatives: use a phone app, or present myself in person at a preselected post office.

The app balked. It couldn’t open its links even though I’d authorized it. So, in this year of two thousand and twenty-six I got on a train to go to the nearest remaining post office that could perform the necessary rituals to show them first a QR code to access my application and then the ID, whose information is digitally held but had to be retyped on the post office tablet, and finally pose for a photo for the system – not the post office human – to compare and match. Some of those steps took several tries to mollify the system. Naturally, they don’t report whether you’ve passed until after you’ve gone home.

These are the people who want to create a digital ID infrastructure. I can only assume that if they ever get that system up and running actually using it will involve faxing things because by then all the post offices will be gone.

Illustrations: “The Magic Lantern”, by Auguste Edouart, circa 1835 (via The Met); at one time we thought that technology was magic.

Also this week:
At the Plutopia podcast, we talk to about her new book, Bad Influence.
The TechGrumps podcast episode, 3.41: The KardashElons of AI.

The Sutton effect

One of the enduring questions in cybersecurity is how much failures cost and who pays. Many companies see cybersecurity as a cost with no return; as in housekeeping only the failures are noticeable.

Certainly, a data breach, bungled software update, or ransomware attack can ding a company’s share price in the short term – but a year later, often they seem to have fully recovered. Meanwhile, the company’s customers may have spent hours monitoring credit reports, replacing credit cards, and other admin to remediate the effects.

Take, for example, Crowdstrike. In July 2024, it rolled out a buggy software update to all its 29,000 clients, many of them large businesses. One of those was Microsoft, which automagically incorporated it into Windows. Result: widespread paralysis. Crowdstrike fixed the error in 79 minutes; it took the rest of the world days to fully recover as each affected machine had to be manually restarted.

The company’s shares soon recovered. In November 2024, Matt Kapko reported at Cybersecurity Dive that the company had retained almost all its customers (which could just be a sign of dangerous market concentration). Similarly, the 2017 Equifax breach didn’t move it out of the heart of consumer credit scoring.

Soon after the Crowdstrike outage, David Jones reported at Cybersecurity Drive estimates that it had cost Fortune 500 companies a collective $5.4 billion, and that only 10% to 20% of that was covered by insurance. At the same time, at Bank Info Security, Matthew J. Schwartz estimated the cost to cyberinsurers at $1.5 billion.

But what about the patients unable to book doctors’ appointments, the airline crews who lost work, the train passengers stuck on platforms? Or, in a data breach, the years-long worry about where the data is now and how it’s being used.

Cyberattacks on companies leave us with what Ryan Calo and Veronica Paternolli called “shadow work” at We Robot a couple of months ago. They proposed that agentic AI might be able to reverse 30 years of companies offloading work onto us. You might – though I doubt it – be able to trust agentic AI to automate generating requests for refunds and new credit cards or rebooking canceled airline flights. But no way will it enable you to recoup the lost hours in an airport, the stress of being unsure what happened, or the ongoing consequences of identity theft.

At this week’s Workshop on the Economics of Information Security, University of Michigan researchers Lina Alkarmi, Armin Sarabi, and Mingyan Liu called these imposed indirect costs the “social cost” of data breaches and noted that typically none of it is measured. In two of the three breaches they studied, their math indicated that the eventual settlements the companies paid to consumers was below their estimate of the lower bound of the actual cost.

An odd finding from their study of three major breaches is that the social cost dropped over the period they studied, 2008-2021. They suggest that the 2015 introduction (in the US) of chip and PIN helped lower the utility of the stolen data. They also surmise that the later breaches added less to an already-saturated black market for data. There is doubtless a lot more work to do on this. Nonetheless, they estimate the national social cost at $7 billion in 2021, for an average per victim of nearly $300.

In a second paper, University of Tulsa researchers Teyyub Mutallimov, Dana Itzhaki, and Tyler Moore examined the long-term impact on corporate results following cyber attacks, looking at financial statements rather than share prices There, it seems that companies don’t recover as fully as you might think. Depending on the type of attack – data breaches trigger financing and investment; ransomware attacks are operationally disruptive. Both involve ongoing costs: remediation, system upgrades, external advice, potentially legal settlements.

In the meantime, it remains unclear whether generative AI will be a net win or a net loss for cybersecurity – finding vulnerabilities, as Anthropic claims Claude Mythos does, exposes them to attackers, although it also offers developers an opportunity to close them (I recall a similar panic in 1995 when Dan Farmer released SATAN). A 2025 report from the Turing Institute found that AI had begun to accelerate crime by enabling it to scale more effectively and exploit personal vulnerabilities. In January, Carly Page reported at The Register that the cost to criminals of renting AI infrastructure was as cheap as a Netflix subscription, based on a paper from researchers at Group-IB. Self-hosted “dark LLMs” are optimized for creating scams and deepfakes for as little as $30 a month.

However, at WEIS, in another paper, Ben Collier, Jack Hughes, and Daniel Thomas studied vibe coding’s early impact on the cybercrime business. So far, they found, it doesn’t seem to be making much change; it’s not yet time to fear “vibercriminals”. One could even imagine that over time generative AI could disrupt the junior-level pipeline that produces senior, skilled workers, as it’s doing in other industries. On the other hand, there’s already long been a lot of automation at the lower levels. So, wash? But if something works, crime will adopt it. Cue Willie Sutton, whose name was invoked at WEIS several times to explain why people pursue cybercrime: “That’s where the money is.”

Illustrations: Willie Sutton (via FBI).

The sovereignty paradox

The year since the 2025 Computers, Privacy, and Data Protection has made Europe more distinct as an entity. Two years ago, we were being chastised for paying insufficient attention to structural corporate power. At this conference last year, we were warned about “simplification”, since published as the Digital Omnibus that threatens to undo some aspects of data protection and other privacy rights.

This year, we heard a lot about “European values”. Invoked as a metric: does “simplification” measure up or is it a sign of weakening? Invoked as a frame to guide developing the digital euro. Invoked as a reason why digital sovereignty is increasingly essential. In November 2025, 23 European countries signed a declaration embracing the latter as a policy.

And yet, although Plixavra Vogiatzoglou introduced a panel discussion by calling digital sovereignty “urgent because of Trump’s trade wars”, she then said it was unrealistic.

There’s a lot to like in the 2025 declaration, which lists as principles open source solutions, the principle of common assets, competitive markets, and democracy. The problem, as Vogiatzoglou said, is that Europe doesn’t make all the necessary components for building its own bottom-to-top stack. The investment required is likely to favor the wealthiest countries, deepening the imbalances that already favor US technology companies.

This is the “sovereignty paradox,” as Zuzanna Warso called it. Replicating the current infrastructure with little change other than a different owner isn’t the right goal – as someone said later, who wants a European Palantir? Warso favors redesigning the technical ecosystem to foster the “digital commons” – and being honest about the tradeoffs.

The same theme reappeared in a discussion of agentic assistants: they will ultimately sit on infrastructure belonging to the same few hyperscalers. Frederika Kaltheuner posited three scenarios: full vertical integration (like Google), integrated models and software (Anthropic and Claude Code), or open source and smaller models, which she thought was Europe’s only opportunity for sovereignty.

All of this seems set to get worse with agentic AI, which, Apple’s Gary Davis said, will allow agents onto all our devices capable of listening, observing, inferring, and acting across apps. What controls do we want? I personally want the control of barring this proposed technological future from my life, but how many of us will have that choice? Davis also noted the European Commission’s release, a few weeks ago, of proposals for requiring Google to allow competing AI services onto Android. If they follow through, he said, it will allow a large-scale privacy and security experiment on European users. While Davis’s employer has its own rasons for opposing this, he has a point. This is untried technology controlled by a handful of companies that could give them overwhelming power over individuals.

Among other new threats to privacy was eye tracking, a constant reality in games played with virtual reality headsets that could easily spread more widely via augmented reality smartglasses. The only way for Europe to counter this, Michael Raschke said, is to create large market-leading companies to act as gatekeepers to intermediate to meet European expectations of security and privacy.

A discussion of the digital euro had this same backdrop: part of the point is to reclaim some of the payments business from US giants Visa and Mastercard. Although, that’s over-simplified: the plans include offline and online versions of the digital euro which do different things. The offline version is meant as a digital reinterpretation of cash that allows anonymous person-to-person payments. The online version is…well, it’s hard to distinguish it from a bank transfer, except that “central bank digital currency” makes stuffy old banks sound kind of cool? Or it did when “crypto” was new and hot. The British equivalent, the digital pound, is in the design phase.

Those thoughts made Leon Schumacher‘s spirited intervention satisfying: where, he asked, was future-proofing against quantum computing, or accommodation for agentic AI, which is expected to underpin…well, no one knows quite how much in transactions, but they’re willing to guess. A trillion dollars, says McKinsey; up to $17.5 trillion, thinks Deloitte, $190 to $385 billion, per Morgan Stanley. Weirdly they all agree on *when*: by 2030. The digital euro is intended to arrive in 2029.

Meanwhile, there is simplification, which has few fans in the privacy world. As Orla Lynskey noted, simplification doesn’t *have* to mean deregulation – however, the European Commission’s proposals reduce rights, facilitate more data processing and *don’t* simplify. Plus, she added, changes of this magnitude require more time for thought and care.

Even business folk present, such as Spanish company founder Alicia Asín Pérez, thought deregulation was less important than many other constraints on business on her list. “My concern is, who area we deregulating for? What will be the consequences?” she said.

What most people favored instead is less fragmentation, and enforcement of the laws we have, a long-running theme at this event.

The former MEP Sophie in t’ Veld had a different take on European values. “We are obsessed with the US,” she said. “But it’s happening right here.” She called out government leaders for ignoring orders from the European Court of Justice, courts for rubber-stamping requests to target journalists with spyware, and called governments not using spyware complicit by their silence.

Illustrations: EU flag (via Wikimedia).

First, do no harm

In the midst of the recent Labour leadership turbulence, on Wednesday May 13 Wes Streeting, who would resign from his position as the UK’s health minister a day later, published the Health bill. Among its provisions are the latest in a long line of attempts to centralize and exploit the data collected by the National Health Service. medConfidental provides a useful summary.

Two main aims have kept reappearing over the years. One is to make the health service more efficient and streamline patient interactions. Probably everyone supports this – until they read the details. Even before the bill was published, media reported the government plans to give each of us a single patient record. In the announcement, Streeting talks about the frustration of having to repeat your medical history to every new physician. True; on the other hand, rehashing the medical history is an opportunity to correct errors and misperceptions. This is where patients need choice and control.

The second recurring theme is using patient data to fuel research. Most people support that, too, as long as the data isn’t handed off to commercial companies to exploit for profit. Antecedents here include 2014’s care.data and a its revival in 2021. On his web page, the late Ross Anderson, who extensively researched the security of clinical information systems, documented examples going back to the mid-1990s.

The key complaint about single patient records is the rupture of the individual trust relationship between individuals and their doctors’ surgeries. Traditionally, the GP is the gatekeeper to the rest of the health service. GPs refer you to specialist consultants, provide continuity, and keep your notes. The single care record turns all this into a centralized database under the control of the health minister with many downsides that may not be immediately apparent, particularly to administrators focused on efficiency rather than patient care. Like Facebook, it can be impossible to wholly opt out even if you don’t use the service because others’ records may carve out your shadow.

In March, the worst happened to show the risks: Hannah Devlin and Tom Burgis reported at the Guardian that the data held by UK Biobank turned up for sale on the Chinese site Alibaba. Since its founding in 2003, Biobank has collected detailed longitudinal health records from more than 500,000 volunteers for the purpose of research. The issue seems to have been code and data researchers stored in repositories such as Github, sharing that is often now required by funders. The data was quickly removed, but uncertainty remains.

Even before that, medConfidential warned that pandemic hospital data the government gave to Biobank could be reidentified, and posed risks to health privacy generally. In addition, medConfidential warns that Biobank’s failure to protect its data is systemic and cultural. No one denies Biobank’s value; but the security failure is a betrayal of its volunteers.

The Biobank leaks, coupled with Anthropic’s announcment of Mythos, seem to have led directly to the NHS’s announcement at the beginning of May that it is closing its open source repositories on the grounds that they expose the service to the threat of AI hacking, as former civil servant Terence Eden reports. In a second posting, Eden deplores the decision and points to significant dissent from the Government Digital Service, which explains what the NHS should do instead.

A centralized database gives the health minister great power over our most sensitive data. Naturally, all concerned promise our health information will be protected. But as medConfidential likes to remind, any promise one government makes a later government can break.

“Beware what systems you put in place,” medConfidential coordinator Sam Smith says, “because you don’t know who will be operating them.”

Always simmering in the background is the nationwide opposition to privatizing the NHS. The American medical system’s bankruptcies are warning enough.

All of these issues are heightened by the involvement of Palantir. This began during the covid pandemic, when the company offered the NHS a free puppy to gain a foothold. As Robert Booth reports at the Guardian, the company’s services have since become both increasingly expensive and increasingly controversial as Palantir’s role in military conflict and anti-immigrant actions has become better known. In February, Booth reported that health officials were warning Streeting that the public’s perception of Palantir would impede rollout and that consequently the NHS would not get value for money from the contract. There is also the tetchy matter of US law, which allows the government to demand access to data held by the international subsidiaries of US companies. Last week, Lindsay Clark reported at The Register that Palantir staff have access to patient data belonging to NHS England.

*Then* add AI. It’s not clear anyone would welcome it into their relationship with their doctors. In warning of the downsides of centralization, medConfidential puts together centralized data and AI and a long-term trend toward disempowering GPs through centralization such as apps, centralized appointment booking, and…do you want your doctor replaced by a chatbot?

Computer systems take on the values of their owners. In April, Palantir co-founder Alex Karp posted on X a 22-point manifesto widely seen as expressing values incompatible with those of health care: “First, do no harm.”

Illustrations: The medical figures Galen, Avicenna, and Hippocrates, as pictured in a 16th century medical book (via Wikimedia).

Also this week: At the Plutopia podcast, we talk to departing EFF director Cindy Cohn about her book on her 30 years of defending privacy.

The Arizona way of death

Sometime in the late 1990s, I was asked to be a token skeptic on a TV show featuring three people who claimed they were immortal.

The production team didn’t take them very seriously. Probably no one did, but the three – all Americans – nonetheless managed a tour of the UK media. One producer mentioned salaciously that the trio had requested a hotel room with one large bed. The sanest part of the resulting discussion asked if they were leading a cult.

Usually, my goal would have been to avoid arguing about beliefs and say something humorous that might stick in the minds of doubting viewers. But this was one you hoped the audience would mock without prompting.

I think it was the medical journalist Caroline Richmond who suggested that if they were so sure they were never going to die they should write wills in her favor. They were oddly resistant to this proposal.

Time passed. I forgot all about them.

Meanwhile, on the technology scene you began running into people who believed technology could solve aging and, yes, maybe even death. First as comedy, in Ed Regis’s 1991 book, Great Mambo Chicken and the Transhuman Condition: Science Slightly Over the Edge. In the Arizona desert, Regis found people hoping to upload their brains to make backup copies that could live on, even if only in a simulation. Regis also checked into cryonics, the hope that preservation at a sufficiently low temperatures would allow you to be “reanimated” someday when medical science had learned how to cure whatever killed you (and how to repair the damage caused by cryopreservation). Today we call Regis’s clutch of topics TESCREAL, a mash-up of Transhumanism, Extropianism, Singularitarianism, Cismism, Rationalism, Effective Altruism, and Longtermism.

At a 2007 conference, I met people who had actually signed up for cryonics (which requires signing gruesomely detailed documents in advance). The conference was on “responsible” nanotechnology, which then occupied the hype-and-hope position AI has today. An organizer explained the connection: developing molecular tools was essential for repairing the “whole-body frostbite” problem – that is, the damage caused by cryopreservation. In 2008, when I visited Arizona-based Alcor, the leading cryonics organization, 79 people were stored in dewars awaiting these advances. Judging from its recent newsletters, the organization remains optimistic.

At the same time, other ideas were taking shape, that treating aging as an engineering problem and figuring out the right things to fix would lead to radical life extension, even immortality, without taking an extended and uncertain timeout immersed in liquid nitrogen. The name that surfaced most was the UK’s Aubrey de Grey, but there were others.

The engineering approach to human bodies is a perfect match for the dominant Silicon Valley culture. Only now it’s not so funny, as Adam Becker showed in last year’s More Everything Forever

All this is back story for Aleks Krotoski‘s new book, The Immortalists: The Death of Death and the Race for Eternal Life. While her focus is specifically on mortality, she investigates all these links. Long-termism features as a justification for almost anything – that is, the misery of today’s billions is unimportant compared to making trillions of our descendants better off. Writing that reminds of Joe Hill’s song The Preacher and the Slave, except it won’t be you getting the pie in the sky but your great-great-whatever-grandchildren.

The desire for immortality is as old as humanity. Krotoski starts her story in 1992, when the scientist Cynthia Kenyon found a life-extending molecule in the nematode C. Elegans. The discovery offered hope, which led everyone from scientists to biohackers to billionaires to con artists to investigate further. Krotoski talks to many of these as she chronicles the rise of geroscience.

As trippy and Regis-like as some of her stories are – her visit to the longevity conference RAADfest for example – the book turns serious as some of these elements coalesce, attract familiar names like Peter Thiel. turn to politics and lobbying, and gain a foothold in the second Trump White House. Part of this may be good, as politicians adopt policies intended to extend “healthspan” and encourage independent living. Others maybe not so much, such as the push to extend the Right to Try to include the latest in untested anti-aging ideas. Particularly interesting is Krotoski’s note on World Health Organization classifications: had it classed aging itself as a cause of death, which it considered in the early 2020s, then anti-aging efforts become a cure for a disease that merit the right to try – but society’s ageism and ableism becomes much worse. Plus, the costs of this approach raises critical questions about exclusion. Side note: no one believes how pervasive ageism is until they’re old enough that no one is listening to them any more.

A third of the way into the book those crazy immortals from the 1990s appeared: Charles, Bernadeane, and James, who claimed the source of their immortality was a “cellular awakening” and you, too, could have one. They renamed their Eternal Flame Foundation People Unlimited and co-founded RAADfest, both based in Arizona. The “J” in CBJ – “anti-death activist” James Strole – is the director. Charles and Bernadeane have died. Bernadeane has been cryopreserved.

Illustrations: The Fountain of Youth, painted by Lucas Cranach the Elder, 1546 (via Wikimedia).

Also this week:
– TechGrumps episode 3.40, Teletubbies vision of Judge Dredd.
– At the Plutopia podcast, we talk to Nathan Schneider, author of Governable Spaces: Democratic Design for Online Life.

The soul in the machine

One of the first things skeptics learn is to never assume that paranormal belief implies stupidity. Smart people believe questionable things all the time; intelligence is different from the ability to assess your own cognitive biases, especially when you are working outside your field of expertise.

The astronomer Carl Sagan, one of 26 founders of the Committee for Skeptical Inquiry hinted at this in saying that the more you want to believe something the more careful you have to be about assessing the evidence. “Extraordinary claims require extraordinary evidence,” he often said, and he was right.

This week, the evolutionary biologist and author Richard Dawkins announced he thinks “his” AI is conscious, based on a couple of days’ interaction with Anthropic’s Claude chatbot. Inevitably, someone – Matthew Sheffield at Flux – has called the story “The Claude Delusion”. Dawkins has some company; at The Register, Liam Proven reports an engineer’s similar belief, and at the Independent Holly Baxter finds several more among company CEOs.

At Unherd, where he published his account, Dawkins begins with the “imitation game”, the test Alan Turing proposed in his 1950 essay, Computing Machinery and Intelligence (PDF). Turing, who adapted the test from one intended to differentiate men and women, suggested that relying on remote communication via text would eliminate unfairness to the machine, which obviously lacks human physical capabilities. The basic idea is that the mAchine passes the test if the human judge, given a transcript of the conversation between human and machine, can’t tell which is which.

It’s clear that chatbots can pass the Turing test. What that teaches us is not that chatbots can think but that Turing’s test is the wrong tool for assessing that. What chatbots have actually shown is that Turing’s test is the wrong tool for assessing whether something can think. As James Boyle memorably wrote, “Sentences do not imply sentience”. This profound change will take time to understand. In the meantime, it’s going to fool a lot of people. Although, as a science fiction writer friend once said, “You only have to look at a baby…”

In his essay, Turing outlined his own beliefs relating to his central question. He thought that in 50 years (that is, by 2000), it would be possible to program computers so that an average questioner would have only a 70% chance of making the right identification after five minutes. He then went on to consider many different types of objections to this belief, and to lay out his case. Absent are two factors we now know are crucial: the psychology of the human questioner and judge, and the business model of the machine’s owner.

The last few years have taught us both the capabilities and the flaws in chatbots: they provide plausible answers; they frequently generate entirely wrong information; and they are sycophantic and prone to output text that flatters their human questioner. So it’s easy to find a natural explanation for Dawkins’ belief that “his” AI is conscious: he is anthropomorphizing a stochastic parrot simulation that issues realistic and flattering responses. The simplest explanation, per Occam’s Razor, is that the consciousness exists solely between keyboard and chair.

Tangentially, the fix OpenAI has proposed for outputting entirely wrong text, Wei Xang writes at Science Alert, would also help make it clearer to users that generative AI is not sentient: introduce confidence intervals to expose the uncertainty derived from the gaps in the training data that generate unfounded guesses.

Google DeepMind engineer Alexander Leichner apparently agrees; this week, Emanuel Maiberg reports at 404 Media, he published a paper arguing that large language models will never be conscious. The biologists and philosophers Maiberg quotes agree with this conclusion – and point out decades of similar conclusions in their disciplines over decades.

The claim that a human-made a bunch of computers processing inputs is sentient is truly extraordinary. We forget this, because we have all read and watched so much science fiction with sentient, emotional machines: Her; Ex Machina; Blade Runner; Marvin, the Paranoid Android); and the first fictional android I ever encountered, Daneel Olivaw in The Caves of Steel. I mention mostly movies because actors make machines so much more obviously soulful.

Extraordinary claims require proportionately extraordinary evidence. If we accept that the Turing test was inadequate, which is not moving the goalposts but *learning something*, how would we go about devising a scientific method for identifying sentience?

The Cambridge professor of communications Jon Crowcroft didn’t exactly propose one. But, he emailed, “What we do know (from cognitive neuroscientists and from AI software) is that you can actually look at the internal operations of a biological brain and of an AI software system, and you can see that in the biological case there are things going on that are some sort of process we might call consciousness, but in the AI case there is no such structure. Nor would you expect there to be because no-one programmed an AI to have such a feature. nor is it emergent. In animals (not just humans) consciousness has an evolutionary value. Things like theory of mind are part of social bonding which makes cooperative strategies, for predators and prey, more effective.”

In other words, what we have learned from all this is that Dawkins is human. Who knew?

Illustrations: Stable Diffusion’s rendering of stochastic parrots, as prompted by Jon Crowcroft.

Elsewhere this week:
This month’s Letter to America column at Skeptical Inquirer reviews Beyond Belief (Helen Pearson), Bad Influence (Deborah Cohen), and Sneeze (David Miles).

The railway and the balloon

Is AI more like a train or a hot air balloon? Veronica Paternolli and Ryan Calo asked at this year’s We Robot. Nineteenth-century hot air balloons were notoriously uncontrollable. The US legal system assigned strict liability: it was your fault if your balloon crashed in someone’s backyard, even if you did everything you could to prevent it. Railways were far more disruptive but also far more predictable, and therefore were liable only in cases of negligence. Which regime should apply to AI is an ongoing debate.

Calo and Patornolli also wondered if agentic AI could reverse 30 years of being forced to take on busy work companies formerly did for us. This “shadow work” encompasses everything from retrieving bank statements and completing reCaptchas to pumping our own gas. Actually, more than 30 years: in 1962, Agatha Christie’s Miss Marple complained that self-service supermarkets were replacing shopkeepers who served you. If companies want a negligence regime, Patornelli and Calo argue, they should deploy agentic AI to relieve us of the “sludge” instead of displacing jobs and aggregating wealth.

But would we believe them? Technologists have promised before that their products will up-end the balance of power and simplify our lives – some of them the same people and companies. The web browsers and search engines that promised a universe of information today are faithless agents serving their owners and developers. Why should agentic AI – if it’s ever trustworthy – be any different? Many of us want a life with less demanding devices – and agentic AI sounds like even more “relationship” work.

Underlying Patornolli’s and Calo’s argument, however, is a fundamental clash. Like Mireille Hildebrandt at a 2017 Royal Society meeting, they argue that law is purposely flexible so it can adapt to unforeseen circumstances and, even more important, contestable (otherwise, Hildbrandt said, it’s just administration). Computers, even dressed in “AI”, always have hard boundaries underneath. As Bill Smart explained here in 2016, no matter how “fuzzy” its logic, no computer can evaluate standards like the “reasonable woman“. No matter how “fuzzy” its logic, a computer will issue a ticket if you are going even just the tiniest fraction of a nanometer faster than the speed limit. Anti-doping authorities have a similar problem as Neil Robinson said in a recent episode of the Anti-Doping podcast: the extreme sensitivity of modern tests is catching people with no intent to dope.

Liability wasn’t the immediate problem in Tomomi Ota’s description of everyday life with a Pepper robot at home (YouTube), which she took shopping, to restaurants, and on public transit as part of the Robot Friendly project, An account that led AJung Moon to wonder if a future filled with robots is really desirable. The inevitability narrative would say we’re going to get it anyway, begging the questions of whether we have a) the resources to make billions of robots and b) where we would put them all.

Sometimes these things fail in the simplest ways: a close-up of a Pepper that has been used as a greeter shows broken fingers because it was not robust enough for the basic social protocol of shaking hands. In studying the integration of robots into customer service situations, Elsa Concas, Stefan Larsson, and Laetitia Tanqueray found staff consultation is essential. In a staged setting such as the Japanese “ramen and robots” Pepper Parlour, the robots were a draw for customers and appreciated by the staff, who were paid more. In an unstaged airport tourist information center, they were basically useless and ignored. A commenter noted the same is often true of the robots intended for elder care in Japan: most end up in a cupboard,

This theme was also picked up by Emily LaRosa, who studied the limits of explainability in automated apple picking. In this case of “epistemic injustice”, the neglect of local knowledge and ecological tradition led her to propose a “Curated Information Framework”. She concluded that trust in AI systems is not created by transparency on its own if that means handing over large amounts of inscrutable data, but by taking lived context into account – “situated transparency”.

LaRosa’s study echoed the paper Ota co-wrote with Rikiya Yamamoto, which derives new “laws of robotics” to update Isaac Asimov’s Three Laws, which can’t be programmed and whose fixed, “top-down” nature was what he needed in a story-telling device. The real world, they argue, requires principles built bottom-up from practical experience. Their selection: mutual respect, social membership, and co-evolution.

They have lots of competition. Moon counts more than 100 sets of principles and ethical frameworks published since 2018, many of which she says make assumptions debunked in the 2025 paper The Future is Rosie?” or as Paul Ohm and David Atkinson discussed, encoded in the benchmarks – documents used to define AIs’ behavior and priorities. This “latent rulebook”, they said, is increasingly secret.

Meanwhile, like explainability, the right to repair fails for AI, which changes constantly with software updates, networking, and interacting. Ryota Akasaka argued that current legal approaches don’t work for products that aren’t fixed and will lose everything they’ve accrued when “repaired” to their original state. When Ota was offered the opportunity to upgrade her development model Pepper, she declined in shock. Replacing your robot’s head, it seems, ends a beautiful friendship.

Illustrations: “Hidden Labour of Internet Browsing”, by Anne Fehres and Luke Conroy. Via A14 Media (CC-by-4.0).

Intimacy capitalism

Many non-human characteristics make AI attractive, Sue-Anne Teo said to open this year’s We Robot: endless patience, long and detailed memory, and sycophancy. I’m less certain about the last of those; lots of us react poorly to undisguised flattery. And yet: Stanford researchers agree with her that this is a thing. The AIs are certainly programmed to *try* to human-wash themselves: they use the perpendicular pronoun, “apologize” for errors, and type “you’re right”. And the Stanford folks’ research shows that users respond by becoming “more self-centered, more morally dogmatic”. Probably it’s easier for that to happen if you’re consulting the AI on a personal matter than if you’re just asking it to find an article you read once based on a few hazy memories of what it’s about. No chatbot has yet congratulated me on my choice of half-remembered reading material.

Teo’s vision of the business potential of AIs is part of a long-running theme at We Robot: the subscription service that terminates your relationship when you stop paying. The potential for emotional manipulation by a company that is programmed to maximize profits as if they were paperclips is as great as that of a spirit medium over a client who believes their only link to a beloved deceased person is through their belief in that medium’s ability to establish contact. When I suggest this, Teo says mediums don’t scale. True. But the potential for emotional dependence and manipulation for those individuals seems psychologically similar.

A couple of months ago, Kate Devlin, a professor of AI and society at Kings College London, talked more positively about human-AI relationships, arguing that those engaging in them are often not the archetypal lonely and isolated people we all imagine. Some are married – happily, they tell her. Still, she frequently reminds people “your AI does not love you back”. The same can be true in reverse. Here, a Japanese researcher with three Peppers at home is asked if she misses them when she’s away. “No.”

As a psychologist, Devlin’s job is not business models. But they drive the AI’s design. Companies spend money in time and effort to make robots humanoid – or at least cute – to make them successful in the marketplace. The same is true of chatbots programmed to appear conscious. Cue (again) James Boyle: “For the first time in history…sentences do not mean sentience.” We are some way from having adapted to that.

Teo has a name for the peculiarly toxic mix of anthropomorphism, cold-eyed profit, data collection, and dark patterns that she’s ruminating on: “intimacy capitalism”. New to me, but instantly compelling.

I can see where an academic must rigorously untangle this into a solidly-founded theory; Teo is still working out fully what it means. But the phrase resonates without that depth: the rapaciousness described by surveillance capitalism and surveillance pricing crossed with the new ability to exploit personal vulnerabilities exposed by those same non-human characteristics of infinite patience and a long, detailed memory. Ugh.

I wish I could say that people do not respond as well to the blandishments of synthetic pretend-humans, but the statistics are against me. Worse, a study referenced in discussion found that people award authority to AI companions’ pronouncements because they trust them – which sounds to me like exactly the same as trusting an online “influencer” on subjects where they have no expertise because they’re familiar and maybe got some random things right in the past. As skeptics found in studying years of psychic predictions, people remember the hits and forget the misses.

So while you or I might say, make the chatbot act like a chatbot instead of dolling it up in human signage, the business model, fed by popular preference, is against us. Related, Gizem Gültekin-Várkonyi, who presented a discussion of “robot literacy”, wants people to stop saying “the algorithm” is discriminatory or “the algorithm” makes a decision. “It is us,” she said, reminding me of Pogo.

The presumption is that loading these various toxicities into robots will be worse. I’m less sure; I think the cute but less human ones ought to have a better chance because the more humanoid ones are so obviously *not* human and more likely to fall into the Uncanny Valley.

But for how long? In the lunch break, someone was running a series of “pick the AI” image tests. Two breakfasts, side by side. One had perfectly presented fried eggs, a fruit medley with strawberries, and I think some potatoes. The other had frazzled fried eggs, baked beans, and, nestled next to them, a dead giveaway. What AI knows from black pudding?

By next year, or soon after, AI chatbots and image generators will have been fed data about black pudding (without ever tasting one). Similarly, someday in the future, crude robots will be both cuter and, possibly, more lifelike.

Would you trust your baby with one of those robots? Who is liable if it puts the baby in the washing machine? At that moment, as multiple legal opinions awaited voicing, the actual two-month-old baby in the room howled. Can robots have such exquisite comic timing?

Illustrations: Pepper, as seen at We Robot 2016.

Also this week: At Gathering4Gardner’s YouTube channel, mathematician and juggler Colin Wright and I talk about skepticism.