net.wars

Technology distinguishable from magic

This week, a regional court in Munich issued what may be a landmark decision: it ruled that Google is liable for the content of its “AI Overviews”. The court was careful to distinguish between these, which Google’s algorithms generate, and search results, which link to content on third-party websites Google does not control. In other words, Google owns its own mistakes. The court awarded 80% of costs to the plaintiffs, two Munich-based publishers who claimed that the company spread damaging false information about them by linking them to scams in these AI-generated summaries.

As Max Bastian writes at Decoder, the ruling could set precedents for other companies. Google has yet to comment, but presumably will appeal. Away from AI is certainly not the direction the company wants to go; it’s spent the last couple of years building generative AI more and more deeply into its search, hoping to keep users locked in instead of chasing off to other sites. (Ironically, bouncing users off to other sites was the reason Yahoo refused to buy it in 1998, when the received business model was keeping users on your own site as long as possible.)

Google tried to argue that users should do their own fact-checking. But, as the court seems to have understood, where search results send you to the source page, AI Overviews look complete and don’t always offer sourcing to check. A study conducted for the New York Times found in an analysis of more than 4,000 searches that AI Overviews produced using the Gemini 2 model were accurate 85% of the time, rising to 91% when using Gemini 3 – about average for these systems. Enter the Law of Truly Large Numbers: 91% only sounds pretty good until you multiply the remaining 9% by billions to calculate the millions of wrong answers being disseminated every single day. Oumi, the startup that performed the analysis, found the AI Overviews included sources such as Facebook and Reddit posts, drew incorrect information even from authoritative sources, and are prone to manipulation. The article notes that Google disputes the analysis, saying that the benchmarks were developed by OpenAI and themselves contain inaccuracies.

The Munich court ordered Google to stop repeating the claims about the publishers, and awarded the publishers 80% of costs. It also rejected Google’s attempt to frame the issue as one of freedom of speech, calling the AI Overviews, “above all an expression of Google’s business activities”.

One reason the judge’s ruling is so significant is that most approaches for dealing with misinformation that have been mooted to date are at human, instead of computer, scale. Fact-checking, for example, while valuable, moves very, very slowly, one claim at a time. If the ruling stands, it will help tackle this type of misinformation at source.

***

Politicians like to talk as if the moon they want is available if the industry would just stop being obstructive. With AI’s capabilities in headlines everywhere, they are now demanding that phones should block children from taking, viewing, or sharing nude photos. This is the policy Keir Starmer announced this week in a speech, based on claims from the British company SafetoNet. The government has since provided more detail. Mic Wright has a round-up of press reactions. At the Daily Telegraph, Big Brother Watch director Silkie Carlo provides a strong civil liberties objection.

So far, neither Apple nor Google has said much. At New Scientist, Chris Stokel-Walker notes that both companies already have some controls in place, but spreading them through third-party apps poses challenges, especially as some phones’ operating systems aren’t recent enough to have the more sophisticated parental controls in the first place.

The moral may be: if you tell people your technology is magic, don’t be surprised when they expect it to *be* magic.

***

This week I had to verify my identity for Companies House. This is supposed to be a straightforward matter of creating a Gov One login, entering some details of a government-issued ID, and uploading a photo. The website was discontented: it couldn’t find my address, (is my century-old home too old to be in the database?), and didn’t accept the details I entered. Eventually, it offered two alternatives: use a phone app, or present myself in person at a preselected post office.

The app balked. It couldn’t open its links even though I’d authorized it. So, in this year of two thousand and twenty-six I got on a train to go to the nearest remaining post office that could perform the necessary rituals to show them first a QR code to access my application and then the ID, whose information is digitally held but had to be retyped on the post office tablet, and finally pose for a photo for the system – not the post office human – to compare and match. Some of those steps took several tries to mollify the system. Naturally, they don’t report whether you’ve passed until after you’ve gone home.

These are the people who want to create a digital ID infrastructure. I can only assume that if they ever get that system up and running actually using it will involve faxing things because by then all the post offices will be gone.

Illustrations: “The Magic Lantern”, by Auguste Edouart, circa 1835 (via The Met); at one time we thought that technology was magic.

Also this week:
At the Plutopia podcast, we talk to about her new book, Bad Influence.
The TechGrumps podcast episode, 3.41: The KardashElons of AI.

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

The sovereignty paradox

The year since the 2025 Computers, Privacy, and Data Protection has made Europe more distinct as an entity. Two years ago, we were being chastised for paying insufficient attention to structural corporate power. At this conference last year, we were warned about “simplification”, since published as the Digital Omnibus that threatens to undo some aspects of data protection and other privacy rights.

This year, we heard a lot about “European values”. Invoked as a metric: does “simplification” measure up or is it a sign of weakening? Invoked as a frame to guide developing the digital euro. Invoked as a reason why digital sovereignty is increasingly essential. In November 2025, 23 European countries signed a declaration embracing the latter as a policy.

And yet, although Plixavra Vogiatzoglou introduced a panel discussion by calling digital sovereignty “urgent because of Trump’s trade wars”, she then said it was unrealistic.

There’s a lot to like in the 2025 declaration, which lists as principles open source solutions, the principle of common assets, competitive markets, and democracy. The problem, as Vogiatzoglou said, is that Europe doesn’t make all the necessary components for building its own bottom-to-top stack. The investment required is likely to favor the wealthiest countries, deepening the imbalances that already favor US technology companies.

This is the “sovereignty paradox,” as Zuzanna Warso called it. Replicating the current infrastructure with little change other than a different owner isn’t the right goal – as someone said later, who wants a European Palantir? Warso favors redesigning the technical ecosystem to foster the “digital commons” – and being honest about the tradeoffs.

The same theme reappeared in a discussion of agentic assistants: they will ultimately sit on infrastructure belonging to the same few hyperscalers. Frederika Kaltheuner posited three scenarios: full vertical integration (like Google), integrated models and software (Anthropic and Claude Code), or open source and smaller models, which she thought was Europe’s only opportunity for sovereignty.

All of this seems set to get worse with agentic AI, which, Apple’s Gary Davis said, will allow agents onto all our devices capable of listening, observing, inferring, and acting across apps. What controls do we want? I personally want the control of barring this proposed technological future from my life, but how many of us will have that choice? Davis also noted the European Commission’s release, a few weeks ago, of proposals for requiring Google to allow competing AI services onto Android. If they follow through, he said, it will allow a large-scale privacy and security experiment on European users. While Davis’s employer has its own rasons for opposing this, he has a point. This is untried technology controlled by a handful of companies that could give them overwhelming power over individuals.

Among other new threats to privacy was eye tracking, a constant reality in games played with virtual reality headsets that could easily spread more widely via augmented reality smartglasses. The only way for Europe to counter this, Michael Raschke said, is to create large market-leading companies to act as gatekeepers to intermediate to meet European expectations of security and privacy.

A discussion of the digital euro had this same backdrop: part of the point is to reclaim some of the payments business from US giants Visa and Mastercard. Although, that’s over-simplified: the plans include offline and online versions of the digital euro which do different things. The offline version is meant as a digital reinterpretation of cash that allows anonymous person-to-person payments. The online version is…well, it’s hard to distinguish it from a bank transfer, except that “central bank digital currency” makes stuffy old banks sound kind of cool? Or it did when “crypto” was new and hot. The British equivalent, the digital pound, is in the design phase.

Those thoughts made Leon Schumacher‘s spirited intervention satisfying: where, he asked, was future-proofing against quantum computing, or accommodation for agentic AI, which is expected to underpin…well, no one knows quite how much in transactions, but they’re willing to guess. A trillion dollars, says McKinsey; up to $17.5 trillion, thinks Deloitte, $190 to $385 billion, per Morgan Stanley. Weirdly they all agree on *when*: by 2030. The digital euro is intended to arrive in 2029.

Meanwhile, there is simplification, which has few fans in the privacy world. As Orla Lynskey noted, simplification doesn’t *have* to mean deregulation – however, the European Commission’s proposals reduce rights, facilitate more data processing and *don’t* simplify. Plus, she added, changes of this magnitude require more time for thought and care.

Even business folk present, such as Spanish company founder Alicia Asín Pérez, thought deregulation was less important than many other constraints on business on her list. “My concern is, who area we deregulating for? What will be the consequences?” she said.

What most people favored instead is less fragmentation, and enforcement of the laws we have, a long-running theme at this event.

The former MEP Sophie in t’ Veld had a different take on European values. “We are obsessed with the US,” she said. “But it’s happening right here.” She called out government leaders for ignoring orders from the European Court of Justice, courts for rubber-stamping requests to target journalists with spyware, and called governments not using spyware complicit by their silence.

Illustrations: EU flag (via Wikimedia).

The soul in the machine

One of the first things skeptics learn is to never assume that paranormal belief implies stupidity. Smart people believe questionable things all the time; intelligence is different from the ability to assess your own cognitive biases, especially when you are working outside your field of expertise.

The astronomer Carl Sagan, one of 26 founders of the Committee for Skeptical Inquiry hinted at this in saying that the more you want to believe something the more careful you have to be about assessing the evidence. “Extraordinary claims require extraordinary evidence,” he often said, and he was right.

This week, the evolutionary biologist and author Richard Dawkins announced he thinks “his” AI is conscious, based on a couple of days’ interaction with Anthropic’s Claude chatbot. Inevitably, someone – Matthew Sheffield at Flux – has called the story “The Claude Delusion”. Dawkins has some company; at The Register, Liam Proven reports an engineer’s similar belief, and at the Independent Holly Baxter finds several more among company CEOs.

At Unherd, where he published his account, Dawkins begins with the “imitation game”, the test Alan Turing proposed in his 1950 essay, Computing Machinery and Intelligence (PDF). Turing, who adapted the test from one intended to differentiate men and women, suggested that relying on remote communication via text would eliminate unfairness to the machine, which obviously lacks human physical capabilities. The basic idea is that the mAchine passes the test if the human judge, given a transcript of the conversation between human and machine, can’t tell which is which.

It’s clear that chatbots can pass the Turing test. What that teaches us is not that chatbots can think but that Turing’s test is the wrong tool for assessing that. What chatbots have actually shown is that Turing’s test is the wrong tool for assessing whether something can think. As James Boyle memorably wrote, “Sentences do not imply sentience”. This profound change will take time to understand. In the meantime, it’s going to fool a lot of people. Although, as a science fiction writer friend once said, “You only have to look at a baby…”

In his essay, Turing outlined his own beliefs relating to his central question. He thought that in 50 years (that is, by 2000), it would be possible to program computers so that an average questioner would have only a 70% chance of making the right identification after five minutes. He then went on to consider many different types of objections to this belief, and to lay out his case. Absent are two factors we now know are crucial: the psychology of the human questioner and judge, and the business model of the machine’s owner.

The last few years have taught us both the capabilities and the flaws in chatbots: they provide plausible answers; they frequently generate entirely wrong information; and they are sycophantic and prone to output text that flatters their human questioner. So it’s easy to find a natural explanation for Dawkins’ belief that “his” AI is conscious: he is anthropomorphizing a stochastic parrot simulation that issues realistic and flattering responses. The simplest explanation, per Occam’s Razor, is that the consciousness exists solely between keyboard and chair.

Tangentially, the fix OpenAI has proposed for outputting entirely wrong text, Wei Xang writes at Science Alert, would also help make it clearer to users that generative AI is not sentient: introduce confidence intervals to expose the uncertainty derived from the gaps in the training data that generate unfounded guesses.

Google DeepMind engineer Alexander Leichner apparently agrees; this week, Emanuel Maiberg reports at 404 Media, he published a paper arguing that large language models will never be conscious. The biologists and philosophers Maiberg quotes agree with this conclusion – and point out decades of similar conclusions in their disciplines over decades.

The claim that a human-made a bunch of computers processing inputs is sentient is truly extraordinary. We forget this, because we have all read and watched so much science fiction with sentient, emotional machines: Her; Ex Machina; Blade Runner; Marvin, the Paranoid Android); and the first fictional android I ever encountered, Daneel Olivaw in The Caves of Steel. I mention mostly movies because actors make machines so much more obviously soulful.

Extraordinary claims require proportionately extraordinary evidence. If we accept that the Turing test was inadequate, which is not moving the goalposts but *learning something*, how would we go about devising a scientific method for identifying sentience?

The Cambridge professor of communications Jon Crowcroft didn’t exactly propose one. But, he emailed, “What we do know (from cognitive neuroscientists and from AI software) is that you can actually look at the internal operations of a biological brain and of an AI software system, and you can see that in the biological case there are things going on that are some sort of process we might call consciousness, but in the AI case there is no such structure. Nor would you expect there to be because no-one programmed an AI to have such a feature. nor is it emergent. In animals (not just humans) consciousness has an evolutionary value. Things like theory of mind are part of social bonding which makes cooperative strategies, for predators and prey, more effective.”

In other words, what we have learned from all this is that Dawkins is human. Who knew?

Illustrations: Stable Diffusion’s rendering of stochastic parrots, as prompted by Jon Crowcroft.

Elsewhere this week:
This month’s Letter to America column at Skeptical Inquirer reviews Beyond Belief (Helen Pearson), Bad Influence (Deborah Cohen), and Sneeze (David Miles).

The railway and the balloon

Is AI more like a train or a hot air balloon? Veronica Paternolli and Ryan Calo asked at this year’s We Robot. Nineteenth-century hot air balloons were notoriously uncontrollable. The US legal system assigned strict liability: it was your fault if your balloon crashed in someone’s backyard, even if you did everything you could to prevent it. Railways were far more disruptive but also far more predictable, and therefore were liable only in cases of negligence. Which regime should apply to AI is an ongoing debate.

Calo and Patornolli also wondered if agentic AI could reverse 30 years of being forced to take on busy work companies formerly did for us. This “shadow work” encompasses everything from retrieving bank statements and completing reCaptchas to pumping our own gas. Actually, more than 30 years: in 1962, Agatha Christie’s Miss Marple complained that self-service supermarkets were replacing shopkeepers who served you. If companies want a negligence regime, Patornelli and Calo argue, they should deploy agentic AI to relieve us of the “sludge” instead of displacing jobs and aggregating wealth.

But would we believe them? Technologists have promised before that their products will up-end the balance of power and simplify our lives – some of them the same people and companies. The web browsers and search engines that promised a universe of information today are faithless agents serving their owners and developers. Why should agentic AI – if it’s ever trustworthy – be any different? Many of us want a life with less demanding devices – and agentic AI sounds like even more “relationship” work.

Underlying Patornolli’s and Calo’s argument, however, is a fundamental clash. Like Mireille Hildebrandt at a 2017 Royal Society meeting, they argue that law is purposely flexible so it can adapt to unforeseen circumstances and, even more important, contestable (otherwise, Hildbrandt said, it’s just administration). Computers, even dressed in “AI”, always have hard boundaries underneath. As Bill Smart explained here in 2016, no matter how “fuzzy” its logic, no computer can evaluate standards like the “reasonable woman“. No matter how “fuzzy” its logic, a computer will issue a ticket if you are going even just the tiniest fraction of a nanometer faster than the speed limit. Anti-doping authorities have a similar problem as Neil Robinson said in a recent episode of the Anti-Doping podcast: the extreme sensitivity of modern tests is catching people with no intent to dope.

Liability wasn’t the immediate problem in Tomomi Ota’s description of everyday life with a Pepper robot at home (YouTube), which she took shopping, to restaurants, and on public transit as part of the Robot Friendly project, An account that led AJung Moon to wonder if a future filled with robots is really desirable. The inevitability narrative would say we’re going to get it anyway, begging the questions of whether we have a) the resources to make billions of robots and b) where we would put them all.

Sometimes these things fail in the simplest ways: a close-up of a Pepper that has been used as a greeter shows broken fingers because it was not robust enough for the basic social protocol of shaking hands. In studying the integration of robots into customer service situations, Elsa Concas, Stefan Larsson, and Laetitia Tanqueray found staff consultation is essential. In a staged setting such as the Japanese “ramen and robots” Pepper Parlour, the robots were a draw for customers and appreciated by the staff, who were paid more. In an unstaged airport tourist information center, they were basically useless and ignored. A commenter noted the same is often true of the robots intended for elder care in Japan: most end up in a cupboard,

This theme was also picked up by Emily LaRosa, who studied the limits of explainability in automated apple picking. In this case of “epistemic injustice”, the neglect of local knowledge and ecological tradition led her to propose a “Curated Information Framework”. She concluded that trust in AI systems is not created by transparency on its own if that means handing over large amounts of inscrutable data, but by taking lived context into account – “situated transparency”.

LaRosa’s study echoed the paper Ota co-wrote with Rikiya Yamamoto, which derives new “laws of robotics” to update Isaac Asimov’s Three Laws, which can’t be programmed and whose fixed, “top-down” nature was what he needed in a story-telling device. The real world, they argue, requires principles built bottom-up from practical experience. Their selection: mutual respect, social membership, and co-evolution.

They have lots of competition. Moon counts more than 100 sets of principles and ethical frameworks published since 2018, many of which she says make assumptions debunked in the 2025 paper The Future is Rosie?” or as Paul Ohm and David Atkinson discussed, encoded in the benchmarks – documents used to define AIs’ behavior and priorities. This “latent rulebook”, they said, is increasingly secret.

Meanwhile, like explainability, the right to repair fails for AI, which changes constantly with software updates, networking, and interacting. Ryota Akasaka argued that current legal approaches don’t work for products that aren’t fixed and will lose everything they’ve accrued when “repaired” to their original state. When Ota was offered the opportunity to upgrade her development model Pepper, she declined in shock. Replacing your robot’s head, it seems, ends a beautiful friendship.

Illustrations: “Hidden Labour of Internet Browsing”, by Anne Fehres and Luke Conroy. Via A14 Media (CC-by-4.0).

Intimacy capitalism

Many non-human characteristics make AI attractive, Sue-Anne Teo said to open this year’s We Robot: endless patience, long and detailed memory, and sycophancy. I’m less certain about the last of those; lots of us react poorly to undisguised flattery. And yet: Stanford researchers agree with her that this is a thing. The AIs are certainly programmed to *try* to human-wash themselves: they use the perpendicular pronoun, “apologize” for errors, and type “you’re right”. And the Stanford folks’ research shows that users respond by becoming “more self-centered, more morally dogmatic”. Probably it’s easier for that to happen if you’re consulting the AI on a personal matter than if you’re just asking it to find an article you read once based on a few hazy memories of what it’s about. No chatbot has yet congratulated me on my choice of half-remembered reading material.

Teo’s vision of the business potential of AIs is part of a long-running theme at We Robot: the subscription service that terminates your relationship when you stop paying. The potential for emotional manipulation by a company that is programmed to maximize profits as if they were paperclips is as great as that of a spirit medium over a client who believes their only link to a beloved deceased person is through their belief in that medium’s ability to establish contact. When I suggest this, Teo says mediums don’t scale. True. But the potential for emotional dependence and manipulation for those individuals seems psychologically similar.

A couple of months ago, Kate Devlin, a professor of AI and society at Kings College London, talked more positively about human-AI relationships, arguing that those engaging in them are often not the archetypal lonely and isolated people we all imagine. Some are married – happily, they tell her. Still, she frequently reminds people “your AI does not love you back”. The same can be true in reverse. Here, a Japanese researcher with three Peppers at home is asked if she misses them when she’s away. “No.”

As a psychologist, Devlin’s job is not business models. But they drive the AI’s design. Companies spend money in time and effort to make robots humanoid – or at least cute – to make them successful in the marketplace. The same is true of chatbots programmed to appear conscious. Cue (again) James Boyle: “For the first time in history…sentences do not mean sentience.” We are some way from having adapted to that.

Teo has a name for the peculiarly toxic mix of anthropomorphism, cold-eyed profit, data collection, and dark patterns that she’s ruminating on: “intimacy capitalism”. New to me, but instantly compelling.

I can see where an academic must rigorously untangle this into a solidly-founded theory; Teo is still working out fully what it means. But the phrase resonates without that depth: the rapaciousness described by surveillance capitalism and surveillance pricing crossed with the new ability to exploit personal vulnerabilities exposed by those same non-human characteristics of infinite patience and a long, detailed memory. Ugh.

I wish I could say that people do not respond as well to the blandishments of synthetic pretend-humans, but the statistics are against me. Worse, a study referenced in discussion found that people award authority to AI companions’ pronouncements because they trust them – which sounds to me like exactly the same as trusting an online “influencer” on subjects where they have no expertise because they’re familiar and maybe got some random things right in the past. As skeptics found in studying years of psychic predictions, people remember the hits and forget the misses.

So while you or I might say, make the chatbot act like a chatbot instead of dolling it up in human signage, the business model, fed by popular preference, is against us. Related, Gizem Gültekin-Várkonyi, who presented a discussion of “robot literacy”, wants people to stop saying “the algorithm” is discriminatory or “the algorithm” makes a decision. “It is us,” she said, reminding me of Pogo.

The presumption is that loading these various toxicities into robots will be worse. I’m less sure; I think the cute but less human ones ought to have a better chance because the more humanoid ones are so obviously *not* human and more likely to fall into the Uncanny Valley.

But for how long? In the lunch break, someone was running a series of “pick the AI” image tests. Two breakfasts, side by side. One had perfectly presented fried eggs, a fruit medley with strawberries, and I think some potatoes. The other had frazzled fried eggs, baked beans, and, nestled next to them, a dead giveaway. What AI knows from black pudding?

By next year, or soon after, AI chatbots and image generators will have been fed data about black pudding (without ever tasting one). Similarly, someday in the future, crude robots will be both cuter and, possibly, more lifelike.

Would you trust your baby with one of those robots? Who is liable if it puts the baby in the washing machine? At that moment, as multiple legal opinions awaited voicing, the actual two-month-old baby in the room howled. Can robots have such exquisite comic timing?

Illustrations: Pepper, as seen at We Robot 2016.

Also this week: At Gathering4Gardner’s YouTube channel, mathematician and juggler Colin Wright and I talk about skepticism.

Long Island AI

The peak of dot-com mania was identifiable even at the time: it was in January 2000, when AOL bought Time Warner for $183 billion. Peak podcast mania arguably came in 2020 when Spotify paid Joe Rogan $100 million (per rumor). It’s now time to call the peak of AI mania: Allbirds, which was loved a few years back for making sustainable wool shoes …has sold off all stock and assets, and is rebranding as Newbird AI to sell “AI compute infrastructure”. At The Overspill, Charles Arthur compares this to 2017, when Long Island Iced Tea renamed itself Long Blockchain, arguably the peak of bitcoin mania. His theory that all Newbird AI has left is a bunch of empty warehouses, so hopes someone will put data centers in them is the only possibility that makes any sense.

Most of those bits of history led to more financial silliness and then a crash. The AOL-TimeWarner marriage was notoriously catastrophic. AOL, which was supposed to modernize old, slow Time-Warner, in fact was already becoming obsolete as users shifted to new technology (broadband) and the wider Internet. By 2003, the company was selling itself off in pieces. Long Island Blockchain cratered; a year later it had abandoned its plans to buy bitcoin mining equipment. Its delisting by NASDAQ was accompanied by the SEC’s charging three people with insider trading. Joe Rogan of course remains hugely popular and Spotify is doing fine, but it’s certainly not controlling the business of podcasting as it appeared to hope it would.

The day following the announcemenet, Newbird AI’s shares rose as much as 700% (briefly), partly on the additional news that it will close a funding round of $50 million in the second quarter of 2026. Even CNBC calls this “pivot” bizarre.

For our purposes, it doesn’t matter if this wacky strategy works (pick your definition of “works”), because when the share price of a basically assetless company goes up 700% because it’s added “AI” to its name we have reached the absurdity that marks the peak of every bubble.

It’s not the only sign (or the only absurdity). In the UK, last month Aisha Down reported at the Guardian that many of the efforts prime minister Keir Starmer – and Rishi Sunak before him – has announced to “mainline AI into the veins” of the British economy are based on what she calls “phantom investments”. She reported faithfully that the Department of Science, Innovation, and Technology said it “rejected these assertions”, but this week we learned that at least one piece of her reporting was absolutely correct.

This week’s news revolves around a project called Stargate, announced in September 2025 and involving the UK-headquartered AI infrastructure provider Nscale, Microsoft, Nvidia, and OpenAI. This week, OpenAI announced it was putting the project – for which it was supposed to build a data center – on hold. OpenAI blames energy prices and, as Joseph Bambridge reports at Politico, the government’s decision last month to shelve proposals to allow data miners to use copyrighted content unless its owner opts out. The proposal was widely opposed by the UK’s creative industry, and was indefinitely delayed in a report issued on March 18 (ReedSmith has a useful legal summary)..

The loss – or delay – of Stargate is a rounding error to companies the size of Microsoft and Nvidia. It’s more significant for Nscale, which according to CNBC raised $2 billion in a funding round just last month with investments from Nvidia, Dell, Lenovo, and other much less famous names; at the same time, it added former Google and Facebook ad business builder Sheryl Sandberg and former Facebook global policy head Nick Clegg to its board. The new funding raised Nscale’s valuation to $14.6 billion. At his blog, Ed Zitron calls the ability to raise funding for a data center that doesn’t exist “weird“, and suggests that AI companies should admit that their chatbots are just “regular old software”.

Meta king Mark Zuckerberg, last seen losing money on the former next-big-thing metaverse, is, Megan Bobrowsky reports at the Wall Street Journal, building an AI agent to get him information and answers faster. This comes on top of last month’s announcement that Meta is buying the AI agent network Moltbook, seemingly mostly in order to hire its two founders for Meta’s Superintelligence Labs. This week, Zuckerberg also announced he was building an AI clone of himself to interact with staff so they can feel more connected to him. Seems like in reality it would make corporate management feel like automated customer service.

The question about bubbles is always: is this one like railroads or like tulips? Tulips left nothing of value behind while railroads went on to be transformative. In 2001, almost everyone knew the Internet would go on growing in size and importance. In the AI case, despite the current silliness, over time we will learn how to use these new capabilities and limit the downsides. But first, we will have to deal with the fallout of the fact that the finances do not add up.

Illustrations: Tulips, (via Wikimedia).

Also this week:
At Skeptical Inquirer, I go to this year’s Gathering 4 Gardner.
At the Plutopia podcast, we interview Tereza Pultarova, who reports on developing military technology in Ukraine.

Waste management

This week, Amazon announced it would cripple a bunch of older Kindle models. Users will be able to go on reading the books that are already stored on their devices, but they won’t be able to add anything new, whether it’s borrowed, bought, or downloaded. The switch will be toggled on May 20, and applies to Kindle ereaders and Kindle Fires released in or before 2012. These devices had already been locked out of the store in 2022.

There is no benefit to consumers from doing this, even though Amazon is offering a substantial discount on newer devices for those who want to switch. There are presumably benefits to Amazon – namely, that it can update its store and other software without having to cater to older devices, plus sell an extra bunch of new ones. But overall, it’s a globally hostile move that creates a new pile of electronic waste composed of functioning hardware. One of these days, that’s going to be your car when some auto manufacturer decides that a “legacy” model isn’t worth supporting any more.

One option is to stop buying devices whose manufacturers demand that you surrender ownership in favor of their ongoing control. The other main possibility is to continue spreading right to repair laws so that a company like Amazon (or John Deere) wishing to shed itself of the responsibility of supporting older devices would be required to open them up to their users and the third-party ecosystem around them that would doubtless form. As the population of Internet of Things devices continues to grow around the world there will be much more of this – when we need much less. It’s absolutely maddening. Try a Kobo and the Gutenberg project.

***

It came out this week that language added in October 2025 to Microsoft’s terms of use says: “Copilot is for entertainment purposes only. It can make mistakes, and it may not work as intended. Don’t rely on Copilot for important advice. Use Copilot at your own risk.”

This is like the disclaimers that psychics and mediums in the UK issue to prospective customers to prevent exploitation of the credulous. A software company, though…if it’s going to admit up front that the service it’s providing can’t be relied on, why force it on people in the first place?

Obviously, the point here is to cover the company’s ass in case of lawsuits. It fits right in with companies’ general refusal to accept liability for problems created by the software they make. I’m glad the company is warning people, but the better solution – if the company can’t bring itself to discontinue it altogether – would be to either remove Copilot and let people download it if they want it or leave it turned off by default, warning people up front instead of in terms of use that normally wouldn’t have been read. Instead, what we have to do is search for instructions to remove it and hope Microsoft hasn’t made following them impossible.

***

In the midst of a lot of science fictional hype-scares about “AI” along come some more real concerns. We’ll start “small”: The New York Times has done a study that finds that Google’s AI Overviews are right 90% of the time. Sounds not-so-bad, until you remember the Law of Truly Large Numbers. That “90%” success rate when the remaining 10% is applied to trillions means the overall result is, as Ryan Whitwam puts it in summarizing the story for Ars Technica, “hundreds of thousands of lies going out every minute of the day”. That’s automation for you. Used to be you needed an army of bots to disseminate misinformation at any sort of scale, and even then it was less effective since it wasn’t backed by an apparently authoritative name (see also Microsoft, above).

More alarming are the reports surfacing about generative AI’s effectiveness at aiding and amplifying online crime. In November, Google announced internal researchers had found hackers experimenting with a script that interacts with Gemini’s API to create just-in-time modifications. Last month, at Hacker Noon polymorphic viruses, but again, this appears to be a step up in sophistication and speed.

At the same time, Casey Newton reports at Platformer, Anthropic’s latest model is likely the first of many that can find and exploit vulnerabilities in software in “ways that far outpace the efforts of defenders”. The announcement, which appears to have been inadvertent, was quickly followed by another, formally launching the new model, Claude Mythos, and Project Glasswing. The latter, per Newton, gives more than 40 of the biggest technology companies early access so they can use the model to find and patch vulnerabilities in both their own systems and open source systems that underpin digital infrastructure.

Unlike most AI-related scare stories, this one is backed by people who are usually sensible, such as Alex Stamos, formerly chief security officer at Facebook and Yahoo, and other security practitioners. These warnings are not coming from AI company CEOs with a concept to sell .

So: on the one hand, (hopefully) better software; on the other, (potentially) newer, more dangerous attacks. Ugh. To restore calm, I recommend Terry Godier’s essay The Last Quiet Thing.

Illustrations: Bill Steele, who in 1970 wrote the environmental anthem Garbage!.

Turn left at the robot

It’s easy to forget, now that so many computer interfaces seem to be getting more inscrutable, that in the early 1990s the shift from the command line to graphical interfaces and the desire to reach the wider mass market introduced a new focus on usability. In classics like Don Norman’s The Design of Everyday Things, a common idea was that a really well-designed system should not require a manual because the design itself would tell the user how to operate it.

What is intuitive, though, is defined by what you’re used to. The rumor that Apple might replace the letters on keys with glyphs is a case in point. People who’ve grown up with smartphones might like the idea of glyphs that match those on their phones. But try doing technical support over the phone and describing a glyph; easier to communicate the letters.

Those years in which computer interfaces were standardized relied on metaphors based on familiar physical items: a floppy disk to save a file, an artist’s palette for color choices. In 1993, leading software companies like Microsoft and Lotus set up usability labs; it took watching user testing to convince developers that struggling to use their software was not a sign the users were stupid.

With that background, it was interesting to attend this year’s edition of the 20-year-old Human-Robot Interaction conference. Robots don’t need metaphors; they *are* the thing itself. Although: why shouldn’t a robot also have menu buttons for common functions?

In the paper I found most interesting and valuable, Lauren Wright examined the use of a speaking Misty robot to deliver social-emotional learning lessons. Wright’s group tested the value of deception – that is, having the robot speak in the first person of its “family”, experiences, and “emotions” – versus a more truthful presentation, in which the robot is neutral and tells its stories in the third person, refers to its programmers, and professes no humanity. The researchers were testing the widely-held assumption that kids engage more with robots programmed to appear more human. They found the opposite: while both versions significantly increased their learning, the kids who used the factual robot showed more engagement and higher scores in the sense of using concepts from the lesson in their answers. This really shouldn’t be surprising. Children don’t in general respond well to deception. Really, who does?

The children’s personal reactions to the robots were at least as interesting. In Michael F. Xu‘s paper, the researchers held co-design sessions and then installed a robot in eight family homes to act as a neutral third-party enforcer issuing timely reminders on behalf of busy parents. Some of the families did indeed report that the robot’s reminder got stuff done more efficiently. On the other hand, the experiment was short – only four days – and you have to wonder if that would still be true after the novelty wore off. There were hints of this from the kids, some of whom pushed back. One simply bypassed a robot reminding him of the limits on his TV viewing by taking the TV upstairs, where the robot couldn’t go. Another reacted like I would at any age and told the robot to “shut up”.

The fact versus fiction presentation included short video clips of some of the kids’ interaction with the robot tutor. In one, a boy placed his hands on either side of the robot’s “face” while it was talking and kept moving its head around, exploring the robot’s physical capabilities (or trying to take its head off?). The speaker ignored this entirely, but the sight hilariously made an important point: the robot’s physical form speaks even when the robot is silent.

We saw this at We Robot 2016, when a Jamaican lawyer asked Olivier Guilhem, from Aldebaran Robotics, which makes Pepper, “Why is the robot white?” His response: “It looks clean.” This week, one paper tried to tease out how “representation bias” – assumptions about gender, skin tone, dis/ability, accessibility, size, age – affect users’ reactions. In the dataset used to train an AI model, bias may be baked in through the historical record. With robots, bias can also present directly through the robot’s design, as Yolande Strengers’ and Jenny Kennedy’s showed in their 2020 book The Smart Wife. Despite its shiny, unmistakable whiteness, Pepper’s shape was ambiguous enough for its gender to be interpreted differently in different cultures. In the HRI paper, the researchers concluded that biases in robot design could perpetuate occupational stereotypes – “technological segregation”. They also found their participants consistently preferred non-skin tones – in their examples, silver and light teal.

“Who builds AI shapes what AI becomes,” said Ben Rosman, who outlined a burgeoning collaborative effort to build a machine learning community across Africa and redress its underrepresentation. The same with robots: many, many cultural norms affect how humans interact with them. That information is signal, not noise, he says, and should be captured to enable robots to operate across wide ranges of human context without relying on “brittle defaults” that interpret human variation as failures. “Turn left at the robot,” makes perfect sense once you know that in South Africa “robots” are known elsewhere as traffic lights.

Illustrations: Rosey, the still-influential “old demo model” robot maid in The Jetsons (1962-1963).

Also this week:
At the Plutopia podcast, we interview Marc Abrahams, founder of the Ig Nobel awards.
At Skeptical Inquirer, the latest Letter to America finds David Clarke conducting the English folklore survey.

A short history of We Robot, 2026 edition

On the eve of We Robot 2026, here are links to my summaries of every year since 2012, the inaugural conference, except 2014, which I missed for family reasons. There was no conference in 2024 in order to move the event back to its original April schedule (covid caused its move to September in 2020). These are my personal impressions; nothing I say here should be taken as representing the conference, its founders, its speakers, or their institutions.

We Robot was co-founded by Michael Froomkin, Ryan Calo, and Ian Kerr to bring together lawyers and engineers to think early and long about the coming conflicts in robots, law, and policy.

2025: Predatory inclusion: In Windsor, Ontario, a few months into the new US administration, the sudden change in international relations highlights the power imbalances inherent in many of today’s AI systems. Catopromancy: in workshops, we hear a librarian propose useful AI completely out of step with today’s corporate offerings, and mull how to apply existing laws to new scenarios.

2024 No conference.

2023 The end of cool: after struggling to design a drone delivery service that had benefits over today’s cycling couriers, we find ourselves less impressed by robots that can do somersaults but not anything obviously useful; the future may have seemed more exciting when it was imaginary.

2022 Insert a human: following a long-held conference theme about “humans in the loop, “robots” are now “sociotechnical systems”. Coding ethics: Where Asimov’s laws were just a story device, in workshops we try to work out how to design a real ethical robot.

2021 Plausible diversions: maybe any technology sufficiently advanced to seem like magic can be well enough understood that we can assign responsibility and liability? Is the juice worth the squeeze?: In workshops, we mull how to regulate delivery robots, which will likely have no user-serviceable parts. Title from Woody Hartzog.

2020 (virtual) The zero on the phone: AI exploitation and bias embedded in historical data become what one speaker calls “unregulated experimentation on humans…without oversight or control”.

2019 Math, monsters, and metaphors. We dissect the trolley problem and find the true danger on the immediate horizon is less robots, more the “pile of math that does some stuff” we call “AI”. The Algernon problem: in workshops, new disciplines joining the We Robot family remind us that robots/AI are carrying out the commands of distant owners.

2018 Deception. We return to the question of what makes robots different and revisit Madeleine Clare Elish’s moral crumple zones after the first pedestrian death by self-driving car. Late, noisy, and wrong: in workshops, engineers Bill Smart and Cindy Grimm explain why sensors never capture what you think they’re capturing and how AI systems use their data.

2017 Have robot, will legislate: Discussion of risks this year focused on the intermediate situation, when automation and human norms must co-exist.

2016 Humans all the way down: Madeline Clare Elish introduces “moral crumple zones”, a paper that will resonate through future years. The lab and the world: in workshops, Bill Smart uses conference attendees in formation to show why getting a robot to do anything is difficult.

2015: Multiplicity: W
When in the life of a technology is the right time for regulatory intervention?

2014 Missed conference

2013 Cautiously apocalyptic: Diversity of approaches to regulation will be needed to handle the diversity of robots, and at the beginning of cloud robotics and full-scale data collection, we envision a pet robot dog that can beg its owner for an upgraded service subscription.

2012 A really fancy hammer with a gun. At the first We Robot, we try to answer the fundamental question: what difference do robots bring? Unsentimental engineer Bill Smart provided the title.

Bedroom eyes

We’ve long known that much of today’s “AI” is humans all the way down. This week underlines this: in an investigation, Svenska Dagbladet and Göteborgs-Posten learn that Meta’s Ray-Ban smart glasses are capturing intimate details of people’s lives and sending them to Nairobi, Kenya. There, employees at Meta subcontractor Sama label and annotate the data for use in training models. Brings a new meaning to “bedroom eyes”.

This sort of violation is easily imposed on other people without their knowledge or consent. We worry about the police using live facial recognition, but what about being captured by random people on the street? In January’s episode of the TechGrumps podcast, we called the news of Meta’s new product “Return of the Glasshole“.

Two 2018 books, Mary L. Gray and Siddharth Suri’s Ghost Work and Sarah T. Roberts’ Behind the Screen made it clear that “machine learning” and “AI” depend on poorly-paid unseen laborers. Dataveillance is a stowaway in every “smart” device. But this is a whole new level: the Kenyans report glimpses of bank cards, bedroom intimacy, even bathroom visits. The journalists were able to establish that the glasses’ AI requires a connection to Meta’s servers to answer questions, and there’s no opt out.

The UK’s Information Commissioner’s Office is investigating, and at Ars Technica Sarah Perez reports that a US lawsuit has been filed.

As the original Swedish report goes on to say, the EU has no adequacy agreement with Kenya. More disturbing is the fact that probably hundreds of people within Meta worked on this without seeing a problem.

In 1974, the Watergate-related revelation that US president Richard Nixon had recorded everything taking place in his office inspired folksinger Bill Steele to write the song The Walls Have Ears (MP3). What struck him particularly was that everyone saw it as unremarkable. “Unfortunately still current,” he commented in his 1977 liner notes. Nearly 50 years later, ditto.

***

A lot of (especially younger) people don’t remember that before 9/11 you could walk into most buildings without showing ID. Many authorities – the EU in particular – have long been unhappy with anonymity online, and one conspiratorial theory about age gating and the digital ID infrastructure being built in many places is that the goal is complete and pervasive identification. In the UK, requiring ID for all Internet access has occasionally popped up as a child safety idea, even though security experts recommend lying about birth dates and other personal data in the interests of self-protection against identity theft.

Now we have generative AI, and along comes a new paper that finds that large language models can be used to deanonymize people online at large scale by analyzing profiles and conversations. In one exercise, they matched Hacker News posts to LinkedIn profiles. In another, they linked users across subReddit communities. In a third, they split Reddit profiles to mimic the use of pseudonymous posting. Pseudonymity doesn’t offer meaningful protection (though I’m not sure how much it ever did), and preventing this type of attack is difficult. They also suggest platforms should reconsider their data access policies in line with their findings.

It’s hard to imagine most platforms will care much; users have long been expected to assess their own risk. Even smaller communities with a more concerned administration will not be in a position to know how many other services their users access, what they post there, or how it can be cross-linked. The difficulty of remaining anonymous online has been growing ever since 2000, when Latanya Sweeney showed it was possible to identify 87% of the population recorded in census data given just Zip code, date of birth, and gender. As psychics know, most people don’t really remember what they’ve said and how it can be linked and exploited by someone who’s paying attention. The paper concludes: we need a new threat model for privacy online.

***

The Internet, famously, was designed to support communications in the face of a bomb outage.

Building it required physical links – undersea cables, fiber connections, data centers, routers. For younger folks who have grown up with wifi and mobile phone connections, that physical layer may be invisible. But it matters no less than it did twenty-five years ago, when experts agreed that ten backhoes (among other things) could do more effective damage than bombs.

This week’s horrible, spreading war in the Middle East has seen the closure of the Strait of Hormuz and the Red see to commercial traffic. Indranil Ghosh reports at Rest of World that that 17 undersea cables pass through the Red Sea alone, and billions, soon trillions, of dollars in US technology investment depends on fiber optic cables running through war zones. There’s been reporting before now about the links between various Middle Eastern countries and Silicon Valley (see for example the recent book Gilded Rage, by Jacob Silverman), but until now much less about the technological interdependence put in jeopardy by the conflict. Ghosh also reports that drones have struck two Amazon Web Services data centers in UAE and one in Bahrein.

The issue is not so much direct damage to the cables as the impossibility of repairing them as long as access is closed. The Internet, designed with war in mind, is a product of peace.

Illustrations: Monument to Anonymous, by Meredith Bergmann.

Also this week: At the Plutopia podcast, we interview Kate Devlin, who studies human-AI interaction.