Month: May 2026

The soul in the machine

One of the first things skeptics learn is to never assume that paranormal belief implies stupidity. Smart people believe questionable things all the time; intelligence is different from the ability to assess your own cognitive biases, especially when you are working outside your field of expertise.

The astronomer Carl Sagan, one of 26 founders of the Committee for Skeptical Inquiry hinted at this in saying that the more you want to believe something the more careful you have to be about assessing the evidence. “Extraordinary claims require extraordinary evidence,” he often said, and he was right.

This week, the evolutionary biologist and author Richard Dawkins announced he thinks “his” AI is conscious, based on a couple of days’ interaction with Anthropic’s Claude chatbot. Inevitably, someone – Matthew Sheffield at Flux – has called the story “The Claude Delusion”. Dawkins has some company; at The Register, Liam Proven reports an engineer’s similar belief, and at the Independent Holly Baxter finds several more among company CEOs.

At Unherd, where he published his account, Dawkins begins with the “imitation game”, the test Alan Turing proposed in his 1950 essay, Computing Machinery and Intelligence (PDF). Turing, who adapted the test from one intended to differentiate men and women, suggested that relying on remote communication via text would eliminate unfairness to the machine, which obviously lacks human physical capabilities. The basic idea is that the mAchine passes the test if the human judge, given a transcript of the conversation between human and machine, can’t tell which is which.

It’s clear that chatbots can pass the Turing test. What that teaches us is not that chatbots can think but that Turing’s test is the wrong tool for assessing that. What chatbots have actually shown is that Turing’s test is the wrong tool for assessing whether something can think. As James Boyle memorably wrote, “Sentences do not imply sentience”. This profound change will take time to understand. In the meantime, it’s going to fool a lot of people. Although, as a science fiction writer friend once said, “You only have to look at a baby…”

In his essay, Turing outlined his own beliefs relating to his central question. He thought that in 50 years (that is, by 2000), it would be possible to program computers so that an average questioner would have only a 70% chance of making the right identification after five minutes. He then went on to consider many different types of objections to this belief, and to lay out his case. Absent are two factors we now know are crucial: the psychology of the human questioner and judge, and the business model of the machine’s owner.

The last few years have taught us both the capabilities and the flaws in chatbots: they provide plausible answers; they frequently generate entirely wrong information; and they are sycophantic and prone to output text that flatters their human questioner. So it’s easy to find a natural explanation for Dawkins’ belief that “his” AI is conscious: he is anthropomorphizing a stochastic parrot simulation that issues realistic and flattering responses. The simplest explanation, per Occam’s Razor, is that the consciousness exists solely between keyboard and chair.

Tangentially, the fix OpenAI has proposed for outputting entirely wrong text, Wei Xang writes at Science Alert, would also help make it clearer to users that generative AI is not sentient: introduce confidence intervals to expose the uncertainty derived from the gaps in the training data that generate unfounded guesses.

Google DeepMind engineer Alexander Leichner apparently agrees; this week, Emanuel Maiberg reports at 404 Media, he published a paper arguing that large language models will never be conscious. The biologists and philosophers Maiberg quotes agree with this conclusion – and point out decades of similar conclusions in their disciplines over decades.

The claim that a human-made a bunch of computers processing inputs is sentient is truly extraordinary. We forget this, because we have all read and watched so much science fiction with sentient, emotional machines: Her; Ex Machina; Blade Runner; Marvin, the Paranoid Android); and the first fictional android I ever encountered, Daneel Olivaw in The Caves of Steel. I mention mostly movies because actors make machines so much more obviously soulful.

Extraordinary claims require proportionately extraordinary evidence. If we accept that the Turing test was inadequate, which is not moving the goalposts but *learning something*, how would we go about devising a scientific method for identifying sentience?

The Cambridge professor of communications Jon Crowcroft didn’t exactly propose one. But, he emailed, “What we do know (from cognitive neuroscientists and from AI software) is that you can actually look at the internal operations of a biological brain and of an AI software system, and you can see that in the biological case there are things going on that are some sort of process we might call consciousness, but in the AI case there is no such structure. Nor would you expect there to be because no-one programmed an AI to have such a feature. nor is it emergent. In animals (not just humans) consciousness has an evolutionary value. Things like theory of mind are part of social bonding which makes cooperative strategies, for predators and prey, more effective.”

In other words, what we have learned from all this is that Dawkins is human. Who knew?

Illustrations: Stable Diffusion’s rendering of stochastic parrots, as prompted by Jon Crowcroft.

Elsewhere this week:
This month’s Letter to America column at Skeptical Inquirer reviews Beyond Belief (Helen Pearson), Bad Influence (Deborah Cohen), and Sneeze (David Miles).

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

The railway and the balloon

Is AI more like a train or a hot air balloon? Veronica Paternolli and Ryan Calo asked at this year’s We Robot. Nineteenth-century hot air balloons were notoriously uncontrollable. The US legal system assigned strict liability: it was your fault if your balloon crashed in someone’s backyard, even if you did everything you could to prevent it. Railways were far more disruptive but also far more predictable, and therefore were liable only in cases of negligence. Which regime should apply to AI is an ongoing debate.

Calo and Patornolli also wondered if agentic AI could reverse 30 years of being forced to take on busy work companies formerly did for us. This “shadow work” encompasses everything from retrieving bank statements and completing reCaptchas to pumping our own gas. Actually, more than 30 years: in 1962, Agatha Christie’s Miss Marple complained that self-service supermarkets were replacing shopkeepers who served you. If companies want a negligence regime, Patornelli and Calo argue, they should deploy agentic AI to relieve us of the “sludge” instead of displacing jobs and aggregating wealth.

But would we believe them? Technologists have promised before that their products will up-end the balance of power and simplify our lives – some of them the same people and companies. The web browsers and search engines that promised a universe of information today are faithless agents serving their owners and developers. Why should agentic AI – if it’s ever trustworthy – be any different? Many of us want a life with less demanding devices – and agentic AI sounds like even more “relationship” work.

Underlying Patornolli’s and Calo’s argument, however, is a fundamental clash. Like Mireille Hildebrandt at a 2017 Royal Society meeting, they argue that law is purposely flexible so it can adapt to unforeseen circumstances and, even more important, contestable (otherwise, Hildbrandt said, it’s just administration). Computers, even dressed in “AI”, always have hard boundaries underneath. As Bill Smart explained here in 2016, no matter how “fuzzy” its logic, no computer can evaluate standards like the “reasonable woman“. No matter how “fuzzy” its logic, a computer will issue a ticket if you are going even just the tiniest fraction of a nanometer faster than the speed limit. Anti-doping authorities have a similar problem as Neil Robinson said in a recent episode of the Anti-Doping podcast: the extreme sensitivity of modern tests is catching people with no intent to dope.

Liability wasn’t the immediate problem in Tomomi Ota’s description of everyday life with a Pepper robot at home (YouTube), which she took shopping, to restaurants, and on public transit as part of the Robot Friendly project, An account that led AJung Moon to wonder if a future filled with robots is really desirable. The inevitability narrative would say we’re going to get it anyway, begging the questions of whether we have a) the resources to make billions of robots and b) where we would put them all.

Sometimes these things fail in the simplest ways: a close-up of a Pepper that has been used as a greeter shows broken fingers because it was not robust enough for the basic social protocol of shaking hands. In studying the integration of robots into customer service situations, Elsa Concas, Stefan Larsson, and Laetitia Tanqueray found staff consultation is essential. In a staged setting such as the Japanese “ramen and robots” Pepper Parlour, the robots were a draw for customers and appreciated by the staff, who were paid more. In an unstaged airport tourist information center, they were basically useless and ignored. A commenter noted the same is often true of the robots intended for elder care in Japan: most end up in a cupboard,

This theme was also picked up by Emily LaRosa, who studied the limits of explainability in automated apple picking. In this case of “epistemic injustice”, the neglect of local knowledge and ecological tradition led her to propose a “Curated Information Framework”. She concluded that trust in AI systems is not created by transparency on its own if that means handing over large amounts of inscrutable data, but by taking lived context into account – “situated transparency”.

LaRosa’s study echoed the paper Ota co-wrote with Rikiya Yamamoto, which derives new “laws of robotics” to update Isaac Asimov’s Three Laws, which can’t be programmed and whose fixed, “top-down” nature was what he needed in a story-telling device. The real world, they argue, requires principles built bottom-up from practical experience. Their selection: mutual respect, social membership, and co-evolution.

They have lots of competition. Moon counts more than 100 sets of principles and ethical frameworks published since 2018, many of which she says make assumptions debunked in the 2025 paper The Future is Rosie?” or as Paul Ohm and David Atkinson discussed, encoded in the benchmarks – documents used to define AIs’ behavior and priorities. This “latent rulebook”, they said, is increasingly secret.

Meanwhile, like explainability, the right to repair fails for AI, which changes constantly with software updates, networking, and interacting. Ryota Akasaka argued that current legal approaches don’t work for products that aren’t fixed and will lose everything they’ve accrued when “repaired” to their original state. When Ota was offered the opportunity to upgrade her development model Pepper, she declined in shock. Replacing your robot’s head, it seems, ends a beautiful friendship.

Illustrations: “Hidden Labour of Internet Browsing”, by Anne Fehres and Luke Conroy. Via A14 Media (CC-by-4.0).

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.