We Robot Archives

The railway and the balloon

Is AI more like a train or a hot air balloon? Veronica Paternolli and Ryan Calo asked at this year’s We Robot. Nineteenth-century hot air balloons were notoriously uncontrollable. The US legal system assigned strict liability: it was your fault if your balloon crashed in someone’s backyard, even if you did everything you could to prevent it. Railways were far more disruptive but also far more predictable, and therefore were liable only in cases of negligence. Which regime should apply to AI is an ongoing debate.

Calo and Patornolli also wondered if agentic AI could reverse 30 years of being forced to take on busy work companies formerly did for us. This “shadow work” encompasses everything from retrieving bank statements and completing reCaptchas to pumping our own gas. Actually, more than 30 years: in 1962, Agatha Christie’s Miss Marple complained that self-service supermarkets were replacing shopkeepers who served you. If companies want a negligence regime, Patornelli and Calo argue, they should deploy agentic AI to relieve us of the “sludge” instead of displacing jobs and aggregating wealth.

But would we believe them? Technologists have promised before that their products will up-end the balance of power and simplify our lives – some of them the same people and companies. The web browsers and search engines that promised a universe of information today are faithless agents serving their owners and developers. Why should agentic AI – if it’s ever trustworthy – be any different? Many of us want a life with less demanding devices – and agentic AI sounds like even more “relationship” work.

Underlying Patornolli’s and Calo’s argument, however, is a fundamental clash. Like Mireille Hildebrandt at a 2017 Royal Society meeting, they argue that law is purposely flexible so it can adapt to unforeseen circumstances and, even more important, contestable (otherwise, Hildbrandt said, it’s just administration). Computers, even dressed in “AI”, always have hard boundaries underneath. As Bill Smart explained here in 2016, no matter how “fuzzy” its logic, no computer can evaluate standards like the “reasonable woman“. No matter how “fuzzy” its logic, a computer will issue a ticket if you are going even just the tiniest fraction of a nanometer faster than the speed limit. Anti-doping authorities have a similar problem as Neil Robinson said in a recent episode of the Anti-Doping podcast: the extreme sensitivity of modern tests is catching people with no intent to dope.

Liability wasn’t the immediate problem in Tomomi Ota’s description of everyday life with a Pepper robot at home (YouTube), which she took shopping, to restaurants, and on public transit as part of the Robot Friendly project, An account that led AJung Moon to wonder if a future filled with robots is really desirable. The inevitability narrative would say we’re going to get it anyway, begging the questions of whether we have a) the resources to make billions of robots and b) where we would put them all.

Sometimes these things fail in the simplest ways: a close-up of a Pepper that has been used as a greeter shows broken fingers because it was not robust enough for the basic social protocol of shaking hands. In studying the integration of robots into customer service situations, Elsa Concas, Stefan Larsson, and Laetitia Tanqueray found staff consultation is essential. In a staged setting such as the Japanese “ramen and robots” Pepper Parlour, the robots were a draw for customers and appreciated by the staff, who were paid more. In an unstaged airport tourist information center, they were basically useless and ignored. A commenter noted the same is often true of the robots intended for elder care in Japan: most end up in a cupboard,

This theme was also picked up by Emily LaRosa, who studied the limits of explainability in automated apple picking. In this case of “epistemic injustice”, the neglect of local knowledge and ecological tradition led her to propose a “Curated Information Framework”. She concluded that trust in AI systems is not created by transparency on its own if that means handing over large amounts of inscrutable data, but by taking lived context into account – “situated transparency”.

LaRosa’s study echoed the paper Ota co-wrote with Rikiya Yamamoto, which derives new “laws of robotics” to update Isaac Asimov’s Three Laws, which can’t be programmed and whose fixed, “top-down” nature was what he needed in a story-telling device. The real world, they argue, requires principles built bottom-up from practical experience. Their selection: mutual respect, social membership, and co-evolution.

They have lots of competition. Moon counts more than 100 sets of principles and ethical frameworks published since 2018, many of which she says make assumptions debunked in the 2025 paper The Future is Rosie?” or as Paul Ohm and David Atkinson discussed, encoded in the benchmarks – documents used to define AIs’ behavior and priorities. This “latent rulebook”, they said, is increasingly secret.

Meanwhile, like explainability, the right to repair fails for AI, which changes constantly with software updates, networking, and interacting. Ryota Akasaka argued that current legal approaches don’t work for products that aren’t fixed and will lose everything they’ve accrued when “repaired” to their original state. When Ota was offered the opportunity to upgrade her development model Pepper, she declined in shock. Replacing your robot’s head, it seems, ends a beautiful friendship.

Illustrations: “Hidden Labour of Internet Browsing”, by Anne Fehres and Luke Conroy. Via A14 Media (CC-by-4.0).

Wendy M. Grossman is an award-winning journalist. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

Intimacy capitalism

Many non-human characteristics make AI attractive, Sue-Anne Teo said to open this year’s We Robot: endless patience, long and detailed memory, and sycophancy. I’m less certain about the last of those; lots of us react poorly to undisguised flattery. And yet: Stanford researchers agree with her that this is a thing. The AIs are certainly programmed to *try* to human-wash themselves: they use the perpendicular pronoun, “apologize” for errors, and type “you’re right”. And the Stanford folks’ research shows that users respond by becoming “more self-centered, more morally dogmatic”. Probably it’s easier for that to happen if you’re consulting the AI on a personal matter than if you’re just asking it to find an article you read once based on a few hazy memories of what it’s about. No chatbot has yet congratulated me on my choice of half-remembered reading material.

Teo’s vision of the business potential of AIs is part of a long-running theme at We Robot: the subscription service that terminates your relationship when you stop paying. The potential for emotional manipulation by a company that is programmed to maximize profits as if they were paperclips is as great as that of a spirit medium over a client who believes their only link to a beloved deceased person is through their belief in that medium’s ability to establish contact. When I suggest this, Teo says mediums don’t scale. True. But the potential for emotional dependence and manipulation for those individuals seems psychologically similar.

A couple of months ago, Kate Devlin, a professor of AI and society at Kings College London, talked more positively about human-AI relationships, arguing that those engaging in them are often not the archetypal lonely and isolated people we all imagine. Some are married – happily, they tell her. Still, she frequently reminds people “your AI does not love you back”. The same can be true in reverse. Here, a Japanese researcher with three Peppers at home is asked if she misses them when she’s away. “No.”

As a psychologist, Devlin’s job is not business models. But they drive the AI’s design. Companies spend money in time and effort to make robots humanoid – or at least cute – to make them successful in the marketplace. The same is true of chatbots programmed to appear conscious. Cue (again) James Boyle: “For the first time in history…sentences do not mean sentience.” We are some way from having adapted to that.

Teo has a name for the peculiarly toxic mix of anthropomorphism, cold-eyed profit, data collection, and dark patterns that she’s ruminating on: “intimacy capitalism”. New to me, but instantly compelling.

I can see where an academic must rigorously untangle this into a solidly-founded theory; Teo is still working out fully what it means. But the phrase resonates without that depth: the rapaciousness described by surveillance capitalism and surveillance pricing crossed with the new ability to exploit personal vulnerabilities exposed by those same non-human characteristics of infinite patience and a long, detailed memory. Ugh.

I wish I could say that people do not respond as well to the blandishments of synthetic pretend-humans, but the statistics are against me. Worse, a study referenced in discussion found that people award authority to AI companions’ pronouncements because they trust them – which sounds to me like exactly the same as trusting an online “influencer” on subjects where they have no expertise because they’re familiar and maybe got some random things right in the past. As skeptics found in studying years of psychic predictions, people remember the hits and forget the misses.

So while you or I might say, make the chatbot act like a chatbot instead of dolling it up in human signage, the business model, fed by popular preference, is against us. Related, Gizem Gültekin-Várkonyi, who presented a discussion of “robot literacy”, wants people to stop saying “the algorithm” is discriminatory or “the algorithm” makes a decision. “It is us,” she said, reminding me of Pogo.

The presumption is that loading these various toxicities into robots will be worse. I’m less sure; I think the cute but less human ones ought to have a better chance because the more humanoid ones are so obviously *not* human and more likely to fall into the Uncanny Valley.

But for how long? In the lunch break, someone was running a series of “pick the AI” image tests. Two breakfasts, side by side. One had perfectly presented fried eggs, a fruit medley with strawberries, and I think some potatoes. The other had frazzled fried eggs, baked beans, and, nestled next to them, a dead giveaway. What AI knows from black pudding?

By next year, or soon after, AI chatbots and image generators will have been fed data about black pudding (without ever tasting one). Similarly, someday in the future, crude robots will be both cuter and, possibly, more lifelike.

Would you trust your baby with one of those robots? Who is liable if it puts the baby in the washing machine? At that moment, as multiple legal opinions awaited voicing, the actual two-month-old baby in the room howled. Can robots have such exquisite comic timing?

Illustrations: Pepper, as seen at We Robot 2016.

Also this week: At Gathering4Gardner’s YouTube channel, mathematician and juggler Colin Wright and I talk about skepticism.

A short history of We Robot, 2026 edition

On the eve of We Robot 2026, here are links to my summaries of every year since 2012, the inaugural conference, except 2014, which I missed for family reasons. There was no conference in 2024 in order to move the event back to its original April schedule (covid caused its move to September in 2020). These are my personal impressions; nothing I say here should be taken as representing the conference, its founders, its speakers, or their institutions.

We Robot was co-founded by Michael Froomkin, Ryan Calo, and Ian Kerr to bring together lawyers and engineers to think early and long about the coming conflicts in robots, law, and policy.

2025: Predatory inclusion: In Windsor, Ontario, a few months into the new US administration, the sudden change in international relations highlights the power imbalances inherent in many of today’s AI systems. Catopromancy: in workshops, we hear a librarian propose useful AI completely out of step with today’s corporate offerings, and mull how to apply existing laws to new scenarios.

2024 No conference.

2023 The end of cool: after struggling to design a drone delivery service that had benefits over today’s cycling couriers, we find ourselves less impressed by robots that can do somersaults but not anything obviously useful; the future may have seemed more exciting when it was imaginary.

2022 Insert a human: following a long-held conference theme about “humans in the loop, “robots” are now “sociotechnical systems”. Coding ethics: Where Asimov’s laws were just a story device, in workshops we try to work out how to design a real ethical robot.

2021 Plausible diversions: maybe any technology sufficiently advanced to seem like magic can be well enough understood that we can assign responsibility and liability? Is the juice worth the squeeze?: In workshops, we mull how to regulate delivery robots, which will likely have no user-serviceable parts. Title from Woody Hartzog.

2020 (virtual) The zero on the phone: AI exploitation and bias embedded in historical data become what one speaker calls “unregulated experimentation on humans…without oversight or control”.

2019 Math, monsters, and metaphors. We dissect the trolley problem and find the true danger on the immediate horizon is less robots, more the “pile of math that does some stuff” we call “AI”. The Algernon problem: in workshops, new disciplines joining the We Robot family remind us that robots/AI are carrying out the commands of distant owners.

2018 Deception. We return to the question of what makes robots different and revisit Madeleine Clare Elish’s moral crumple zones after the first pedestrian death by self-driving car. Late, noisy, and wrong: in workshops, engineers Bill Smart and Cindy Grimm explain why sensors never capture what you think they’re capturing and how AI systems use their data.

2017 Have robot, will legislate: Discussion of risks this year focused on the intermediate situation, when automation and human norms must co-exist.

2016 Humans all the way down: Madeline Clare Elish introduces “moral crumple zones”, a paper that will resonate through future years. The lab and the world: in workshops, Bill Smart uses conference attendees in formation to show why getting a robot to do anything is difficult.

2015: Multiplicity: W
When in the life of a technology is the right time for regulatory intervention?

2014 Missed conference

2013 Cautiously apocalyptic: Diversity of approaches to regulation will be needed to handle the diversity of robots, and at the beginning of cloud robotics and full-scale data collection, we envision a pet robot dog that can beg its owner for an upgraded service subscription.

2012 A really fancy hammer with a gun. At the first We Robot, we try to answer the fundamental question: what difference do robots bring? Unsentimental engineer Bill Smart provided the title.

Predatory inclusion

The recent past is a foreign country; they view the world differently there.

At last week’s We Robot conference on technology, policy, and law, the indefatigably detail-oriented Sue Glueck was the first to call out a reference to the propagation of transparency and accountability by the “US and its allies” as newly out of date. From where we were sitting in Windsor, Ontario, its conjoined fraternal twin, Detroit, Michigan, was clearly visible just across the river. But: recent events.

As Ottawa law professor Teresa Scassa put it, “Before our very ugly breakup with the United States…” Citing, she Anu Bradford, she went on, “Canada was trying to straddle these two [US and EU] digital empires.” Canada’s human rights and privacy traditions seem closer to those of the EU, even though shared geography means the US and Canada are superficially more similar.

We’ve all long accepted that the “technology is neutral” claim of the 1990s is nonsense – see, just this week, Luke O’Brien’s study at Mother Jones of the far-right origins of the web-scraping facial recognition company Clearview AI. The paper Glueck called out, co-authored in 2024 by Woody Hartzog, wants US lawmakers to take a tougher approach to regulating AI and ban entirely some systems that are fundamentally unfair. Facial recognition, for example, is known to be inaccurate and biased, but improving its accuracy raises new dangers of targeting and weaponization, a reality Cynthia Khoo called “predatory inclusion”. If he were writing this paper now, Hartzog said, he would acknowledge that it’s become clear that some governments, not just Silicon Valley, see AI as a tool to destroy institutions. I don’t *think* he was looking at the American flags across the water.

Later, Khoo pointed out her paper on current negotiations between the US and Canada to develop a bilateral law enforcement data-sharing agreement under the US CLOUD Act. The result could allow US police to surveil Canadians at home, undermining the country’s constitutional human rights and privacy laws.

In her paper, Clare Huntington proposed deriving approaches to human relationships with robots from family law. It can, she argued, provide analogies to harms such as emotional abuse, isolation, addiction, invasion of privacy, and algorithmic discrimination. In response, Kate Darling, who has long studied human responses to robots, raised an additional factor exacerbating the power imbalance in such cases: companies, “because people think they’re talking to a chatbot when they’re really talking to a company.” That extreme power imbalance is what matters when trying to mitigate risk (see also Sarah Wynn-Williams’ recent book and Congressional testimony on Facebook’s use of data to target vulnerable teens).

In many cases, however, we are not agents deciding to have relationships with robots but what AJung Moon called “incops”, or “incidentally co-present”. In the case of the Estonian Starship delivery robots you can find in cities from San Francisco to Milton Keynes, that broad category includes human drivers, pedestrians, and cyclists who share their spaces. In a study, Adeline Schneider found that white men tended to be more concerned about damage to the robot, where others worried more about communication, the data they captured, safety, and security. Delivery robots are, however, typically designed with only direct users in mind, not others who may have to interact with it.

These are all social problems, not technological ones, as conference chair Kristen Thomasen observed. Carys Craig later modified it: technology “has compounded the problems”.

This is the perennial We Robot question: what makes robots special? What qualities require new laws? Just as we asked about the Internet in 1995, when are robots just new tools for old rope, and when do they bring entirely new problems? In addition, who is responsible in such cases? This was asked in a discussion of Beatrice Panattoni‘s paper on Italian proposals to impose harsher penalties for crime committed with AI or facilitated by robots. The pre-conference workshop raised the same question. We already know the answer: everyone will try to blame someone or everyone else. But in formulating a legal repsonse, will we tinker around the edges or fundamentally question the criminal justice system? Andrew Selbst helpfully summed up: “A law focusing on specific harms impedes a structural view.”

At We Robot 2012, it was novel to push lawyers and engineers to think jointly about policy and robots. Now, as more disciplines join the conversation, familiar Internet problems surface in new forms. Human-robot interaction is a four-dimensional version of human-computer interaction; I got flashbacks to old hacking debates when Elizabeth Joh wondered in response to Panattoni’s paper if transforming a robot into a criminal should be punished; and a discussion of the use of images of medicalized children for decades in fundraising invoked publicity rights and tricky issues of consent.

Also consent-related, lawyers are starting to use generative AI to draft contracts, a step that Katie Szilagyi and Marina Pavlović suggested further diminishes the bargaining power already lost to “clickwrap”. Automation may remove our remaining ability to object from more specialized circumstances than the terms and conditions imposed on us by sites and services. Consent traditionally depends on a now-absent “meeting of minds”.

The arc of We Robot began with enthusiasm for robots, which waned as big data and generative AI became players. Now, robots/AI are appearing as something being done to us.

Illustrations: Detroit, seen across the river from Windsor, Ontario with a Canadian Coast Guard boat in the foreground.

Wendy M. Grossman is the 2013 winner of the Enigma Award. Her Web site has an extensive archive of her books, articles, and music, and an archive of earlier columns in this series. She is a contributing editor for the Plutopia News Network podcast. Follow on Mastodon or Bluesky.

Catoptromancy

It’s a commonly held belief that technology moves fast, and law slowly. This week’s We Robot workshop day gave the opposite impression: these lawyers are moving ahead, while the technology underlying robots is moving slower than we think.

A mainstay of this conference over the years has been Bill Smart‘s and Cindy Grimm‘s demonstrations of the limitations of the technologies that make up robots. This year, that gambit was taken up by Jason Millar and AJung Moon. Their demonstration “robot” comprised six people – one brain, four sensors, and one color sensor. Ordering it to find the purple shirt quickly showed that robot programming isn’t getting any easier. The human “sensors” can receive useful information only as far as their outstretched fingertips, and even then the signal they receive is minimal.

“Many of my students program their robots into a ditch and can’t understand why,” Moon said. It’s the required specificity. For one thing, a color sensor doesn’t see color; it sends a stream of numeric values. It’s all 1s and 0s and tiny engineering decisions whose existence is never registered at the policy level but make all the difference. One of her students, for example, struggled with a robot that kept missing the croissant it was supposed to pick up by 30 centimeters. The explanation turned out to be that the sensor was so slow that the robot was moving a half-second too early, based on historical information. They had to insert a pause before the robot could get it right.

So much of the way we talk about robots and AI misrepresents those inner workings. A robot can’t “smell honey”; it merely has a sensor that’s sensitive to some chemicals and not others. It can’t “see purple” if its sensors are the usual red, green, blue. Even green may not be identifiable to an RGB sensor if the lighting is such that reflections make a shiny green surface look white. Faster and more diverse sensors won’t change the underlying physics. How many lawmakers understand this?

Related: what does it mean to be a robot? Most people attach greater intelligence to things that can move autonomously. But a modern washing machine is smarter than a Roomba, while an iPhone is smarter than either but can’t affect the physical world, as Smart observed at the very first We Robot, in 2012.

This year we are in Canada – to be precise, in Windsor, Ontario, looking across the river to Detroit, Michigan. Canadian law, like the country itself, is a mosaic: common law (inherited from Britain), civil law (inherited from France), and myriad systems of indigenous peoples’ law. Much of the time, said Suzie Dunn, new technology doesn’t require new law so much as reinterpretation and, especially, enforcement of existing law.

“Often you can find criminal law that already applies to digital spaces, but you need to educate the legal system how to apply it,” she said. Analogous: in the late 1990s, editors of the technology section at the Daily Telegraph had a deal-breaking question: “Would this still be a story if it were about the telephone instead of the Internet?”

We can ask that same question about proposed new law. Dunn and Katie Szilagyi asked what robots and AI change that requires a change of approach. They set us to consider scenarios to study this question: an autonomous vehicle kills a cyclist; an autonomous visa system denies entry to a refugee who was identified in her own country as committing a crime when facial recognition software identifies her in images of an illegal LGBTQ protest. In the first case, it’s obvious that all parties will try to blame someone – or everyone – else, probably, as Madeleine Clare Elish suggested in 2016, on the human driver, who becomes the “moral crumple zone”. The second is the kind of case the EU’s AI Act sought to handle by giving individuals the right to meaningful information about the automated decision made about them.

Nadja Pelkey, a curator at Art Windsor-Essex, provided a discussion of AI in a seemingly incompatible context. Citing Georges Bataille, who in 1929 saw museums as mirrors, she invoked the word “catoptromancy”, the use of mirrors in mystical divination. Social and political structures are among the forces that can distort the reflection. So are the many proliferating AI tools such as “personalized experiences” and other types of automation, which she called “adolescent technologies without legal or ethical frameworks in place”.

Where she sees opportunities for AI is in what she called the “invisible archives”. These include much administrative information, material that isn’t digitized, ephemera such as exhibition posters, and publications. She favors small tools and small private models used ethically so they preserve the rights of artists and cultural contexts, and especially consent. In a schematic she outlined a system that can’t be scraped, that allows data to be withdrawn as well as added, and that enables curiosity and exploration. It’s hard to imagine anything less like the “AI” being promulgated by giant companies. *That* type of AI was excoriated in a final panel on technofascism and extractive capitalism.

It’s only later I remember that Pelkey also said that catoptromancy mirrors were first made of polished obsidian.

In other words, black mirrors.

Illustrations: Divination mirror made of polished obsidian by artisans of the Aztec Empire of Mesoamerica between the 15th and 16th centuries (via Wikimedia

A short history of We Robot 2012-

On the eve of We Robot 2025, here are links to my summaries of previous years. 2014 is missing; I didn’t make it that year for family reasons. There was no conference in 2024 in order to move the event back to its original April schedule (covid caused its move to September in 2020). These are my personal impressions; nothing I say here should be taken as representing the conference, its founders, its speakers, or their institutions.

We Robot was co-founded by Michael Froomkin, Ryan Calo, and Ian Kerr to bring together lawyers and engineers to think early about the coming conflicts in robots, law, and policy.

2024 No conference.

2023 The end of cool. After struggling to design a drone delivery service that had any benefits over today’s cycling couriers, we find ourselves less impressed by robot that can do somersaults but not do anything useful.

2022 Insert a human. “Robots” are now “sociotechnical systems”.

Workshop day Coding ethics. The conference struggles to design an ethical robot.

2021 Plausible diversions. How will robots rehape human space?

Workshop day Is the juice worth the squeeze?. We think about how to regulate delivery robots, which will likely have no user-serviceable parts. Title from Woody Hartzog.

2020 (virtual) The zero on the phone. AI exploitation becomes much more visible.

2019 Math, monsters, and metaphors. The trolley problem is dissected; the true danger is less robots than the “pile of math that does some stuff”.

Workshop day The Algernon problem. New participants remind that robots/AI are carrying out the commands of distant owners.

2018 Deception. The conference tries to tease out what makes robots different and revisits Madeleine Clare Elish’s moral crumple zones after the first pedestrian death by self-driving car.

Workshop day Late, noisy, and wrong. Engineers Bill Smart and Cindy Grimm explain why sensors never capture what you think they’re capturing and how AI systems use their data.

2017 Have robot, will legislate. Discussion of risks this year focused on the intermediate sitaution, when automation and human norms clash.

2016 Humans all the way down. Madeline Clare Elish introduces “moral crumple zones”.

Workshop day: The lab and the world. Bill Smart uses conference attendees in formation to show why building a robot is difficult.

2015 Multiplicity. A robot pet dog begs its owner for an upgraded service subscription.

2014 Missed conference

2013 Cautiously apocalyptic. Diversity of approaches to regulation will be needed to handle the diversity of robots.

2012 A really fancy hammer with a gun. Unsentimental engineer Bill Smart provided the title.