Someone or something attempted to persuade you to change your mind at some point between when you last updated your feed and when you began reading this sentence. Perhaps it was successful. Perhaps it didn’t. The disturbing thing is that you can no longer truly take the measurement.
Researchers at the University of Zurich conducted a covert experiment on the r/ChangeMyView subreddit last year, using AI-generated comments intended to influence viewpoints on contentious issues. No one in the community was informed. After learning about it, moderators were, by most accounts, incensed. In the end, the researchers made a sort of apology, using the kind of carefully crafted statement that academics make when they’ve stepped over a boundary they didn’t fully understand. Although that is important, the violation of consent is not what remains from that incident. It’s the informality of it. The idea that a small group of individuals operating out of a single European university could penetrate one of the most contentious areas of the internet and prod actual people without anyone noticing for months.
The distinction between “AI safety research” and “running uncontrolled experiments on the public” has essentially vanished, according to a more subdued argument among researchers who focus on these topics professionally. Every recommendation system is based on conjecture. Each launch of a chatbot is a multivariate test. The labs are no longer hidden behind glass. They are on the platform your adolescent refuses to put down at dinner, in your pocket, and in your group chats.
The AI safety researcher Dr. Roman Yampolskiy, who has spent years cautioning about systems we don’t understand, has publicly stated that he is almost positive we are in a simulation. The headlines are generated by that. In some ways, it’s more difficult to refute the less popular theory that we’re in a persuasion experiment because every significant tech company’s engagement metrics provide the proof.
Considering who gains from the framing is worthwhile. As astrophysicist Franco Vazza noted earlier this year, the simulation hypothesis typically originates from the tech sector—people who would like you to think that their machines are capable of creating something as rich as reality. In contrast, the persuasion experiment is not conjectural. It is recorded. Nearly ten years ago, Cambridge Analytica was a clumsy forerunner, the Wright brothers’ glider in a field that has since invented supersonic flight. It now appears almost charming. The current system is much faster, much more customized, and mostly undetectable. It appears that investors are okay with this. Regulators appear to think that someone else will take care of it.
Most of the scientists working on the project are not bad people. Speak with any of them, and you’ll get the familiar mix of sincere interest, genuine worry, and the belief that someone worse will research this if they don’t. Over the past century, a lot of human research has been justified by this argument; some of it has been praised, while others are now taught as cautionary tales in ethics courses. It’s difficult to ignore the pattern. Usually, the damage comes before the framework.
What does living this way mean? In actuality, not much changes every day. You vote, read the news, argue with strangers, and are persuaded to buy a new pair of running shoes. Every one of those tiny deeds is now data that is fed back into systems that will come back tomorrow with a marginally improved ability to convince the next you. Keeping all of this in mind, there’s a sense that the experiment’s most effective trick has been persuading its participants that nothing out of the ordinary is occurring. The labs are available. There was never a lock on the doors. We simply gave up trying to find them.
