As you approach the summit, the first thing you notice is how unremarkable everyone appears. No swagger from a DEF CON afterparty, no hoodies pulled low. Outside a conference hall that could have been the site of a dental convention the previous week, there were only engineers, behavioral scientists, product managers, and a few academics wearing slightly rumpled blazers, drinking bad coffee from paper cups. The badges are labeled “Persuasive Technology.” If you lean in, you can see that the conversations are about unfamiliar topics.
These are the individuals that Ramsey Brown referred to as “brain hackers” when he appeared on camera for 60 Minutes and the term became popular. Brown was a young programmer who had become uneasy with what he and his colleagues were creating—the tiny tricks that turn idle thumbs into hours of scrolling, the slot-machine logic built into apps. The field he warned about has developed into an industry with its own peer-reviewed journals, conferences, and quiet professional culture years later. They all appear at the summit.
A first-time observer is taken aback by how scholarly the atmosphere seems. Citing the kinds of frameworks Rosana Montañez and her colleagues outlined in their paper on social engineering and human cognition a few years ago, a researcher from a university lab presents work on cognitive load and phishing susceptibility. The neurocognitive triggers that John Heslen described in his work on what he called neurocognitive hacking—subliminal cues that push people toward suspicion of out-group members—are covered in another talk, which is drier than it sounds. The audience gives a nod. Some use iPads to take notes. No one appears surprised.
Sitting in the background of these sessions, there’s a feeling that the distinction between manipulation and persuasion design has been so blurred for so long that it’s no longer worth debating. For them, what works is the intriguing question. Which notification sound, which copy, which color, and which delay? The workshop on engagement loops attracts a larger audience than the ethics panel, which is scheduled for late on the second afternoon.
Naturally, the AI discussion has altered everything. You can hear it in fragments as you walk the hallways: generative tools that can create a convincing voice clone of a chief executive in less than a minute, large language models being trained to write phishing emails that feel personal. A few weeks ago, The Economist published an article about a hacking conference where machines were shown protecting networks from other machines. With machines learning to read people more accurately than people can read themselves, the summit seems to be the opposite.
Drawing on the ResearchGate paper from a few years ago, one presenter—a woman who once worked at a consumer app you’ve undoubtedly used—discusses something she refers to as the prevalence paradox. The basic idea is that we are more likely to miss an attack if it seems rarer. The one that has been tailored to their precise emotional state will be easily overlooked by people who are scanning their inboxes for the obvious scam. She says it without sounding alarmed. She describes a feature in the manner of an engineer.
It’s difficult to avoid feeling a little uneasy after reaching the summit. None of the people in the room appeared menacing. The majority of them were reflective, and some of them were blatantly conflicted, just like Brown was when he first began speaking in public years ago. The discomfort stems from how commonplace everything has become. The way it is framed has changed. Optimization is the new term for what was formerly known as manipulation. What was formerly referred to as a trick is now called a pattern.
Two attendees are quietly debating whether their company’s new onboarding process goes too far as they stand by a rental car in the parking lot outside the hall. With a shrug, one of them settles into the passenger seat. The other person lights a cigarette. They don’t appear to anticipate a response.
