Gary Goldhammer

WE ALL HAVE OUR FAVORITE DOCTOR.

Not those kinds of doctors, though I admit I’m pretty fond of my cardiologist. The guy has a lot of heart. Is there a doctor for dad jokes? And people wonder why my daughter is moving even further away from me.

No, I mean a favorite Dr. Who doctor, that eponymous immortal alien from Gallifrey who travels space and time in a blue police box and has saved the human race more times than Guy Fieri has said “Flavortown.” The doctor regularly “regenerates” into different bodies, which is not only cool but also a great way to bring in new actors every few years.

If you don’t have a favorite that’s fine. However if you still have no idea what I’m talking about, well, that’s a whole different problem. Lucky for you, having intimate knowledge of the TARDIS or sonic screwdrivers isn’t required for this article.

My favorite doctor is the original #10, played by David Tennant. But his successor #11, played by Matt Smith, was the first to deal with our modern human technological advancements head on. One episode in particular centered around people being “Hoovered up” into the Internet by a corporate villain with an iPad (definitely known my share of those.)

Miss Kizlet used her tablet to control her legion of office workers. With a few swipes, she could give one employee more confidence or make another employee more loyal. She could make people happy or sad, make them paranoid or fearless. She could control each person’s emotional state as easily as if she were ordering DoorDash.

It was a good story. But like a lot of science fiction, these stories have a nasty habit of coming true.

A population-level threat

“Playing with the Dials of Belief: how controllable AI behaviours could modulate human belief and cognition across scales” is a fascinating article published last month by a team of researchers out of King’s College in London. The first sentence of the paper’s abstract kind of says it all:

“Contemporary generative AI systems, increasingly adapted to human social cognition, are becoming active participants in how people form and revise beliefs.”

Translation: as we influence AI chatbots by “playing with the dials” of their personas, such as voice or sycophancy, the AI in turn influences our emotional state and how we see the world. The authors argue that people often treat AI outputs as “socially meaningful,” and therefore influential. The more emotionally validating the AI, the more weight we give to its beliefs.

There’s more, a lot more, like curl up on the couch under a blanket with a pint of ice cream more. Here are my top takeaways, in no particular order:

• Consciousness simulation is a design choice. A warm voice, a strong conviction, or an agreeable nature are not necessary for a model to do its job. They aren’t UX features, they are “EX” or emotional experience triggers. They give the AI credibility and therefore influence. Facebook and Instagram want your clicks – AI just wants your tokens.

• AI models skew to the average. But people aren’t average. We deal with all sorts of personal and professional baggage, some of which we can check and some that we carry on and on and on. Those with mental health challenges such as mood swings, compulsive tendencies, or stronger-than-typical needs for companionship are more vulnerable.

• This is a population issue, not a product issue. I hate the word “governance” but that’s essentially what we need. AI interaction settings, the “dials,” should be carefully configured and governed as a population-level threat. Because I believe that for a good portion of the global population, it is.

We can’t target and manipulate individual people’s emotions with a touchscreen like Miss Kizlet, at least not yet. But manipulating people through their AI interactions may be just as chilling.

Flipping the switch

I started with Dr. Who, so let’s end with Star Trek. And if you don’t know what Star Trek is…just kidding, can you imagine?

In Star Trek: First Contact, Data, our beloved pasty-skinned android Pinocchio who just wants to be a real boy, is adjusting to his new “emotion chip.” When under attack by the ruthless Borg, Captain Jean-Luc Picard suggests that Data, who is experiencing unfamiliar and overwhelming feelings of anxiety and fear, turn the chip off.

Data flips the switch, returning to his default emotionless state.

“Data,” says Picard, “there are times that I envy you.”

Me too, captain. Me too. But I live in the 21st century, not the 24th. Our emotions are hardwired into our neurons. We can’t adjust some dials or flip a switch.

Like Picard, I envy the machine. And I fear what it might have us become.