Nine years ago, in a commercial AI lab affiliated with Caltech, I witnessed something extraordinary. My colleague Andrej Szenasy was wrapping up a long day’s work training NeuralEye, an AI initially developed for the Mars Rover program, and I was a few cubicles away, plowing through NeuralEye’s test data. “Hey, check this out!” he shouted. Our lab’s mission was to train NeuralEye to see as humans do, with the ability to recognize things, not just record them as a camera does. NeuralEye was built originally to discern different soil types on Mars, but we were teaching it to identify Earth’s inhabitants: animals, plants and individual humans. We believed AI could greatly improve face recognition, so that it could be used in cybersecurity, replacing passwords. The first step in teaching NeuralEye to identify people was to get it to match various photos of a single person’s face. Typically, one photo would reside in NeuralEye’s training dataset of 14,000 faces; another — a different photo of the same person — would serve as the “prompt.” When NeuralEye successfully matched these two photos out of the thousands in its dataset, it got the digital equivalent of a doggie treat. In AI, this method is known as reinforcement learning, and with NeuralEye, it was working. That night in the lab, for fun, Szenasy had prompted NeuralEye with a photo of his son, Zachie. Szenasy’s face was in NeuralEye’s dataset; Zachie’s wasn’t. Zachie, who has Down Syndrome, was a sweet 8-year-old. Round face, thick glasses, mop of black hair. Dad was tall and thin, no glasses, blonde with a receding hairline. If there was a physical resemblance between them, I couldn’t see it. Szenasy sat me in front of his computer and again prompted NeuralEye with a photo of Zachie’s face. NeuralEye spun through its cache of stored faces looking for Zachie —and up popped a photo of Szenasy. Without any specific instruction, NeuralEye had somehow picked up a faint family resemblance. Out of those 14,000 faces, it selected Szenasy’s face as the third closest match with Zachie’s. The next morning I phoned the AI engineer who’d written NeuralEye’s algorithm while at the Jet Propulsion Lab, home of the Mars Rover program. I asked him how NeuralEye could have seen a connection between Zachie and his father. He waxed philosophical for a few minutes, and then, when pressed, admitted he had no clue. That’s the thing about AI: Not even the engineers who build this stuff know exactly how it works. This Zachie episode took place in 2014, a time in AI that now seems prehistoric. Training datasets then had records in the thousands, not hundreds of millions, and large language models like GPT were just a gleam in Sam Altman’s eye. Today, AIs are writing novels, passing the bar exam, piloting warfighter drones. According to a recent University of Texas study widely reported on cable news, an AI in Austin is effectively reading minds: After an in-depth CAT-scan and 16 hours of one-on-one training with someone, it can read neural brain patterns and suggest what the subject is thinking with surprising accuracy. But in those halcyon AI days nearly a decade ago, we in our small lab were amazed that NeuralEye could do something as basic as spot a link between Szenasy and his son. While the best AI scientists obviously know a great deal about AI, certain aspects of today’s thinking machines are beyond anyone’s understanding. Scientists cleverly invented the term “black box” to describe the core of an AI’s brain, to avoid having to explain what’s going on inside it. There’s an element of uncertainty — even unknowability — in AI’s most powerful applications. This uncertainty grows as AIs get faster, smarter and more interconnected. The AI threat is not Hollywood-style killer robots; it’s AIs so fast, smart and efficient that their behavior becomes dangerously unpredictable. As I used to tell potential tech investors, “The one thing we know for certain about AIs is that they will surprise us.” When an AI pulls a rabbit out of its hat unexpectedly, as NeuralEye did on a small scale with Zachie, it raises the specter of runaway AI — the notion that AI will move beyond human control. Runaway AIs could cause sudden changes in power generation, food and water supply, world financial markets, public health and geopolitics. There is no end to the damage AIs could do if they were to leap ahead of us and start making their own arbitrary decisions — perhaps with nudges from bad actors trying to use AI against us. Yet AI risk is only half the story. My years of work in AI have convinced me a huge AI dividend awaits if we can somehow muster the political will to align AI with humanity’s best interests. With so much at stake, it’s time we in the United States got serious about AI policy. We need garden variety federal regulation, sure, but also new models of AI leadership and governance. And we need to consider an idea that would have been unthinkable a year ago. We need to nationalize key parts of AI. Read the rest of Charles’ essay here.
| 
