ChatGPT has doctors thinking about how bots can deliver better patient care. But when it comes to the bigger picture of integrating artificial intelligence into health care, Gary Marcus, an AI entrepreneur and professor emeritus of psychology and neuroscience at NYU, is worried the government isn’t paying close enough attention. “My biggest concern right now is there's a lot of technology that is not very well regulated— like chat search engines that could give people medical advice,” he told Digital Future Daily. There aren’t any systems in place, he said, to monitor how these new technologies are being used and assess whether they’re causing harm. Marcus is both a doomsayer and an optimist. He believes that the future of medicine involves personalized medicine delivered by artificial intelligence. But he also wants to see proof that it works. “We need to have actual clinical data here,” he said. And, he said, systematic studies need to answer key questions before unleashing this technology on the public: “How well do these things work? In what contexts do they work? When do they not work?” In his new podcast, Humans vs. Machines, the first story he tells is of the rise and fall of IBM’s Watson, a technology that was supposed to revolutionize the field of health care and ultimately failed. In 2011, Watson, an AI platform designed by IBM, famously won Jeopardy. While Watson was still warm under the glow of the spotlight, the company quickly tried to figure out how it could capitalize on the moment. So it pivoted to health care and struck up a deal with Memorial Sloan Kettering to come up with personalized cancer treatments for patients. The episode outlines many problems with this project, including, perhaps, IBM’s narrow focus on generating profit over validating its technology. But, Marcus said, the bigger issue was that Watson could not do what humans do, which is to be able to come to their own conclusions. “Watson was not a general purpose AI system that, for example, could read the medical literature with comprehension,” he said. Watson was like a search tool on steroids, he said, good at locating information relevant to a specific topic. When doctors diagnose someone they are taking what they know about the individual patient along with existing medical literature and molecular biology and trying to come up with a solution. “That requires a sophisticated level of reasoning and Watson just wasn't capable of doing that— it was not a reasoning engine,” he said. Watson may feel like an irrelevant precursor to current bots like ChatGPT. But even the latest bots suffer from these same problems. They can’t think on their feet. “One of the things people forget about machines is that they're not people,” said Marcus. He’s worried that tools like ChatGPT are really good at convincing people that they are human, as evidenced by Kevin Roose’s conversation with Bing. In another instance, a Belgian man ended his life after talking to a bot named Eliza about climate change. Marcus is concerned people will turn to these online bots not only for medical advice, but for mental health treatment. “We're messing with human beings here when we portray these systems as being more human than they are,” he said. That’s why he sees a need for regulators to get involved. Regulators are starting to step in. The Food and Drug Administration has taken steps to regulate artificial intelligence in medicine, particularly algorithms that offer up a diagnosis or make recommendations to doctors about patient care. And in niche contexts that kind of AI is making progress. For example, research shows progress in helping doctors spot sepsis. But when it comes to online chatbots that offer up bad information, regulators are totally out of the loop. “We know these systems hallucinate, they make stuff up,” Marcus said. “There's a big question in my mind about what people will do with the chat style searches and whether they can restrict them so that they don't give a lot of bad advice.”
| 
