House, A.I.

The potential and limits of chatbots

A recent study examined the effect of artificial intelligence on physicians’ ability to diagnose patients’ maladies. The diagnoses were done in three ways: doctors alone, doctors with AI, AI alone. The researchers weren't surprised to discover that the doctors with AI did better than the doctors alone. They were surprised that AI alone did better than either the doctors alone or the doctors with AI.

The sample sizes were small, and the study was far from definitive. Yet it was suggestive. And, to some, disturbing.

Why did the AI alone do better than the doctors with AI? Apparently the doctors sometimes overruled AI, rejecting a correct diagnosis. The doctors may have extrapolated unwarrantedly from their individual practices, where AI had access to many practices.

There’s every reason to think the diagnostic abilities of AI will improve rapidly. There's not much reason to think the diagnostic abilities of physicians will improve at the same rate. House, M.D., might be replaced by House, A.I.

Doctors won't go away. They do much more than diagnose ailments. The practice of medicine is a human art as much a biological science. Robots won't match the docs in this for a while, if ever.

I'm a doctor. My graduate school diploma says so. I'm not the kind of doc who can write prescriptions or be of use in a medical emergency, yet as a practicing historian I'm often asked to do something similar to what physicians do when they diagnose disease. Students and readers worried about the health of the American body politic ask me to diagnose what ails our democracy and even to prescribe treatment.

I can't order blood tests and MRIs, but I do gather pertinent evidence. I compare that evidence with evidence from previous moments in history that resemble the present. I draw inferences where appropriate, while acknowledging that the past is never a perfect guide to the present and future.

The physicians have an advantage over me. The collective size of their database is much larger than mine. By the time medical science isolated the cause of malaria, many millions of people had contracted the disease. When I'm asked whether an emergent recession will become a depression, or whether a rise in tension between the United States and China will lead to war, the usefully comparable episodes from the past can be counted on one hand.

The physicians have another advantage in their cases of infectious disease. Each such diseases is caused by a single agent: a virus, a bacterium, a parasite. To be sure, there are contributing factors: genetic, behavioral, environmental. But absent the infecting agent, the disease doesn't appear.

Events in human affairs, especially events involving large numbers of people, are almost always multicausal. This by itself makes a diagnosis difficult. It makes prognosis and prescription harder still. Even if historians agreed on the cause of China's Taiping Rebellion, they’d be at a loss to predict when a modern revolt against China's ruling regime might occur. Too many factors are in play.

The biggest advantage the medical diagnosticians have over the historical diagnosticians is that the consumers of the medical diagnoses have strong incentive to receive the right ones. Accuracy can be a matter of life or death. Is my stubborn cough a cold or cancer?

Moreover, medical knowledge is cumulative. Once medical scientists discovered that malaria is caused by the Plasmodium parasite—rather than bad air, as the name implies—treatments and prophylaxis could focus on it.

Historical diagnosticians operate under different rules. There is no “right” answer as to what caused the industrial revolution. There are lots of answers that are partially right. And the weights accorded the different answers change over time. The study of history doesn't steadily advance the way the study of medicine does. In fact, it's not clear it advances at all. Old explanations get recycled. Karl Marx has been in and out of fashion half a dozen times since the 1840s.

Moreover, lives don't hang upon the outcome of historical debates. In their day, it mattered greatly whether Grant was a better general than Lee. But after Lee's surrender, the debate over their talents was often vigorous but never consequential.

Which is why I'm not worried that historians are about to be replaced by chatbots. It's not that we’re too valuable to be replaced. It's that we're not valuable enough. Whichever AI company builds the best medical chatbot stands to make trillions of dollars. The best historical chatbot might put the inventor's kid through college.