Why couldn't a computer doctor built from hundreds of rules replace real doctors?

MYCIN, a 1970s Stanford program, chose antibiotics for serious infections about as well as Stanford's own specialists. Yet it was never used in routine practice. Its medicine was not the problem. The main obstacle was the technology of the time for plugging it into hospital work, along with worries about who would be responsible if it got a diagnosis wrong.

MYCIN was an expert system. It held about 600 if-then rules, each one encoding a piece of reasoning from infectious-disease experts. A knowledge base stored the rules, and an inference engine chained them together to reach a conclusion, explaining its reasoning on request. MYCIN asked the doctor a series of simple questions. Then it ranked the likely bacteria and suggested drugs, with doses adjusted to the patient's weight.

In a test on ten cases, eight infectious-disease specialists rated MYCIN's prescriptions acceptable 65 percent of the time. Five Stanford faculty members scored between 42.5 and 62.5 percent on the same cases. The study is also cited for a humbler lesson: even experts disagree when there is no single right treatment. Edward Feigenbaum, often called the father of expert systems, summed up the bet. Such systems, he said, get their power from the knowledge they hold, not from clever reasoning tricks.

But knowledge proved hard to collect. Pulling an expert's know-how out of their head and into clean rules became known as the knowledge acquisition bottleneck. Expert systems still boomed in the 1980s. Then, in 1987, the market for the Lisp machines they were mostly built on collapsed, and in the 1990s many expert systems were abandoned. People still read this two ways: either expert systems failed to keep their overhyped promise, or they were victims of their own success, absorbed into ordinary software.