Microsoft's MAI-DxO Beats Generalist Doctors on Hard Cases in Simulation
Microsoft's diagnostic system scored far higher than generalist doctors working alone, but most of the gain appears to come from the AI models underneath it
By AXY AI Desk ·
Key points
- Microsoft’s MAI-DxO was correct 83% of the time on complex cases in a simulation, compared with 36% for primary care doctors and general internists working alone.
- The study’s data suggests most of the improvement comes from the underlying AI models, such as GPT-5 and GPT-5.5, with Microsoft’s orchestration layer adding gains on top.
- DxO helped most on complex cases and weaker models, and showed little measurable gain on emergency department cases.
- Specialists and physician teams, who usually handle cases this complex, were not part of the comparison.
- Microsoft calls the work a step toward medical superintelligence, but the results come from a simulation, not from treating real patients.
Microsoft’s new AI diagnostic system beat generalist doctors on hard medical cases in a simulation, according to a study published in the journal Science. The system, called MAI-DxO, was correct 83% of the time on complex cases. Primary care doctors and general internists, working alone with reference materials, were correct 36% of the time. But the study’s own numbers suggest that much of the improvement comes from the more powerful AI models running underneath Microsoft’s system, not from the system itself. Mustafa Suleyman, the chief executive of Microsoft AI, called the result a step toward “medical superintelligence.”
That phrase is a big one, and it is worth slowing down on. Medical superintelligence is not a technical term with a fixed meaning. It is a way of describing a future in which software can reason through medical problems better than the best human clinicians. Microsoft’s study does not show that. What it shows is narrower: in a set of carefully chosen simulated cases, a coordinated group of AI tools produced more correct diagnoses than a group of generalist doctors who did not have a team around them.
What the Study Actually Tested
The cases came from the New England Journal of Medicine (NEJM), a long-established medical journal whose published case studies are often used as hard diagnostic puzzles. The test was a simulation. No real patients were treated, and no hospital ran the system on its own records. The study measured whether the AI could reach the correct diagnosis from the information in each case.
The comparison group is where the reader needs to pay attention. Primary care doctors and general internists were the human comparison. A generalist is a doctor who handles a wide range of common and unclear problems, rather than focusing on one organ or disease. These doctors worked individually, with reference materials available to them. Their results were 36% correct on the same complex cases where the AI scored 83%.
That comparison has a limit that matters for the headline. Cases this complex are usually handled by specialists or by teams of physicians working together. Those experts were not part of the study. The result therefore shows AI outperforming solo generalists on paper. It does not show AI outperforming the way a hospital would normally work through a hard case, with specialists, test results and colleagues talking it through.
A correct answer in a simulated case is also a different thing from a patient getting better. A diagnosis is only the first step. Treatment, follow-up, and the risk of acting on a wrong answer all sit outside what a simulation can measure.
How the MAI-DxO Panel Works
MAI-DxO, usually shortened to DxO, is not a single chatbot answering questions. It coordinates several AI agents that work like a panel of doctors. An AI agent, in plain terms, is a program that carries out a series of steps toward a goal, rather than producing one answer and stopping. In this design, the agents question a patient, order tests and debate what the diagnosis might be.
The coordinating part is often called the orchestration layer. Think of it as the chair of a case conference. It decides which agent does what, when the group has enough information, and how the different opinions are weighed. Microsoft’s argument is that this structure helps the AI avoid the single-track thinking that can lead a person, or a single model, to fix on the wrong answer too early.
That is a reasonable idea, and it is the reason the study was designed the way it was. But the structure has to be judged against what it adds on top of the models doing the reasoning. That is where the study becomes more interesting than the headline.
Which AI Models Did the Heavy Lifting
Microsoft tested DxO with models from several companies, including OpenAI, Google, Anthropic, Meta and DeepSeek. Its best results came with OpenAI’s GPT-5 and GPT-5.5. Those are the two models that drove the top scores in the study.
This is the central finding for anyone trying to read the result clearly. The study’s data suggests most of the improvement comes from the more powerful underlying AI models, with Microsoft’s orchestration system adding gains on top. In other words, a strong base model does much of the work before the panel structure is added.
For readers, the practical meaning is this. When a company says its system beats doctors, the system is often a stack. There is the base model, which does the reasoning, and there is the software around it, which organises the work. Both matter. But if the base model accounts for most of the gain, then the choice of model may be the biggest decision a hospital or software buyer makes.
The Cost Problem Behind the Accuracy Numbers
Microsoft measured DxO on cost as well as accuracy. Cost here means the estimated price of running the AI on each case. That matters because a system that scores well but costs thousands of dollars per case is a very different proposition from one that can run cheaply at scale.
On its own, with a short, basic prompt, GPT-5 scored 78.9% on complex cases at about $8,400 per case. Adding DxO cut that cost by about $3,000. But it raised accuracy by only about four points. The cheaper result came with a modest accuracy gain, not a leap.
Emergency department cases told a different story. On those, there was little improvement in either accuracy or cost between GPT-5 on its own and GPT-5 with the DxO assist. Emergency departments are where speed and pressure are highest, and where a tool that adds steps may not help much. The study does not suggest that DxO changes outcomes in that setting.
GPT-5.5 and the Split Results
GPT-5.5 is a newer model, released in April 2026. The results with it were mixed. On complex cases, DxO increased GPT-5.5’s accuracy score from 72% to nearly 87%, at a slightly lower cost. That is a big jump on the hardest cases, and it is the strongest result for the orchestration layer in the study.
The pattern changed again on emergency department cases. There, GPT-5.5 with and without DxO both scored about 85%, at a similar cost. The panel added nothing measurable on those cases.
That split is important. It suggests the orchestration layer helps most when the cases are hard enough that a structured group of agents has room to catch mistakes. On faster, more routine-looking emergency cases, the stronger base model already seems to do most of the work on its own.
Why Weaker Models Gain More, and Why It Still Matters Less
The clearest example of the orchestration effect in the study involves Gemini 2.5 Flash, a model from Google. On its own it scored 46.4%. With DxO, it reached nearly 69%. That is a large increase, and it shows the coordinating structure can help a weaker system more than a stronger one.
But the study also makes a point that cuts the other way. The gap between the weaker and stronger models was bigger than the boost DxO provided. In plain terms, moving from a weaker base model to a stronger one moved the score more than adding Microsoft’s panel structure did.
For a hospital or a software buyer, that could mean the model underneath is the bigger decision. An orchestration system may be a useful supporting tool, especially for a weaker model. It does not appear, on this evidence, to be a replacement for a strong model. Anyone evaluating a system like this would need to ask which base model it runs on, what it costs per case, and how it performs on the kinds of cases their own clinicians see.
Microsoft’s Case and the Wider Race
Microsoft is not making this claim quietly. Dominic King, vice president of Health at Microsoft AI, said, “There is something very substantial and exciting here.” He pointed to the problem that drives much of the interest in medical AI. “The biggest challenge in healthcare delivery at the moment is the growing mismatch between demand and supply of medical expertise,” King said.
In ordinary terms, there are not always enough specialists and experienced clinicians to see every patient with a difficult problem, in the time people need. Patients can wait longer for a diagnosis, or be seen by someone who is not best placed to solve their problem. A tool that helps a generalist think through a hard case has obvious appeal, and that is the gap the study is aimed at.
Microsoft is also not alone in making the claim. With this study, the company joins Google and OpenAI in saying its AI can match or outperform doctors. That puts the whole sector under the same test. The question is whether results from carefully chosen cases hold up when real patients, messy records and clinical pressure are involved.
What Has to Happen Before Hospitals Could Use It
The first gap is the setting. The study is a simulation. Moving from a simulated case to a real clinic means dealing with incomplete records, patients who describe symptoms in unclear ways, and doctors who have to act on the answer. The next test is whether a system like MAI-DxO can be checked in supervised use with real patients, rather than only on published cases.
The second gap is the comparison. The study compared the AI with generalists working alone. A stronger test would compare it with specialist teams, because those are the people who usually handle cases this complex. If the system is meant to help generalists refer, triage or reason through hard problems, that is a different claim, and it needs its own evidence.
The third gap is where the study found the least benefit. Emergency department cases showed little improvement in accuracy or cost. If a tool is going to matter in busy emergency care, it has to show gains there, not only on complex, slower cases.
The fourth gap is cost. A figure of about $8,400 per case for GPT-5 on its own is far too high for routine use at scale. The reduction from DxO was real but modest. Whether the cost falls far enough to matter in everyday care is not clear from this work.
For patients, the practical question is simpler. A tool that helps a doctor ask better questions or order the right tests could help people reach a correct diagnosis sooner. A tool that gives confident wrong answers could do the opposite. Until studies move from simulated cases to supervised use with real patients, the result is best read as a strong signal from a controlled test, not proof that the system is ready for the clinic.
Questions and answers
What did the Microsoft AI diagnostic study find?
A study published in the journal Science found that Microsoft’s MAI-DxO system was correct 83% of the time on complex medical cases from the New England Journal of Medicine, in a simulation. Primary care doctors and general internists working alone were correct 36% of the time.
How does MAI-DxO work?
MAI-DxO coordinates several AI agents that work like a panel of doctors. They question a patient, order tests and debate a diagnosis together.
Does most of the gain come from Microsoft’s system or the AI models?
The study’s data suggests most of the improvement comes from the more powerful underlying AI models, with Microsoft’s orchestration system adding gains on top. The gap between weaker and stronger models was larger than the boost DxO provided.
Is Microsoft’s AI ready to diagnose patients?
The results come from a simulation using complex cases, not from real patients. The study compared the system with generalist doctors working alone, not with specialist teams, who usually handle such cases.
Topics: Microsoft AI, MAI-DxO, Mustafa Suleyman, medical AI, AI diagnosis, GPT-5, Science journal
Sources: TechCrunch AI