83%: The Accuracy Gap That Could Reshape Medical Diagnosis

For patients navigating the complexities of modern healthcare, the landscape of diagnostic support is shifting. A new study published in Science reveals that an artificial intelligence system developed by Microsoft correctly identified complex medical conditions in a simulation with an 83% success rate. This figure significantly surpasses the 36% accuracy recorded by generalist physicians working individually with standard reference materials. While these difficult cases are typically managed by specialized teams, the results suggest that AI could soon serve as a critical decision-support tool for clinicians dealing with ambiguous symptoms.

The system, known as MAI-DxO, functions not as a single algorithm but as a coordinated panel of AI agents. These agents mimic the workflow of a medical board, interrogating patient data, ordering virtual tests, and debating potential diagnoses among themselves. Microsoft AI CEO Mustafa Suleyman described this architecture as a foundational step toward achieving medical superintelligence, a concept defined as the ability of AI to surpass the strongest current human-led health systems in both reasoning and coordination across the entire care cycle.

The performance metrics highlight a clear divergence based on case complexity and the underlying model used. When tested using complex cases from the New England Journal of Medicine, the system’s accuracy varied depending on the foundation model. Using OpenAI’s GPT-5.5, the orchestration system boosted accuracy from 72% to nearly 87%. In contrast, on standard emergency department cases, the improvement was minimal, with both the standalone model and the orchestrated system scoring around 85%. This suggests that the added value of the multi-agent debate is most pronounced in scenarios where diagnostic uncertainty is highest.

Cost remains a critical factor in deploying such technology at scale. The study measured the estimated cost of running the AI for each case. While GPT-5 alone achieved a 78.9% accuracy rate at a cost of approximately $8,400 per case, the DxO system reduced that cost by roughly $3,000 while adding only about four percentage points to the accuracy score. For less capable models, such as Gemini 2.5 Flash, the orchestration system provided a more substantial lift, raising accuracy from 46.4% to nearly 69%, though the gap between weaker and stronger models remained larger than the boost provided by the system.

Microsoft is not acting alone in this push. The company is currently working with the Mayo Clinic to train a dedicated foundation model for healthcare, with a release planned for next year. Suleyman believes this specialized model will enable significant leaps in capability, potentially allowing for the streaming of highly accurate predictions about how medical records might unfold. He characterized this foresight as invaluable for clinicians, comparing the expected progress to the rapid improvements seen in coding and chat-based AI in recent years.

To validate these laboratory results, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston for a real-world evaluation. The study will run in parallel with standard patient care, comparing the recommendations of the DxO system against those of human doctors in emergency and urgent care settings. Dominic King, vice president of Health at Microsoft AI, noted that the company aims to provide compelling evidence on the system's ability to be deployed at scale, addressing the growing mismatch between the demand for medical expertise and the available supply.

This development places Microsoft in a competitive space alongside Google and OpenAI, both of which have recently claimed similar capabilities for their AI systems. Google, for instance, recently published a study in The Lancet evaluating its AMIE diagnostic AI in urgent care visits. Microsoft’s approach, however, emphasizes integration into the broader clinical workflow rather than just answering isolated medical questions. The company already sells dictation and radiology tools to clinicians, viewing DxO as a natural expansion of its existing healthcare portfolio.

Suleyman, who previously co-founded DeepMind and led its health initiatives, expressed strong confidence in the trajectory of the technology. He expects healthcare models to become an order of magnitude better within the next two to three years, driven by a flywheel effect where models are iterated upon using real-world usage data. The goal is to move beyond benchmarking individual tasks to creating a system that is deeply integrated into the daily operations of hospitals, ensuring that the technology is not just accurate in simulations but effective in the chaotic, high-stakes environment of actual patient care.