Quality of physicians responses using a conversational artificial intelligence system: a randomized vignette experiment in Latin America

Physicians using conversational AI in Latin America achieved significantly higher validity scores (median 2.83 vs. 2.46) than usual search methods in clinical case responses.

• Why it matters: Improving point-of-care information retrieval is crucial for enhancing clinical decision-making, especially in low- and middle-income settings where evidence-based resources are limited.
• What they did: A randomized vignette experiment involved 71 physicians answering four simulated cases with or without AI support, scored across six dimensions by blinded specialists, totaling 1,136 responses.
• The result: AI-supported physicians were more likely to meet validity thresholds, with higher median scores and an odds ratio of 3.61, indicating the system's potential to improve response quality in clinical practice.