Why this gap matters now
Language models have moved from pilots to daily work. Organisational use of AI rose from 55% to 78% in a single year (Maslej, 2025), and two-thirds of US physicians surveyed in 2024 reported using AI in practice (American Medical Association, 2025). Vendors have responded by grounding models in retrieved documents and by adding verification layers, and the research community has produced detectors, benchmarks and surveys (Huang et al., 2025; Farquhar et al., 2024).
Yet the numbers that decision-makers need are the hardest to find. Published rates range from about 1% in carefully engineered clinical summarisation (Asgari et al., 2025) to more than half of answers for general models on legal questions (Dahl et al., 2024). These figures are not contradictory: they measure different tasks, data and definitions. That is the gap. Without a standard for what counts as a consequential error, and without measurement on real cases, no one can say how safe a specific deployment is.
Why the gap persists
Three obstacles keep the question open. First, real workflow data are sensitive, so most studies use exam questions, vignettes or mock consultations; only 5% of healthcare LLM evaluations used real patient data (Bedi et al., 2025). Second, hallucination is defined and scored differently from study to study, which a recent review identifies as a barrier to comparison (Basu & Huynh, 2026). Third, systems change quickly, so a result can date before it is published.
- Commercial tools rarely publish error rates by task or severity, which is why independent audits matter (Magesh et al., 2025).
- Detection methods are mostly tested on short question-answer pairs, not long professional documents.
- Evaluation leaderboards can reward guessing, which works against calibrated abstention (Kalai et al., 2025).
The result is a field that is strong on benchmarks and detection methods and thin on evidence from live use. Hard hallucinations, the ones that sit closest to the truth, are also the hardest for automatic detectors to catch (Pandit et al., 2025), which is why human review and its limits belong in the same study.
Is this feasible for a doctorate?
Yes, if the project is scoped to one task in one setting. A realistic thesis could combine an expert consensus study on error severity, a preregistered measurement study on real cases, and an experiment on detection by humans or uncertainty signals. Each strand can stand as a paper, and together they answer one question: how often does this system get it dangerously wrong, and would anyone notice?
| Consideration | What to plan for |
|---|---|
| Data access | A partner organisation, a data-sharing agreement and de-identification before any annotation. |
| Ethics and governance | Approval covering real records, annotator welfare and any use of external model APIs. |
| Annotation | At least two blinded domain experts per item and an agreement statistic. |
| Model versions | Record model, prompt and retrieval versions with dates; re-run key tests before submission. |
| Skills | NLP evaluation, statistics for rare events, and qualitative or mixed methods for the human strand. |
