Why this gap matters now
An academic misconduct finding can delay a degree, end a scholarship or damage a career. When a detector score is one of the pieces of evidence behind that finding, its error rate stops being a technical detail and becomes a question of procedural fairness. The early evidence was stark: in one widely cited study, most essays written by non-native English writers were labelled as machine-generated (Liang et al., 2023). Some institutions responded by switching detectors off. Vanderbilt University, for example, disabled Turnitin's AI detector, noting that even a 1% false-positive rate could affect around 750 of the 75,000 papers it had submitted in 2022 (Vanderbilt University, 2023).
Yet the literature has not settled. Later work using large, carefully sampled assessment data found no bias against non-native speakers (Jiang et al., 2024), while a study of scholarly abstracts found that bias appeared mainly when humans used AI to polish their own writing (Pratama, 2025). These results are not necessarily in conflict; they test different tools, texts and thresholds. That is exactly what makes this an evidence gap in the sense described by Müller-Bloch and Kranz (2015) and Miles (2017).
Why the gap persists
Three features of the field keep the question open. First, most studies use convenience corpora: public essay sets, test responses or published abstracts. These are easier to obtain than coursework, but they differ in length, genre and the writing support students actually use. Second, detectors change quickly, and so do the models they try to catch, so findings date fast (Dugan et al., 2024). Third, the decisions that matter happen inside confidential integrity processes, which researchers rarely see.
- Commercial vendors seldom publish method details or error rates by writer group.
- Ground truth is hard to establish: a researcher must know, not guess, that a text is human-written.
- Access to case records requires institutional trust, data-protection safeguards and ethics approval.
The result is a body of evidence strong on what detectors do in the laboratory and thin on what they do to students in practice (Weber-Wulff et al., 2023; Perkins et al., 2024).
Is this feasible for a doctorate?
Yes, if the scope is drawn tightly. A realistic thesis might combine one benchmark study on verified human coursework from two or three writer groups, one controlled experiment on assistive tools, and one qualitative or vignette study with integrity staff. Each strand can produce a publishable paper, and together they answer a single question about fairness in decision-making.
| Consideration | What to plan for |
|---|---|
| Data access | Consent-based collection of drafts and final texts, or a partnership with an integrity office. |
| Ethics | Approval covering student data, anonymisation and the risk of re-identification in case records. |
| Tool versions | Record detector versions and dates; re-test before submission because tools change. |
| Skills | Classification metrics, fairness measures, and qualitative or mixed-methods analysis. |
| Disciplinary home | Education, applied linguistics, information science or computer science, with co-supervision across them. |
A focused study of this kind would be valuable whatever it finds. Showing that bias is small under realistic conditions matters as much as showing that it is large, because either result tells institutions how much weight a score can bear.
