The source and why it matters
The paper under review is Baig and Yadegaridehkordi (2024), "ChatGPT in the higher education: A systematic literature review and research challenges", published open access in the International Journal of Educational Research (volume 127, article 102411). Both authors are based at CQUniversity in Sydney. The review sets out to explore the trends, adoption measures, applications and current limitations of research on ChatGPT in higher education.
It matters to doctoral researchers for two reasons. First, it is widely cited as a map of the first wave of ChatGPT studies, so many literature reviews on generative artificial intelligence (AI) in education will meet it. Second, it is a useful teaching case: a review conducted quickly on a fast-moving topic, where the strengths and weaknesses of systematic review methods are easy to see. Reading it critically is good practice for appraising any secondary study you intend to rely on.

What the review did and found
The authors followed the systematic literature review guidelines of Kitchenham and Charters (2007), which were developed for software engineering. They searched publisher databases including Emerald, ERIC, MDPI, SAGE, Elsevier, SpringerLink, Frontiers, PLOS ONE, Wiley and Taylor & Francis, added a manual Google Scholar search, and included English-language research articles on ChatGPT in higher education published between January 2023 and mid-January 2024. Reviews, editorials, conference papers and studies of school or pre-university education were excluded. After screening and a quality assessment, 57 articles were analysed (Baig & Yadegaridehkordi, 2024).
The review was organised around four questions. The table summarises what it reported for each.
| Research question | Main findings reported |
|---|---|
| What are the trends in ChatGPT research in higher education? | Rapid growth through 2023 (a peak of 12 articles in September 2023); studies from more than 25 countries; quantitative designs most common (49%), then qualitative (18%) and mixed methods (3%), with 30% not stating a method; surveys the main data collection tool (44%) |
| How have adoption and use been measured? | Studies grouped into post-adoption (continued use), intention to use, and acceptance and use; the Technology Acceptance Model (TAM), UTAUT and UTAUT2 most often applied; social influence, perceived usefulness, effort expectancy and facilitating conditions most often reported as significant |
| How is ChatGPT applied? | Academic staff: designing assignments and examinations, supervision, lesson planning, curriculum development. Students: language learning, writing support, coding help, personalised learning. Researchers: idea generation, data analysis, writing and editing. Non-academic staff: content generation and administrative support |
| What are the limitations and future directions? | Few studies of staff and non-academic users; heavy reliance on student surveys; a need for qualitative and mixed methods; concerns about privacy and plagiarism needing deeper study; uneven geographic coverage |
The screening process can be reconstructed from the paper as a simple flow.
- 1209 records from database searches
- 242 after filtering by publication type
- 333 after inclusion and exclusion criteria
- 463 after adding a manual Google Scholar search
- 557 after removing six studies rated weak
Strengths
- Timely and transparent scope. The search strings, databases, dates and inclusion criteria are stated, so a reader can see exactly which slice of the literature is covered.
- Broad view of stakeholders. By separating applications for academic staff, students, researchers and non-academic staff, the review moves beyond the student-only focus of many single studies and exposes how little was known about staff use.
- Honest about the evidence base. Reporting that 30% of included studies did not state a method, and that surveys dominated, gives readers the information they need to discount weak findings.
- Useful research agenda. The closing challenges (more qualitative work, more stakeholders, privacy and plagiarism) are specific enough to guide new projects, including doctoral ones.
Limitations and methodological issues
What the quality assessment measured
Each study was scored on four criteria: whether it explains ChatGPT, addresses higher education, is a research article and states its method (Baig & Yadegaridehkordi, 2024). These are useful relevance and reporting checks, but they do not appraise sampling, measurement validity, analysis or risk of bias. One criterion repeats an inclusion rule. The step therefore removed poorly reported papers rather than methodologically weak ones, and it is notable that studies with no stated method still passed. Readers should not treat "after quality assessment" as meaning the 57 studies are of sound quality.
Search and selection
Searching publisher platforms rather than bibliographic databases such as Scopus or Web of Science, and then adding about 30 articles from Google Scholar, makes the search hard to reproduce. The paper does not report whether two reviewers screened and scored studies independently, and it does not follow the PRISMA 2020 reporting checklist (Page et al., 2021), which would have documented reasons for exclusion at each stage. These are common gaps: a tertiary review of AI in higher education found that 65% of the evidence syntheses it examined were of critically low to medium quality, with frequent omissions in database searching, quality assessment and inter-rater reliability (Bond et al., 2024).
Counting constructs is not synthesis
The finding that social influence, perceived usefulness, effort expectancy and facilitating conditions are the leading adoption factors is based on how many studies reported each construct as significant. Such vote counting ignores sample size and effect size, and it is shaped by which constructs the original studies chose to test. Because most studies applied TAM (Davis, 1989), UTAUT (Venkatesh et al., 2003) or UTAUT2 (Venkatesh et al., 2012), the constructs of those models were bound to dominate. Most data were also self-reported intentions from students, and intention does not reliably translate into use. Bagozzi (2007) argued that TAM's parsimony leaves out much of what drives technology use, a caution that applies with force to a tool whose use is shaped by assessment rules, integrity policies and discipline norms.
A fast-moving object
The window from January 2023 to mid-January 2024 covers the first year of public access to ChatGPT, when models, institutional policies and student habits were changing month by month. Excluding conceptual papers and editorials, defensible for an empirical review, also omitted much of the early debate on assessment and integrity. The findings are best read as a dated snapshot rather than a stable description.
How the review sits in the wider literature
Earlier work on AI in higher education already warned that research was dominated by computer science and quantitative methods, with little critical reflection on risks and weak links to pedagogical theory (Zawacki-Richter et al., 2019). Baig and Yadegaridehkordi's findings suggest the first wave of ChatGPT studies repeated that pattern. A contemporaneous scoping review of 69 studies reached broadly similar conclusions about uses by teachers, students and researchers, but found that only 27% of its studies were based on empirical data and that most came from high-income countries (Ansari et al., 2024). Read together, the two reviews show how much early claims rested on commentary and small surveys.
On integrity, the review records plagiarism and privacy as concerns but does not analyse them in depth. Cotton et al. (2024), one of the most cited papers in its corpus, set out practical institutional responses such as policies, training and assessment redesign; a doctoral reader looking for guidance on integrity should go to such sources directly. For research degrees specifically, interview evidence from Australian postgraduate research students suggests that ChatGPT is already changing the division of labour in supervision (Dai et al., 2023), an area the review touches only briefly under supervision tasks. Related Academy reviews look at supervising PhD students with AI and at a hands-on doctoral course on responsible AI.
What it means for doctoral researchers
If you cite this review, cite it for what it can support: a description of what was studied in 2023, by whom and with which methods, and a list of open problems. Avoid citing it as evidence that particular factors drive adoption, or that ChatGPT improves learning, because it did not test either.
If you plan your own review, the paper shows what to add. The table contrasts what Baig and Yadegaridehkordi reported with what a stronger doctoral review would report.
| Review element | In Baig and Yadegaridehkordi (2024) | What to report in your own review |
|---|---|---|
| Review type | Systematic literature review following Kitchenham and Charters | Name the review type and justify it; a mapping or scoping design may fit an emerging field better (Gough et al., 2012) |
| Search | Publisher platforms plus manual Google Scholar search | Full search strings for named bibliographic databases, search dates, and how supplementary searching was done |
| Screening | Counts reported; reviewer procedure not described | A PRISMA 2020 flow diagram with reasons for exclusion, and independent double screening of at least a sample |
| Quality appraisal | Four relevance and reporting criteria | A recognised appraisal tool suited to the designs included, applied by two reviewers |
| Synthesis | Frequency counts of significant constructs | Effect sizes or a structured narrative synthesis that weighs study quality and sample size |
| Currency | Search to January 2024 | Date the evidence explicitly and update the search before submission |
Worked example (illustrative)
A hypothetical doctoral student studying how engineering lecturers use generative AI drafts this sentence: "Social influence is the strongest predictor of ChatGPT adoption in universities (Baig and Yadegaridehkordi, 2024)." The claim overstates the source. A more accurate version reads: "In an early systematic review of 57 studies, mostly student surveys, social influence was the construct most often reported as significant (Baig and Yadegaridehkordi, 2024); evidence on academic staff was scarce, which this study addresses." The revision states the evidence base, keeps the claim within what the review measured, and turns the gap into a justification for the student's own project.
Common mistakes when using systematic reviews of generative AI
- Citing a review's frequency counts as if they were pooled effect sizes.
- Assuming that a "quality assessment" step guarantees methodological quality without reading the criteria.
- Treating findings about one model version and one year as a stable description of generative AI in education.
- Generalising student survey findings to academic staff, supervisors or non-academic staff.
- Copying a review's research gaps into your proposal without checking whether later studies have filled them.
- Relying on one review when other syntheses of the same period reached different or more cautious conclusions.
Key takeaways
- Baig and Yadegaridehkordi (2024) mapped 57 studies of ChatGPT in higher education from 2023 to early 2024 by trend, adoption measure, application and limitation.
- Its quality criteria assessed relevance and reporting, so the included studies were not appraised for risk of bias.
- The leading adoption factors it reports are frequency counts within TAM and UTAUT studies, not estimates of effect (Davis, 1989; Venkatesh et al., 2003).
- Wider syntheses confirm a field dominated by surveys, high-income settings and weak review methods (Bond et al., 2024; Ansari et al., 2024).
- Doctoral researchers should cite it as a dated map of early research and use PRISMA 2020 to report their own reviews (Page et al., 2021).
