Critical Review of ChatGPT in PhD Mentoring
Critical Review of ChatGPT in PhD Mentoring
Exploring the Potential of Generative AI for Academic Guidance and Sustainable Educational Practices
Introduction
The paper authored by Iatrellis, Bania, Samaras, Kosmopoulou, and Panagiotakopoulos (2025) tells about the new role of generative AI, especially ChatGPT, in PhD academic mentoring. It is placed within the current debates about doctoral supervision, workload increase, and sustainable education, and through these controversies, the study aims to see if generative AI could really be a good support tool in the doctoral domain. They have done it so by looking at the ability of ChatGPT to give structured, context-aware academic support, and exploring different ways of asking questions and their impact on the quality of AI outputs.
The main goal of the research is twofold. It starts with examining if ChatGPT is able to provide academically appropriate and contextually relevant guidance when varying degrees of input specificity are given. The second part is about introducing a “tripartite mentoring model” in which doctoral students, supervisors, and AI systems cooperate while retaining their own roles and responsibilities. The article primarily targets higher education researchers, doctoral supervisors, and academic policymakers, and its formal academic style and structured organisation are indeed suitable for this expert audience. In summary, the article positions itself as an innovative and timely contribution to the debate regarding AI-assisted doctoral education, but at the same time, it lacks in some areas the necessary empirical support, thus not being very convincing in its claims.
Summary of the Article
The authors adopt a qualitative, exploratory research method, which is based on a single and detailed case study of a PhD project in disaster risk management. They select this field due to its multidisciplinary nature and conceptual challenge, which they regard as a testing ground for AI-assisted academic advising of high quality. A systematic evaluation framework is established to dissect ChatGPT’s response to various extents and types of contextual information in the prompts.
The authors adopt a qualitative, exploratory research method, which is based on a single and detailed case study of a PhD project in disaster risk management. They select this field due to its multidisciplinary nature and conceptual challenge, which they regard as a testing ground for AI-assisted academic advising of high quality. A systematic evaluation framework is established to dissect ChatGPT’s response to various extents and types of contextual information in the prompts.
The ChatGPT outputs are rated by five academic professionals by applying qualitative criteria like relevance, coherence, depth, and usefulness. The inter-rater reliability is measured by using Kendall’s coefficient of concordance (W), which is the method the authors use to present the agreement among expert judgments. The results show that the prompts having more complex conceptual structure and more contextual specificity lead to better AI outputs, while the prompts based on keywords or with minimal context produce fragmented and less coherent guidance.
In light of these findings, the authors claim that ChatGPT could be a great help in the process of brainstorming ideas, synthesising concepts, and mapping research pathways in the early stages of doctoral research. The research finally leads to the introduction of a tripartite mentoring model wherein AI plays the role of an auxiliary support tool under human guidance.
Critique
Significance and Contribution to the Field
A significant benefit of the article is that it addresses a very relevant and neglected issue in research about higher education. In several countries, the supervision of PhD students is getting more and more complicated due to the rising number of students, the shortage of supervisors, and the need for educational institutions to be effective and profitable. By situating generative AI in such a scenario, the authors are indeed bridging a significant gap in the literature.
This paper moves one step further in the discussion of past research that has considered generative AI’s involvement in writing, assessment and general student support and zeroes in on doctoral mentoring in particular. The proposed three-tier mentoring model is a major theoretical contribution since it alters our perception of AI; it is turned from being a replacement for supervision into being a partner in the broader mentoring ecosystem.
Nonetheless, the contribution is mostly of a theoretical nature. The model is depicted as an anticipatory framework instead of one supported by proven educational results. Consequently, even though the publication enhances the existing knowledge by introducing new research avenues, it does not go as far as offering strong proof that the suggested model can significantly impact doctoral education or supervision efficiency in real-life situations.
Methodology and Research Design
The qualitative case study method chosen by the authors is suitable for a first-time inquiry into a new phenomenon. By systematically changing the types of prompts, the researchers made a good methodological choice, which allowed the comparison of different levels of contextual input to be quite useful. Adding expert evaluators and calculating inter-rater reliability makes the internal consistency of the study even stronger.
On the other hand, the methodological limitations are huge. A single case study in one academic discipline is the basis of the entire study; thus, the generalizability of the findings is very limited. Unless other disciplines or institutions are involved, it will be difficult to know if the effects seen in this study are applicable in general or only in specific contexts.
Plus, the lack of a control condition—like a traditional supervision without AI participation—curbs the potential of the study to separate ChatGPT’s specific input. The qualitative expert judgement used exclusively is, although very informative but, on the other hand, is not supported by quantitative performance indicators or measurable learning outcomes. Consequently, assertions about effectiveness, efficiency, or sustainability are still only implied and not conclusively proven.
Argumentation and Use of Evidence
The article’s argumentation is very logical and consistent with its own premises. The authors assert that prompt design is the main reason for AI’s capacity to provide academic assistance, which is a conclusion that supports the current studies on prompt engineering and human-AI interaction. The types of prompts comparison is supported by expert evaluations, and thus, it is very clear.
The strength of the argument, however, sometimes goes beyond what evidence can support. The claims about the tripartite mentoring model being transformative or even “revolutionary” are not completely backed by the data. The research does not really show that using AI leads to better PhD outcomes or provides any longitudinal proof of a lasting academic benefit. So, while the results are convincing in their exploratory range, wider claims about reform in doctoral supervision are still to be verified.
Ethical Considerations and Omissions
The writers mention behind-the-scenes matters that are ethical and related to integrity, data management, and the possible undue dependence on AI systems, but these concerns are not discussed in relation to their importance to the topic matter.
The authorship issue, the question of who is responsible for the work and the concern that AI assistance might get so close to human guidance that it will no longer be possible to tell how much the person has developed independently and how much they have been helped by the machine, are all questions that the article does not address. The lack of any reference to the already existing ethical frameworks or the governance models has a negative effect on the practical applicability of the proposed mentoring model and on its readiness for real-world use.
Writing Style and Structure
The writer has demonstrated that they are an academic writer through their clear and logical writing and the appropriateness of their content for the academic audience. The movement from the theoretical framework through methods and results to the conceptual implications is transparent and straightforward. The analyst has made use of both tables and thematic interpretation, and they not only support the analysis but also facilitate the understanding of the readers.
But then, if the writer had countered arguments and alternative views more critically, the discussion would have been definitely more convincing. The paper seems to weigh more the benefits of AI-assisted mentoring than its drawbacks, thus creating a somewhat optimistic tone that might be a cause to ignore the structural and ethical challenges.
Conclusion
To sum up, Iatrellis et al. (2025) carry out a timely and stimulating discourse on the possible role that ChatGPT might have in PhD mentoring. In the middle of all this, the article gives a significant conceptual contribution by proving the need for prompt design and suggesting a three-way mentoring model that allows AI to be part of the doctoral supervision process, and at the same time, human expertise is not replaced.
Nonetheless, the exploratory nature of the study, the case study approach, and the lack of quantitative or longitudinal evidence are all aspects that weaken the conclusions drawn from the study. The ethical issues involved have been mentioned, but are not thoroughly discussed as they should be considering their importance. Nevertheless, these limitations do not take away the merit of the article that it is a stepping stone for future researchers and also provides fascinating perspectives for scholars and decision-makers who are interested in the sustainable, technology-enhanced PhD education.
The later research should go for mixed-methods, consider the views of doctoral students, and check the long-term academic outcomes in different disciplines and universities. This line of inquiry, supported by a firmer empirical base and more thorough ethical scrutiny, could ultimately have a great impact on the transformation of doctoral mentoring practices.
Reference:
Iatrellis, O., Bania, A., Samaras, N., Kosmopoulou, I., & Panagiotakopoulos, T. (2025). ChatGPT in PhD mentoring: Exploring the potential of generative AI for academic guidance and sustainable educational practices. Preprints. https://doi.org/10.20944/preprints202502.0041.v1
We offer our Greatness in Various Parts of Research, and we help you with any phase of your Process. Make a Smart Decision and get your Paper Published.