Research gapComputer science

Anti-phishing training: why a heavily studied defence still does not reliably protect organisations

Phishing remains the most common attack reported by UK businesses (Department for Science, Innovation and Technology, 2026), and most organisations answer it with awareness training. Laboratory studies usually report benefits (Prümmer et al., 2024), but large field studies in real organisations find little or no effect (Ho et al., 2025; Lain et al., 2022).

PhD Research Lab InsightsEditorial team, reviewed by a subject specialist
Posted
5 October 2026
Last checked
5 October 2026
Reading time
9 min
Sources
8
Research proposal support
The gap

We do not know which anti-phishing interventions, if any, measurably reduce compromise in real organisations over time. Most studies report positive effects under controlled conditions (Prümmer et al., 2024), while the largest field studies find that annual and embedded training make little difference, or can even increase susceptibility (Ho et al., 2025; Lain et al., 2022), and the mechanisms behind these results are only beginning to be tested (Lain et al., 2024).

Evidence gap (laboratory versus field)Methodological gapPractical-knowledge gapTheoretical gap

What we know

  • A systematic review of 142 papers found that most report positive effects of cybersecurity training, but most used non-experimental pre-test and post-test designs with a median of 96 participants, often not employees, and little long-term follow-up (Prümmer et al., 2024).
  • An earlier comparative review found measurable improvements from anti-phishing education, varying with programme design, and contradictory evidence on which demographic groups are most at risk (Jampen et al., 2020).
  • In a 15-month study of more than 14,000 employees, embedded training during simulated phishing had unexpected side effects that could make employees more susceptible; crowd-sourced reporting by staff worked well (Lain et al., 2022).
  • In an eight-month study of 19,500 employees at UC San Diego Health, completing annual training was not significantly related to avoiding phishing, and embedded training reduced clicking by only about 2% (UC San Diego, 2025; Ho et al., 2025).
  • In a study of 4,554 employees, most did not read the training material; simple deterrent warnings worked about as well as full embedded training, and rewards for reporting showed no observable benefit (Lain et al., 2024).

What we do not know yet

  • Why controlled studies and organisational field studies reach such different conclusions.
  • Whether any training design produces lasting change in behaviour, rather than short-lived reminders.
  • Which outcome should count as success: fewer clicks, more reports, faster detection or fewer actual compromises.
  • How much of the risk can be shifted from users to technical controls and organisational processes.
  • How results transfer to small organisations, the public sector and low- and middle-income countries.

Research questions a PhD could ask

  1. Under which conditions (lure type, timing, role, workload) does embedded phishing training change behaviour in real organisations?
  2. Do reminder-style nudges and deterrent warnings sustain lower click rates beyond a few weeks?
  3. How do click rate, reporting rate and time to detection relate to actual account compromise in an organisation?
  4. What explains the gap between positive laboratory results and null field results: sample, measures, or the simulation itself?
  5. Can crowd-sourced reporting combined with technical filtering outperform training as the main defence?
Methods that could close the gap
ApproachWhy it fits
Randomised field experimentAssigns training conditions within a real organisation, following the designs of Lain et al. (2022) and Ho et al. (2025), with ethical safeguards for staff.
Systematic review with meta-analysisSeparates laboratory and field studies and codes design quality, extending Prümmer et al. (2024) and Jampen et al. (2020).
Longitudinal log analysisUses email security and incident logs to link simulated-phishing behaviour with real reporting and compromise over time.
Mixed methods with employeesCombines interviews with behavioural data to explain non-engagement with training material, building on Lain et al. (2024).
Socio-technical design studyBuilds and evaluates reporting tools or warnings that reduce reliance on user vigilance, informed by the success of crowd-sourced reporting (Lain et al., 2022).
Gap typeHeavily researched, still unsolved

The problem

Phishing uses deceptive messages to trick people into clicking links, opening attachments or giving away credentials. It is the most prevalent type of breach or attack reported by UK businesses (Department for Science, Innovation and Technology, 2026) and the most frequently reported crime type to the FBI's Internet Crime Complaint Center (Federal Bureau of Investigation, 2025).

Why it happens

Attackers target human attention rather than technical flaws. Field evidence suggests that people who click are often distracted rather than uninformed, and that they rarely read training material (Lain et al., 2024; UC San Diego, 2025). Lure design matters too: click rates in one study ranged from under 2% to over 30% depending on the message (UC San Diego, 2025).

Where it is felt most

Every sector that uses email is exposed. Healthcare is a notable case, as one of the largest field studies was run in a university health system (Ho et al., 2025). Business email compromise, which often begins with phishing-style deception, causes the second-highest reported losses in the United States (Federal Bureau of Investigation, 2025).

How big is the problem

What studies say, and at what level of evidence
LevelWhat it shows
Systematic reviewsMost of 142 studies report positive training effects, but designs are weak, samples small and follow-up short (Prümmer et al., 2024); an earlier review notes contradictory findings on who is most at risk (Jampen et al., 2020).
Large field studiesEmbedded training showed side effects that could raise susceptibility among more than 14,000 employees (Lain et al., 2022); annual and embedded training showed little benefit among 19,500 employees (Ho et al., 2025).
Field experiments on mechanismsTraining seems to act as a reminder rather than a lesson; warnings alone worked about as well, and reporting rewards did not help (Lain et al., 2024).
Official statisticsPhishing is the most common attack type in UK business surveys (Department for Science, Innovation and Technology, 2026) and the most reported crime type to the FBI IC3 (Federal Bureau of Investigation, 2025).
What is missingField trials that link training to real compromise, follow staff for longer than a year, and include smaller organisations and settings outside Europe and North America.

Who should take this on

Scholar profile

  • A security professional with access to an organisation's phishing simulation and incident data.
  • A computer scientist interested in usable security and large-scale measurement.
  • A behavioural or cognitive scientist studying attention, habit and decision-making under time pressure.
  • An information systems researcher interested in security culture and organisational policy.

Subjects and PhD specialisations

  • Computer science (usable security)
  • Cybersecurity
  • Information systems
  • Human-computer interaction
  • Cognitive and organisational psychology

Countries where the problem is greatest

Scope for future research

  1. Field trials that measure actual compromise and detection time, not only simulated click rates.
  2. Long-term studies of nudges, warnings and reporting tools as alternatives to classroom-style training.
  3. Standard outcome measures and reporting guidelines for phishing-training research.
  4. Studies of small organisations, public services and low- and middle-income countries.
  5. Interventions aimed at the organisation (workload, processes, verification rules) rather than the individual.

How a doctoral project could address it

  1. Define

    Choose one outcome that matters (reporting, compromise, detection time) and one population, and type the gap explicitly.

  2. Review

    Map existing studies by setting (laboratory or field), design and outcome, starting from the systematic reviews (Prümmer et al., 2024; Jampen et al., 2020).

  3. Partner

    Agree access with an organisation's security team, including what data can be used and how staff will be informed.

  4. Design

    Plan a randomised or quasi-experimental field study with a long enough follow-up, learning from the designs of Lain et al. (2022) and Ho et al. (2025).

  5. Ethics

    Simulated phishing involves deception: secure ethics approval, debriefing and a no-blame policy for staff.

  6. Outputs

    Aim for a review, a field-study paper and a design or mechanism paper, which together can form the thesis.

Why it is worth solving

Organisations spend time and money on phishing training, often because policies or insurers expect it. If commonly deployed programmes add little protection (Ho et al., 2025), that effort could be redirected to measures that work, such as easy reporting and technical controls (Lain et al., 2022).

For science, the question tests how far findings from controlled studies transfer to real behaviour at work. Answering it would improve not only phishing defence but the evaluation of security interventions more generally (Prümmer et al., 2024).

Why so much research has not solved it

Phishing training is one of the most studied topics in usable security. The trouble is where and how most of that research is done. A systematic review of 142 papers found that most report benefits, yet most also relied on pre-test and post-test designs with small samples, often of students rather than employees, and few measured whether effects lasted (Prümmer et al., 2024). Under those conditions, people know they are being tested and attend closely.

Field studies remove that advantage. In a real organisation, a phishing message arrives during a busy day. When researchers tracked more than 14,000 employees for 15 months, embedded training did not deliver the expected protection and showed side effects that could raise susceptibility (Lain et al., 2022). In a health system with 19,500 employees, annual training was not linked to fewer failures, and many employees closed training pages without reading them (UC San Diego, 2025; Ho et al., 2025).

  • Outcome mismatch: studies measure clicks on simulations, while organisations care about real compromise.
  • Attention, not knowledge: employees often already know the warning signs (Lain et al., 2024).
  • Lure variation: results depend heavily on which message is sent (UC San Diego, 2025).
  • Access: field data sit inside organisations and require trust, legal agreements and ethics approval.

Where a doctoral project can add value

A doctorate can contribute without promising to end phishing. One route is explanatory: testing why laboratory and field results diverge, for example by running the same intervention in both settings. A second is design-led: evaluating alternatives to training, such as deterrent warnings or reporting tools, which field evidence suggests may matter more (Lain et al., 2022; Lain et al., 2024). A third is measurement: linking simulation results to real incidents so that future studies use outcomes that matter.

Realistic doctoral angles on phishing defence
AngleTypical dataMain skill
Laboratory versus field comparisonSimulation results in two settingsExperimental design
Alternatives to trainingReporting and warning logsField experiments, statistics
Outcome validationSimulation, incident and detection recordsLog analysis, data linkage
Employee experienceInterviews and surveysQualitative or mixed methods

Whatever the angle, treat staff fairly. Studies that blame individuals can damage security culture, while those that improve systems benefit everyone.

What to do now

  • Read the two systematic reviews and the three field studies, and list how each defines success (Prümmer et al., 2024; Jampen et al., 2020; Lain et al., 2022; Ho et al., 2025; Lain et al., 2024).
  • Decide whether your contribution is explanatory, design-led or methodological.
  • Approach a security team early about data access, ethics and a no-blame policy.
  • Write a one-paragraph gap statement and test it with your supervisor before drafting research questions.
Your next step

Could this gap become your thesis?

  1. 1Read the evidence and the open questions
  2. 2Talk to a mentor about fit, data and method
  3. 3Build a defensible proposal
See proposal support

Questions scholars ask

If field studies show training does not work, is the question closed?

No. They show that common forms of training add little (Ho et al., 2025); they do not show what would work instead, or why laboratory results differ.

Can I study this without access to a company?

Partly. A systematic review or a controlled experiment is possible without one, but the strongest contribution comes from field data, so look for a partner early.

Is sending simulated phishing to staff ethical?

It can be, with ethics approval, organisational consent, debriefing and safeguards so that no one is punished for their results.

Which discipline should I register in?

Computer science, information systems and psychology all host this work; interdisciplinary supervision suits it well.

Sources

  1. Department for Science, Innovation and Technology, & Home Office. (2026, April 30). Cyber security breaches survey 2025/2026 [Official statistics]. GOV.UK. https://www.gov.uk/government/statistics/cyber-security-breaches-survey-20252026/cyber-security-breaches-survey-20252026
  2. Federal Bureau of Investigation, Internet Crime Complaint Center. (2025). Internet crime report 2024. https://www.ic3.gov/AnnualReport/Reports/2024_IC3Report.pdf
  3. Ho, G., Mirian, A., Luo, E., Tong, K., Lee, E., Liu, L., Longhurst, C. A., Dameff, C., Savage, S., & Voelker, G. M. (2025). Understanding the efficacy of phishing training in practice. In Proceedings of the 2025 IEEE Symposium on Security and Privacy (pp. 37–54). IEEE. https://doi.org/10.1109/SP61157.2025.00076
  4. Jampen, D., Gür, G., Sutter, T., & Tellenbach, B. (2020). Don't click: Towards an effective anti-phishing training. A comparative literature review. Human-centric Computing and Information Sciences, 10, 33. https://doi.org/10.1186/s13673-020-00237-7
  5. Lain, D., Jost, T., Matetic, S., Kostiainen, K., & Capkun, S. (2024). Content, nudges and incentives: A study on the effectiveness and perception of embedded phishing training. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS '24). ACM. https://doi.org/10.1145/3658644.3690348
  6. Lain, D., Kostiainen, K., & Capkun, S. (2022). Phishing in organizations: Findings from a large-scale and long-term study. In 2022 IEEE Symposium on Security and Privacy (SP) (pp. 842–859). IEEE. https://doi.org/10.3929/ethz-b-000588856
  7. Prümmer, J., van Steen, T., & van den Berg, B. (2024). A systematic review of current cybersecurity training methods. Computers & Security, 136, 103585. https://doi.org/10.1016/j.cose.2023.103585
  8. University of California San Diego. (2025). Cybersecurity training programs don't prevent employees from falling for phishing scams [News release]. UC San Diego Today. https://today.ucsd.edu/story/cybersecurity-training-programs-dont-prevent-employees-from-falling-for-phishing-scams

How to cite this page

Choose your referencing style, then copy.

PhD Research Lab Insights. (2026, October 5). Anti-phishing training: why a heavily studied defence still does not reliably protect organisations. PhD Research Lab. https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness/

PhD Research Lab Insights. “Anti-phishing Training: Why a Heavily Studied Defence Still Does Not Reliably Protect Organisations.” PhD Research Lab, 5 Oct. 2026, www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness. Accessed [access date].

PhD Research Lab Insights (2026) Anti-phishing training: why a heavily studied defence still does not reliably protect organisations. PhD Research Lab. Available at: https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness/ (Accessed: [access date]).

PhD Research Lab Insights. “Anti-phishing Training: Why a Heavily Studied Defence Still Does Not Reliably Protect Organisations.” PhD Research Lab. October 5, 2026. https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness.

PhD Research Lab Insights, ‘Anti-phishing Training: Why a Heavily Studied Defence Still Does Not Reliably Protect Organisations’ (PhD Research Lab, 5 October 2026) <https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness> accessed [access date].

PhD Research Lab Insights. Anti-phishing training: why a heavily studied defence still does not reliably protect organisations [Internet]. PhD Research Lab; 2026 Oct 5 [cited [access date]]. Available from: https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness/

PhD Research Lab Insights, “Anti-phishing training: why a heavily studied defence still does not reliably protect organisations,” PhD Research Lab, Oct. 5, 2026. [Online]. Available: https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness. [Accessed: [access date]].

@online{phishing_training_effectiveness_2026,
  author = {{PhD Research Lab Insights}},
  title = {{Anti-phishing training: why a heavily studied defence still does not reliably protect organisations}},
  organization = {PhD Research Lab},
  date = {2026-10-05},
  url = {https://www.phdassistance.com/insights/r-and-d/research-gap/phishing-training-effectiveness},
  urldate = {[access date]}
}

  • Phishing
  • Security awareness training
  • Usable security
  • Field experiments
  • Human factors
Anti-phishing training: why a heavily studied defence still does not reliably protect orgaResearch gap · Computer science
Proposal support