Kantyra
Kantyra
Alain Rees · 10-07-2026 · 7 min read
Almost every organisation does it: an annual awareness training, a quarterly phishing simulation and a poster next to the coffee machine. The question rarely asked alongside it is whether behaviour actually changes. In recent years that question has finally been studied at scale, with tens of thousands of employees in real organisations, and the answer is sobering. For the standard approach, the mandatory annual training with a lesson after every wrong click, the research finds hardly any effect. For a few other things, it does. Strikingly, much of this research comes from Dutch universities.
This article belongs to the Resolve phase of the model behind Kantyra: behaviour is a control, and like every control it must not only exist but also work.
The fundamental problem has a name: the gap between knowledge and behaviour. Maria Bada, Angela Sasse and Jason Nurse analysed why awareness campaigns fail and arrived at a simple diagnosis: knowing what is safe does not automatically translate into acting safely, and campaigns built mainly on fear tend to backfire. The Australian validation research around the HAIS-Q, a widely used questionnaire that measures knowledge, attitude and behaviour separately, confirms that picture: knowledge of policy correlates more strongly with attitude than with behaviour. Whoever only tests knowledge after the training is measuring the wrong thing.
Tommy van Steen of Leiden University and colleagues put their finger on the cause. They analysed seventeen government-led awareness campaigns using the toolkit of behavioural science and established that these campaigns lean almost entirely on providing information, while barely using the proven techniques for behaviour change. Most awareness, in short, is not designed to change behaviour. It is then no surprise that it does not.
Two recent studies, both carried out in real organisations over many months, have sharpened the debate. Researchers at ETH Zurich followed more than fourteen thousand employees of a large company for fifteen months and tested the approach almost every vendor sells: whoever clicks on a simulated phishing mail gets a lesson page on the spot. The result: those lessons did not make the clickers better, and could even make them more susceptible. A follow-up study by the same team unravelled why. What little effect such simulations have comes from the periodic reminder that phishing exists, not from the lesson content, which employees barely read for lack of time. Rewards for good behaviour did not help either.
The second study is, if anything, even more direct. Grant Ho and colleagues ran an eight-month randomised experiment among more than nineteen thousand employees of a large American healthcare organisation and found no significant relationship between having recently completed the annual awareness training and the likelihood of falling for a phishing simulation. Their conclusion leaves little room: standard anti-phishing training probably offers little protective value in practice.
There was one outspokenly positive finding in the Zurich research, and it is instructive. A report button with which employees could forward suspicious mails did work: the collective of employees turned out to be a fast and scalable detection mechanism for new phishing campaigns, with a manageable workload for the security team. Not preventing every click, but seeing the attack early turned out to be the gain. That connects seamlessly to what I wrote earlier about reporting behaviour.
The Dutch research gives the answer more relief. Jan-Willem Bullee and Marianne Junger of the University of Twente conducted a meta-analysis of nineteen studies with more than twenty-three thousand participants and found large differences between interventions: some work well, others do nothing at all. The pattern: intensive interventions beat light ones, and a narrow focus beats a broad story. Earlier field research from Twente had already shown that fleeting means fail: warnings and pointing out dangers just before the moment of risk did not reduce the disclosure of personal data, and could even increase it. And a Twente experiment with anti-phishing lessons at primary schools demonstrated the shelf-life problem: recognition rose clearly right after the training, but after four weeks the effect had already disappeared. Training once a year is thereby closer to a ritual than to a control.
What does work is active and repeated. Van Steen and Deeleman showed that a serious game improved not only attitudes but also (self-reported) behaviour, where information leaflets did not. American research added a surprising factor: who delivers the message matters. Factual advice landed best coming from a security expert, while stories about incidents worked most strongly coming from colleagues, people like yourself. And the common thread through everything: repetition. Short, targeted prompts at some regularity beat the annual one-hour sit.
The conclusion from twenty years of research is not that you should stop doing awareness, but that you should stop doing awareness as a standalone obligation. Van Steen recently published a usable five-step framework for this: start from the behaviour you want to change, not from the content you happen to have, choose interventions with a proven working mechanism, and measure behaviour instead of attendance. That is also how I look at it at Kantyra. In the register, awareness is a control like any other, with an owner, an effectiveness test and a test date. The question in that test is not whether everyone completed the e-learning, but whether behaviour shifts: does the number of successful phishing attempts fall, does the number of reports rise, and how quickly does the first report of a new campaign come in? Whoever works with a training partner, as we do with 2LRN4, can simply put those questions to the partner; a good partner wants to be judged on them.
If you take one thing from this article, let it be this: judge awareness by behaviour, not by attendance. Ask yourself three questions. After the training, do you measure anything other than attendance and a knowledge quiz? Is the number of reported suspicious mails rising, and how quickly is a new phishing campaign flagged by your own people? And if an awareness activity has shown no measurable behavioural effect for two years, what is the reason to renew it? The same euros can go to short, targeted and repeated interventions, to a report button with fast feedback, and to technical controls that make the click less damaging. The research is unanimous on this: better a small effect that is really there than a large programme that mainly reassures.
This article is a substantiated synthesis of existing scientific research and not independent, peer-reviewed research of its own. Where I interpret or explain, that interpretation is my own responsibility.
In a 30-minute demo we walk through the model based on your situation.
Book a demoThe exercise succeeds, the evaluation is positive, and next year the same findings appear in the report. Leiden research shows why lessons do not stick, and what makes an exercise worthwhile: repetition with reflection beats scale.
Between the treatment plan and demonstrable execution, many organisations harbour a gap. How to bridge it with controls that get tested, policies with a lifecycle, projects for what is missing and formal exceptions for what you accept.