Kantyra
Kantyra
Alain Rees · 10-07-2026 · 10 min leestijd
Open any risk register and you will find it: the matrix of likelihood times impact, usually five by five, with green, yellow and red cells. The risk matrix is the most widely used instrument in risk management, from safety engineering to accountancy and from healthcare to information security. Kantyra, too, has one at the heart of its risk module. That is precisely why I have to be honest about something few vendors say out loud: the scientific literature is remarkably critical of this instrument. In this article I place that criticism alongside daily practice and show how, despite its limitations, you can work with a matrix responsibly.
This article builds on the scientific basis of the model, in which I substantiate the four phases of Detect, Assess, Resolve and Demonstrate. The risk matrix belongs to the Assess phase, and it deserves the same honest treatment.
The most fundamental criticism comes from Tony Cox, an American risk scientist who devoted two articles to it, one with colleagues in 2005 and one on his own in 2008, that have since become classics. His analysis in Risk Analysis, the leading journal of the field, is mathematical in nature and can be summarised in three findings.
First, a matrix has coarse discriminating power. Because likelihood and impact are pressed into a handful of categories, risks that differ strongly in numerical terms receive the same colour, and a matrix can only say correctly which of two randomly chosen risks is larger in a limited share of cases. Second, a matrix makes demonstrable ranking errors: there are situations in which a risk that is numerically smaller still lands in a higher cell than a larger risk. Third, and this is the harshest conclusion, for risks where likelihood and impact are negatively correlated, that is, where the rare events are precisely the severe ones, a matrix performs in the worst case no better than random guessing. Cox uses the now famous phrase that the matrix is then "worse than useless".
Researchers from the oil and gas sector later sharpened this. Thomas, Bratvold and Bickel showed in 2014 that the design choices of the matrix itself, such as the number of categories, the division of the scales and the scores attached to the cells, also change the final ranking of risks profoundly. Two organisations that assess exactly the same risks but use a different matrix layout arrive at a different priority list, without anyone having made a mistake. The ranking that comes out is thereby partly a consequence of the instrument rather than of reality.
The second line of criticism is not about the mathematics but about the human filling in the cells. David Ball and John Watt investigated in 2013 how different assessors place the same hazard on a matrix. The scatter turned out to be large, and did not disappear after reflection and explanation either. Moreover, the differences turned out not to stem from carelessness, but from diverging worldviews and convictions that rarely reach the table during a risk session.
On top of that comes a language problem. Terms such as "likely", "rare" and "considerable" mean something different to everyone. David Budescu and colleagues had test subjects translate probability expressions from the reports of the IPCC, the climate panel of the United Nations, into numbers. Even with the official definitions alongside, the interpretations diverged strongly. Saying "likely" may evoke thirty percent in one person and ninety in another. Every risk matrix that works with such words inherits this problem.
And beneath all of that lies the oldest insight of this field. Amos Tversky and Daniel Kahneman described as early as 1974 how people estimate probabilities using rules of thumb that produce systematic errors of thought: we overestimate what is recent or vivid, we anchor on the first number that comes along and we are structurally too sure of our own judgement. A risk session in which experts pick cells without further support is susceptible to all of those effects.
No, and that is not my judgement but that of the same literature. Nijs Jan Duijm, affiliated with the Technical University of Denmark, published a balanced review of the weaknesses in 2015, together with concrete recommendations for design and use. His core message: the matrix is usable as an instrument for prioritising and for structuring the conversation about risks, as long as you do not treat it as a measuring instrument and you define the categories carefully and anchor them in numbers.
This question has also been worked on at Delft University of Technology. Ben Ale, emeritus professor of safety and disaster abatement, showed with colleagues in 2015 that probability-consequence diagrams have a legitimate mathematical basis, but become misleading as soon as they are used without knowledge of the rules that come with them. Floris Goerlandt and Genserik Reniers, the latter also a professor in Delft, added an essential point in 2016. A dot in a matrix conveys an impression of certainty, while one judgement rests on years of incident data and another on a first estimate made in a working session. They therefore argue for assessing and making visible the strength of the underlying knowledge alongside the position in the matrix. That insight returns in the practical habits further on: the cell says little without the reasoning attached.
More recent experimental research from the Winton Centre in Cambridge points in the same direction. Holly Sutherland and colleagues tested in randomised experiments with almost two thousand seven hundred participants how people understand matrices. A matrix turned out not to be self-evidently better than plain text for conveying risk information, and the design mattered greatly: for scales that do not increase linearly, labelling that makes this explicit helped considerably. A follow-up study by Rossa Proto and colleagues from 2023 examined the colours themselves and found that coloured bands distort judgement: participants attached more value to risk reductions that crossed a colour boundary than to equally large reductions within the same colour. The border between yellow and orange then starts to weigh in as if it were a property of the risk, while it is a property of the picture.
The sum of it: the matrix is not a measuring instrument and must not be used as one, but as a structured form of conversation and prioritisation it has value, provided it is well designed and its users know its limitations.
The scientific critics almost all advocate the same alternative: calculate with numbers instead of with cells. Douglas Hubbard and Richard Seiersen worked this out furthest for information security. They show that even with little data you can express probabilities and losses in ranges, that you can train experts to make calibrated estimates, and that even a simple quantitative model removes most of the distortion of the matrix.
For an organisation with its own risk team and sufficient data, that is the better road. But here the SME research from my earlier article returns: for most organisations without a full-time security department, a fully quantitative approach is not feasible at this moment, and a poorly executed quantification creates more false certainty than a well-understood matrix. So it is not a choice between pure and sinful. The realistic route is a well-designed matrix for the breadth of the register, with quantitative deepening for the few risks on which the biggest decisions depend.
Five habits follow from the literature that make the difference between a matrix that misleads and a matrix that helps.
For completeness: this is also how Kantyra is set up. The scales of the matrix can be defined per organisation, so that you can anchor them in numbers. Every risk asks for a rationale alongside the score by default (each organisation can switch that requirement on or off), serious incidents that touch a risk automatically queue up a reassessment, and every change is preserved in the audit log with user and timestamp. The matrix itself thereby remains what the literature allows it to be: an instrument for prioritising and for conversation, embedded in a working method that absorbs its weaknesses.
If you take one thing from this article, let it be this: do not distrust the matrix, distrust a bare cell. A score without a numerical definition, without a second assessor, without a rationale and without reassessment when new signals arrive is exactly the symbolic risk management the literature warns against. The same matrix, embedded in those four habits, is a defensible and workable instrument, even in front of a critical auditor who has read Cox.
So ask yourself three questions. Are the categories of your matrix defined numerically anywhere, or does "likely" mean something different to everyone in your organisation? Can you retrieve the reasoning behind the judgement for your five biggest risks? And when was that judgement last tested against what has actually happened since? Whoever can answer these three questions uses the matrix the way the science permits. Whoever cannot does not have a matrix problem but an assessment problem, and that, fortunately, can be solved.
This article is a substantiated synthesis of existing scientific research and not independent, peer-reviewed research of its own. Where I interpret or explain, that interpretation is my own responsibility.
In een demo van 30 minuten lopen we het model door aan de hand van jouw situatie.
Plan een demoProcurement, IT and information security each hold their own conversation with the same supplier, with TPRM hovering above as a fourth term. On the differences, what research says about supplier assessment, and a six-step process that stands up to NIS2 and DORA.
The spreadsheet with the NIS2 framework suggests everything is sorted, until someone asks whether you can prove it. Why that spreadsheet has value, which questions it leaves unanswered, and how to move from filling in to demonstrating in three steps.