Highlights
"We've been bit by this": the training existed, and the social engineering incident happened anyway
Tools bought for the auditor sit barely used while the people they cover stay exposed
One leader hand-builds his risk scores in a spreadsheet because his platform never joins the data
The harmless baseline
The consensus about annual security training is that it is table stakes. It satisfies the auditor, it costs little, and if it does not help much, at least it does no harm. Nobody defends it with enthusiasm, but nobody removes it either, and that quiet consensus is exactly what makes it dangerous.
This week I spoke with a head of security at a growing company who described his annual training, delivered through his HR platform, in six words: "it is nothing actually useful." Then he added the part that matters. "We're going to get bit by this. We've been bit by this." His team is still writing the postmortem on a social engineering incident that walked through an MFA prompt. The training was in place. Completion was logged. The incident happened anyway.
Here is the claim: training that satisfies the auditor and changes no behavior is worse than no training, because it converts a live risk into a closed ticket. The organization believes the human column is handled. It is not.
The checkbox illusion
The pattern is not limited to training. Another security leader this week walked me through his stack: a SIEM purchased primarily to check a compliance box and barely used, a phishing platform inherited from a predecessor that nobody loves and nobody has time to replace. His security engineer is leaving. His cloud security renewal is doubling. Every hour is spoken for.
Tools acquired this way share a lineage. They were bought for the auditor, they are tolerated by the employee, and they are invisible to the attacker. The attacker does not read your compliance report. The attacker reads your people, and a person who clicked through forty slides in January is the same person in September, except now the org believes they are trained.
What the incident teaches
The head of security was unusually honest about where he stands. "I'm still kind of old world security right now," he said, describing the bandwidth he does not have to adopt AI-driven defenses, and in the same breath: "we're at the precipice of a pretty scary time." That pairing is the whole problem in one conversation. The attacker column is moving at the speed of the model. The human column has not moved in a decade, and the artifact that says it has is a completion certificate.
A third conversation this week made the alternative concrete. A security lead at a financial services organization runs monthly phishing campaigns and, because his platform tracks opens, clicks, and reports but never maps them to each other, exports the data and hand-calculates a composite risk score per employee in a spreadsheet. What he actually wants is to watch an individual's risk "ebbing and flowing over time." Not a snapshot. A trajectory. And a bank buyer in a separate call asked for the same thing from the other direction: a one-minute remediation tied to the specific mistake a specific person just made, not a generic module.
The narrow claim
If the most measurable output of your training program is the completion certificate, you have an audit artifact, not a control. The only metric that counts is the change in a person's behavior between one failure and the next. Measure that, and the annual module either earns its place or reveals itself.
Key Takeways
Pull the completion rate off your board slide and replace it with repeat-failure rate per person.
Tie every remediation to the specific failure that triggered it, within days, not at the next annual cycle.
Audit your stack for tools bought to satisfy a framework and ask who, if anyone, acts on their output.
Track individual risk as a time series so you can see whether anything you do actually bends it.
