The Importance of Teaching Meaningful Cybersecurity Metrics for Management Reporting
In business, I've often come across the phrase, “If you can’t measure it, you can’t improve it.”
This concept holds true when it comes to metrics reporting, and, more specifically, cybersecurity metrics reporting. In cybersecurity, in order to provide the best value for management to make key decisions, it is critical to provide meaningful indicators that align with an organization's strategy and risk appetite.
There isn’t a one-size-fits-all approach, however. In this article, I will provide guidance that educators can begin to share with cybersecurity students to prepare them using real-world context. The metrics and examples I provide are by no means exhaustive, but they are meant to bring awareness to the importance of reporting key performance indicators, and key risk indicators to provide a clear picture of the state of cybersecurity within an organization to those tasked with making key strategy decisions.
The Basics: KPIs and KRIs
I’ll start with the aforementioned terms key performance indicators (KPIs), and key risk indicators (KRIs) to provide a primer on how these metrics are used. I’ll also add that, in some instances, key performance indicators can actually be used as key risk indicators (more on that shortly).
Over the years, I have determined that using analogies is a good way to level set a topic. Analogies allow everyone in the room to have the same understanding.
In this case, I’d like to use the analogy of smoke detectors to KPIs, and KRIs (this analogy assumes that the smoke detector is not hard-wired to the electrical system). If you’ve ever heard the constant beeping noise from a smoke detector (not necessarily a loud alarm, but a cadence of beeps), this is an example of a KPI notifying you that the batteries in your smoke detector are weak. Specifically, the performance of the batteries is beginning to deteriorate, and the beeps are informing you to take action and change them.
Additionally, the smoke alarm itself (with fresh batteries of course) has the sole purpose of detecting smoke. This detection of smoke is a key risk indicator to seek safety immediately. It’s essentially a heads up to take action before things get progressively worse.
I should also add that the KPI of weak batteries can also be a KRI that you will not be made aware that smoke or fire is nearby.
Cybersecurity Metrics: MTTD and MTTR
It's no secret that cybersecurity is at the top of everyone’s mind; this includes senior leaders within all organizations. It’s safe to say that traditional concerns such as talent retention, the economy, competition, and strategic direction are still top of mind for the C-Suite, however cybersecurity risk is becoming more of a prominent topic.
With looming regulations focused on data breach reporting on the horizon, insomnia may be increasing for many executives. What’s more alarming for executives would be the average rate of time it takes an organization to detect and contain a data breach, which is approximately 287 days. Additionally, it is noted that data breach vulnerabilities can sit undetected for about seven months, with another 75 days to contain the vulnerabilities. This information speaks directly to the first set of cybersecurity metrics that I would like to share: Mean Time to Detect (MTTD), and Mean Time to Recovery (MTTR).
MTTD and MTTR are KPIs that are commonly used in cybersecurity. MTTD focuses on the average (or mean) time that it takes to detect a cyber incident from the time it occurs. This metric provides insight into the proficiency of an organization to discover or detect threats. Organizations with shorter MTTD metrics will generally have minimal disruptions.
MTTR is based on how long it takes for an organization to bounce back from an incident. This metric is sometimes referred to as mean time to repair and it speaks directly to the skillset, capacity, and tools available for cyber and IT professionals to bring systems back to normal operations. Referring back to the fact that KPIs can be leveraged as KRIs, if it takes a considerable amount of time to recover from an incident, that’s a direct correlation to the resiliency of an organization to resume normal operations.
Other Key Cybersecurity Metrics
Another important metric for management to be aware of can be summarized as training effectiveness. This metric can provide details and insight into KRIs, and KPIs focused on completion rates of security training and the number of users that clicked on an “infected” email link during phishing campaigns.
I’ve often said people are the weakest link when it comes to the cybersecurity ecosystem of confidentiality, integrity, and availability. You can have the most secure network perimeter, and access controls however it takes just one person to click on an email with malware to cause havoc in an organization.
Additionally, if you consider the attack surface based on the average number of devices per person— about 3.6 per person globally in 2023—the probability is pretty high that an attack aimed at individuals can manifest from a vulnerability to an incident very quickly. This is the precise reason that tracking training effectiveness can provide insights to management of the performance of security training (KPI) and the potential that an attack can occur based on the number of users that fall susceptible to internally managed phishing campaigns (KRI).
I’d like to conclude by reiterating that these KPIs and KRIs are just examples and are absolutely NOT meant to be an exhaustive list. More importantly, every organization is different, so certain metrics may not be of interest based on the risk appetite, strategy, and culture.
It’s also important to educate your students to not focus on every aspect of cybersecurity metrics when presenting to management as they may not be interested in all of the technical details. Instead, encourage students to understand the business and products for their future organizations and prepare them to ALWAYS be ready to answer the following questions:
- Are We Secure?
- Are We Compliant?
- Have There Been Incidents?
- Is Our Security Program Effective?
- Is Our Security Program Efficient?
In other words, ask the students what cyber-related issues would keep them up at night if they had to run a business.
To learn more about teaching meaningful cybersecurity metrics, consider requesting a review copy of Fundamentals of Information Systems Security, Fourth Edition. This product includes content on business drivers, including business impact analysis and assessing risk as well as information and discussion points on emerging technologies and the risks, threats, and vulnerabilities associated with our digital world.
Request Your Digital Review Copy
About the Author:
Rodney F. Davis is an adjunct professor at Syracuse University’s College of Professional Studies where he teaches courses focused on Enterprise Risk Management, Cybersecurity, Networking, Forensic Accounting (Fraud Prevention), and Vendor Risk Management. Rod has a total of 29 years professional experience, 27 of which are focused on operational risk, regulatory oversight, technology, and cyber security within the financial services industry. Rod is also a member of an international team of cyber risk professionals responsible for creating and approving certification exam items for ISACA (Information Systems Audit and Control Association).
Related Content:
- Top Cybersecurity Trends for Educators to Watch
- The Vital Link for Cybersecurity Instructors: Bridging the Gap Between Technical Skills and Business Acumen
- Instructors Provide Guidance on Teaching the Cybersecurity Workforce of the Future
Related Products: