We recently had an opportunity to sit down with Doug Hubbard, cybersecurity risk expert and author of How to Measure Anything in Cybersecurity Risks, to talk about measuring cybersecurity.
Hubbard started at Cooper and Lybrand before it merged with Price Waterhouse to become PwC. During his 33 years of experience, he’s always been in some form of quantitative management consulting role.
“The Director of Managed Consulting in the group I was working in was a RAND Corporation guy,” Hubbard said. “He looked at everything as a quantitative problem, and I did, too. But I noticed that along the way, we would deal with clients that would state something was immeasurable.”
However, there were times Hubbard said when they had measured that exact thing with a previous client.
“I just started to doubt after a while if people really knew what they were talking about when they said that something was immeasurable. And eventually, I decided there were only a few reasons why anybody ever thought anything was immeasurable. And they’re all illusions,” he said.
Uncertainty Reduction Rather than Exact Measurements
His first book was about how to measure anything and how to find the value of intangibles. He discussed the difficulty of measurement problems in general. “Measurement number one doesn’t mean an exact number,” he said. “It means uncertainty reduction based on observation and quantitatively expressed.”
Figuring out how to measure cybersecurity risk fit neatly into this narrative. While you may not be able to quantify everything based on the data you have, you can make significant inferences. “There’s a lot of misconceptions about the kinds of inferences you can make from even just a few observations,” Hubbard said. “The math is pretty clear on this. And know we’re better off making better bets, even with just a little bit more information.”
“When people start to define things in terms of their observable consequences,” he said. “I say that they’re halfway to measuring it, and the rest is trivial math.”
Measuring Cybersecurity
“There used to be this old saying that you couldn’t prove the return on investment for cybersecurity, because you can’t make things fail,” Hubbard said. “You don’t know when you are going to get back and with so many unknowns, you can’t prove the unknown.”
However, Hubbard notes that you can evaluate risk. “We can actually show whether things are working or not, and then use that to inform your portfolio optimization to really understand what is working,” he said. This allows you to make better decisions about your cybersecurity portfolio.
“It’s an irrational move to say that because I can’t prove something for certain, I’m not immune to reducing my uncertainty,” said Hubbard. “If that were the case, then the insurance industry wouldn’t exist.”
Beat the Bear?
You’ve probably heard this one. Two hikers get ready to go hiking in the woods. One of them notices his buddy is putting on running shoes and asks why. The other responds that he heard there were bears in the woods and he wants to be able to run faster. The first hiker, wearing hiking boots, says that’s ridiculous and tells his buddy you can’t unrun a bear!
Yes, the other hiker responds. “I don’t have to outrun the bear. I just have to outrun you.”
“Just because something can’t be proven or certain or perfect, you can’t just default to an inferior model,” Hubbard said. “All you have to do is be marginally better.”
How to Measure Cybersecurity Risk
“There are really only two ways (to assess threats and risks),” Hubbard said. “Run deliberate experiments, clinical trials, and things like that, or wait for things to happen and hope everybody else is sharing enough data when it happens to them.”
If enough data is available, you can make logical assumptions about what might happen. For example, he said, by looking at 1,000 companies over a year, you can get some good frequency data on various events. However, he warns, that’s not perfect. “Not everyone is sharing everything that happens to them,” Hubbard said. “There are some regulatory requirements in many spaces, but we still don’t see everything.”
For example, an organization might be required to report a data breach, but they may not disclose everything about the breach. “We don’t necessarily know the details of exactly what happens in each of those cases,” Hubbard said. So, it’s a little harder — based on that information alone — to work out how what happened to them would apply to another organization.
“If everybody were sharing lots of data, then we’d get lots of feedback right away,” he said. However, we only get to see pieces of the overall picture. “The only way I’m going to get a lot of data is experiencing.”
SightGain Provides Experiential Data and Threat Assessment
SightGain’s cybersecurity risk management solution can help you prove and improve your cybersecurity posture. Look inside your security programs by using real-world threat performance data by testing your systems, processes, and people using live-fire threats and attacks because of current security threats.
While some tools provide you with metrics about what threats are being stopped, they fail to provide actionable information about where you are falling short. SightGain helps operationalize cybersecurity threat intelligence in multiple ways:
- Security threat exposure management: Gain cybersecurity risk management with threat intelligence, including prioritization for improvement.
- Security operations validation: Exercise people, processes, and technologies using live-fire threats.
- Security staff training: Automate training assignments, administration, and tracking.
- Standards compliance assessment: Automate reporting for NIST, Zero Trust, ISO, CMMC, and more.
SightGain’s continuous threat exposure management platform helps you to:
- Find security gaps and redundancies.
- Measure the effectiveness of your tools and SOC analysts.
- Conduct live testing and training of analysts within your production environment.
- Quantify risk, including the likelihood and financial impact of breaches.
If you want to know more about your threat exposure, how to assess risk, and verifiable insights and recommendations to improve your security posture, schedule a demo of SightGain today.
