Jessica Ireland will teach you how to gauge your security capability.


“If you cannot measure it, you cannot improve it”. That was the view of Lord Kelvin, who discovered the theory of absolute zero, and came up with the Kelvin scale. But can you measure your organisation’s performance at cybersecurity? And what would that look like?

Analytics are sweeping every other aspect of technology, it seems. Companies and governments are searching for insights in mounds of numbers, hoping to tweak a marketing approach here and a public policy there. At SecTor this October, Jessica Ireland, research manager for security and risk at Info-Tech Research, will shed some light on how cybersecurity professionals can use metrics to improve their game.

A metrics program can be a useful weapon in your arsenal, she asserted – after all, it’s difficult to manage what you’re not measuring.

Value over quantity is the mantra for her talk, or, to put it another way, increasing the signal to noise ratio. A carefully thought-out metrics strategy can help cybersecurity practitioners to find actionable intelligence that can help to protect their organisations more effectively, she said.

What kind of intelligence, exactly?

One of the most basic insights is to find out which of your solutions is catching cybersecurity incidents, and which are not. That can give practitioners some clues in finding more effective solutions, believes Ireland.

Cybersecurity pros can use metrics to better understand gaps in their process, she promised. “It’s an opportunity to understand what’s working and what’s not working at a more detailed level and eventually improve on the success of projects and resource allocation,” she suggested.

“There are also many opportunities to do some in-depth analysis around trends, and correlations,” she continued. “Metrics add great context to technology solutions in terms of making sense of increased visibility. On its own, having more visibility into what’s happening is still valuable, but metrics can take it a step further.”

Frameworks for cybersecurity metrics

There are different approaches to implementing cybersecurity metrics. One of them is outlined by the National Institute for Standards and Technology. In its Performance Measurement Guide for Information Security, NIST breaks down cybersecurity measurements into three broad areas: implementation, effectiveness, and impact measurements.

Implementation measurements could include the percentage of systems covered by approved system security plans, say, or the percentage of information systems with correctly configured password policies. These can also link back to standard IT operations, measuring the percentage of servers in the organisation that have a standard configuration, perhaps.

Efficiency and effectiveness can tell the cybersecurity pro how well these measures are working. Expect to see information such as the number of security incidents caused by improperly configured access controls here, advises NIST.

At the more sophisticated end of the spectrum, impact measurements can tell cybersecurity teams things like the levels of security budget devoted to specific areas of information security, and perhaps even assess the level of employee awareness. This is an important tool when justifying cybersecurity investments to the rest of the business.

All of this may sound daunting at first, but the NIST definitions are linked explicitly to levels of maturity, with the measurement of basic implementation seen as the lowest level of maturity, and business impact as the highest. This points to an incremental approach when tackling cybersecurity metrics.

Organisations in Canada that haven’t yet explored metrics for cybersecurity purposes needn’t feel behind on the bell curve, encouraged Ireland. Baby steps are fine.

“Developing effective metrics programs is not widespread. So if you haven’t yet, don’t feel behind,” she said. “Why organisations can be hesitant to start is that it seems overwhelming, but the key is not to boil the ocean. Focus on a few key metrics, even if they’re simple, because as long as you’re consistently tracking something, you can begin to build on the program and mature each year.”

This will be important, because in a fully-developed metrics program, information will come from a variety of sources. Companies should beware of looking for all their information in one place as they expand their capabilities, Ireland warned.

“Logs are not the only focus of security analytics, but they provide good raw data. The challenge is that they only really tell one story,” she said. “The goal is to get them from raw to verified data, and then to the point where you can take action on a particular issue.”

User awareness and process efficiency are examples of areas that would source information outside logs, she said. “How long is it taking you to respond to security incidents? How much is that costing you?” she muses.

Identifying and crafting those different data sources will take time, as will drawing them into a metrics system capable of converting them into actionable intelligence. Do the hard work up front, though, and it could pay off in spades later on.

“Security is poised to take advantage of this, and it should be as an industry, because it’s critical to the success of business nowadays,” Ireland concluded.

Interested in finding out more? Register at SecTor, which takes place at Metro Toronto Convention Centre in downtown Toronto on October 20-21, with a training day on October 19.