Fences and controls: three metrics to help you better measure your cyber security capabilities

Effective cyber controls are much like effective fences. They must be well maintained, encircle the whole perimeter, and be able to block the threats that seek to bypass them. But how can you find out how you are faring on these parameters? Let’s have a look.

There are three factors to think of:

1. Maturity (how well the fence is maintained),
2. Coverage (what proportion of the perimeter it encircles), and
3. Technical effectiveness (how it blocks threats seeking to go through it).

Each factor helps to determine the control’s true effectiveness, so all must be measured accurately. The factors should be measured and scored as a percentage (or a 1 to 5 rating that’s further converted to a percentage) before multiplying all three scores together to calculate the overall, more accurate effectiveness. A control that’s technically brilliant but only covers half the estate will only stop half of attacks at best, while a control with comprehensive coverage but crumbling due to a lack of maintenance will see threats slip through it. 

Once installed, without maintenance, even the most advanced fence will become vulnerable as gaps may begin to appear that attackers can exploit (be that a fox, a burglar, or a hacker).

The Capability Maturity Model Integration (CMMI) levels can help to  determine if a control’s management is:

1. Initial (ad hoc or poorly managed),
2. Managed (owned by a named individual/team),
3. Defined (managed in line with documented policies and standards),
4. Quantitatively managed (metrics produced help objectively track performance),
5. Optimising (management is automated with fine-tuning occurring based on metrics).                                                                                                                       

An effective fence is built and maintained against a standard by a dedicated team, with metrics determining when to apply repairs (e.g. a certain depth of rot is acceptable in a fence post before filler or full replacement is required).

An effective anti-malware control is owned by the security operations centre (SOC), with data such as the volume and types of attacks produced, stored and analysed, to help flag hazardous user behaviour. 

If it’s possible to walk around, bypass the fence and enter the building, then it’s not very effective. The size of the gap dictates how much of the coverage is missing.

Coverage can be measured by using percentage (%), with scoring determined by assessing what proportion of the organisation and/or its assets requires the control to be in place and then verifying whether the control applies to them.

An effective fence encircles the entire perimeter. If a perimeter is 100-metres and there is a 10-metre gap, then the coverage is 90%.

An effective patch management solution is installed on all endpoints and systems that require it. If 90/100 of the organisation’s laptops and servers have the patch management solution installed, then the control has 90% coverage.

Regardless of a control’s maturity and coverage, it will be ineffective unless it can block the threats it is supposed to stop. A white-picket fence that is in perfect condition and encircles the entire perimeter is useless for stopping a car driving over it.

Technical effectiveness can be scored on a 5-point scale:

1. Ineffective
2. Mostly ineffective
3. Somewhat effective
4. Mostly effective
5. Effective

Consider what each level of technical effectiveness should look like and have clear indicators for each can help provide a clear view of capability effectiveness. Our Cyber Risk Insights team have defined thresholds for the cyber security controls, by working with experts in the field of security architecture and penetration testing.

An effective fence is 6-meters high, constructed of reinforced concrete, with razor wire on the top.

An effective password policy utilises Single Sign-On (SSO), with multi-factor authentication (MFA), and a password manager that requires 40-character phrases that are rotated every 30-days.

In addition to providing a more accurate view of control effectiveness, measuring comprehensively provides operational insights that enable security teams to know how to strengthen controls. While most assessments will indicate to CISOs which controls they need to improve, measuring control effectiveness in this more accurate way enables them and their teams to know how to make those improvements.

Measuring the effectiveness of cyber security controls can be achieved by understanding what makes an effective fence. By assessing maturity, coverage, and technical effectiveness, practitioners can both more accurately measure control effectiveness, and have the information to know how a control can be further strengthened.

Do you understand how effective your controls are? And what can you be doing differently to strengthen your organisational cyber security fence? To discuss what improved control effectiveness can mean for your organisation, feel free to reach out to Michael Yeomans.