Elevating Your AppSec Program with Metrics

As organizations begin to recognize the value of implementing a robust Application Security program, we expect significant investment in related tooling and additional resources to perform more manual activities, such as penetration testing and threat modeling. While these modern solutions and exercises are critical to maintaining strong AppSec practices, organizations often struggle to quantify the success of the AppSec program and their overall security posture across their entire application portfolio due to insufficient metrics and reporting capabilities. 

Capturing and accurately reporting on relevant metrics is essential in managing both risks to the organization and gaining better visibility of the performance of application security investments. Organizations must measure the governance, risk, and compliance of their AppSec programs and communicate the effectiveness of these programs and how they impact business risk. Metrics are also crucial in effectively managing and monitoring the building blocks of all security programs - people, processes, and technologies. For example, if gaps in an organization's security processes require additional skilled resources, quantifying and conveying these gaps' impact on the business would help create a more compelling argument supporting why additional resources should be allocated. Additionally, as organizations consider expanding their AppSec programs, reporting a desirable trend in their performance over time increases the probability of greater investment in the program and security altogether. Regardless of program maturity, organizations should prioritize establishing actionable and practical metrics to ensure visibility into risk level, provide the ability to track progress and enable more informed decision-making processes.

Since most organizations are on the path of DevSecOps, security teams should strive to utilize the CI/CD pipeline to capture dynamic AppSec metrics. Organizations should incorporate real-time visualization and analysis when building out AppSec metrics, and the resulting reporting should assist in providing insights that can mitigate application risks more effectively, increase visibility into known security risks and new incidents, and identify components within the SDLC that require additional support. Some metrics organizations might consider tracking are:

• Mean Time to Remediate (MTTR): This metric provides insights into how effectively the organization addresses vulnerabilities within its defined SLA(s) and can be analyzed per business unit, application team, or other grouping that might apply to the organization. A lower MTTR minimizes the window of opportunity for adversaries to compromise the application(s).
• Percentage of Critical/High Severity Open Vulnerabilities per Application: This metric provides a comprehensive view of the robustness of the AppSec program itself. A lower number of open critical and high vulnerabilities might indicate more security-aware development teams and better adherence to security standards.
• False Positive Rate (FPR): This metric measures the quality of alerts identified by vulnerability detection tooling. A high FPR can result in legitimate findings being buried under irrelevant ones and decreased confidence in tools invested in. A low FPR indicates more accurate vulnerability detection tooling and processes and better visibility into veritable threats.

AppSec metrics should be top of mind for organizations looking to secure their Software Development Lifecycle (SDLC). Metrics play a pivotal role in evaluating, improving, and maintaining an organization's security posture, demonstrating compliance, driving informed decisions, and fostering a culture of security awareness. Organizations can proactively safeguard their applications from cyber threats by assessing and monitoring key security indicators and embracing metrics as an essential part of their AppSec programs.