Introduction
In the world of computing, security is still a relatively new concept for many organizations. Within the security realm, how to quantify security is still a relatively new area. Combine these two factors and you are likely to find organizations that are frustrated by trying to justify investments in security. One of the hardest questions to answer as a security practitioner is “What exactly is my level of risk?” Part of the reason is that it is very difficult if not impossible in some cases to assign a quantifiable number to a qualitative object. What does this mean? For example, what number would you assign to internal user security awareness? Sure, you could give quizzes and surveys and determine a number that could be used for a relative comparison but there are some caveats as well. This exercise may very well be impractical within a large organization and possibly the number you generate indicates a high level of security awareness but you still can not quantify the malicious intent of Joe in accounting (Sorry Joe). This ultimately changes the quantified number into a relative quantified qualitative number.Before you decide to quit your security career and start a career growing tomatoes, know that with a bit of work and a sound approach you can get a lot closer today than you ever could in quantifying organizational risk.
Where to begin? There are numerous approaches and frameworks that may be specific to your industry or objectives and as such it is worthwhile to perform some investigation based on your context. For example, application developers may wish to leverage Cigitals Security Program and its touch points or infrastructure engineers may wish to use OISSG’s ISSAF framework (if that wasn’t enough alphabet soup for you, throw in their CCIF as well).
All frameworks aside, the basics of metric analysis remains the same:
- Work With Your Customer - Work with your customer (line of business, client, etc) to determine what are their objectives for the service/application you are going to measure. This will help align IT with business and ensure that the deliverables match closely to the expectations.
- What To Measure - Determine what it is that you are trying to analyze with your measurements – Maybe you wish to determine how your new desktop AV product has changed your exposure to virus incidents.
- Relevancy - Determine which measurable metric(s) are relevant to this analysis – the concept of ‘If it cannot be measured, it cannot be managed’ holds true. In the above example, you could look at measurements of number of viral outbreaks, number of viruses caught by AV, number of help desk tickets for virus issues, and even the effectiveness of the product with the ability to maintain software currency.
- Collection Methods - Determine how to collect these metrics – this can range from simple log parsing to advanced correlation engines.
- Establish a Baseline – If historically you had 30 outbreaks a month; this could be the basis for your baseline. Metrics are only useful as a tool for comparison and must be consistent in their collection. A metric of 20 outbreaks is only relevant if your measurement of the month before was 30 outbreaks.
- Review and Optimize - Review your metrics regularly to determine if they are still relevant – while it is not likely to happen; how relevant is the anti-virus example of above if your company no longer uses desktops?
