Design Flaws In Cyber Security Reports And Related CISO Sleep Patterns

I recently stopped using my fitness tracker, though not due to a cyber security breach or privacy concerns. Rather, it came down to the overwhelmingly negative reports provided by the app.

Like many CISOs, I don’t sleep much; in my case, getting by on five to six hours of sleep a night is hereditary. Although the tracker collected detailed telemetry, the app only provided comparative reports against other people. Despite my experience, the app alarmingly claimed I’d been having terrible problems sleeping for weeks in a row.

Producing highly accurate reports without individual customization is a consistent design flaw of many cyber security solutions available today.

Some solutions offer highly detailed telemetry and reporting based on the individual organization. Before organizations deployed Security Information and Event Management (SIEM) solutions, we needed to wade through all the logs from their Endpoint Detection Response (EDR), Antivirus (A/V), firewall, Intrusion Detection System (IDS), Intrusion Prevention System (IPS), the insider risk management platform, the privileged access management platform, various platform logs like syslog and the Windows event log, and other logs.

See Related: How To Create Enterprise Security Champions

Avoiding Panic Or Fatigue From Data Overload

SIEM solutions promised to be the single pane of glass to view these alerts, which unfortunately led to our culture of alert fatigue among Security Operations Center (SOC) analysts. Giving organizations all these data without sufficient context regularly produced one of two outcomes: panic, because everything’s beeping and must be urgent; or fatigue, because everything’s beeping and there’s no way to know if that’s normal.

Other cyber security solutions offer great comparison data based on industry vertical or industry size. Threat intelligence is a great concept, as organizations can see what’s happening to similar organizations. The ability to constantly simulate attacks and breaches as seen by similar-size companies could provide value and help find controls that aren’t configured correctly or deployed at all. The way to see your organization and your supply chain as a threat actor would similarly provide value.

However, solutions in this space tend to be ambiguous, and lack the organizational customization necessary to provide value. Knowing that organizations like yours tend to get attacked by nation states for financial reasons is only useful if you can detect that those attacks have happened or are being blocked. Knowing that the software supply chain in your market vertical tends to be vulnerable to JavaScript injection leading to Magecart-style attacks only makes a difference if you can assess if your actual software packages and platforms are vulnerable. Otherwise, these solutions only help reinforce the message that bad things are happening in the world.

See Related: 6 Criteria For Building A Security Culture

Strive For Data-Driven Decision Making

A single vendor’s report each month crosses my desk and brings a smile to my face. The report shows the number of events, the number of new events that haven’t been seen before at our organization, the typical number of events observed in similarly sized organizations in the same vertical, and the new events those other organizations are seeing. There are other data, too, but the report is only four pages including a page of analyst remarks and recommendations. A voluminous amount of data exists behind the scenes processed by a mixture of artificial intelligence and human analysts, but it’s summarized so that it provides context and supports decision-making.

CISOs might sleep better at night knowing not only what’s specifically going on at their organization but also how that fits into the global threat landscape, and how that compares to similar organizations. As you consider your cyber security investment budget for 2020 and beyond, consider giving up on those solutions that have poorly designed reports that either create stress, or create ambiguity that hinders decision-making.

The role of the CISO isn’t to read all the data and derive a conclusion on their own. Rather, it’s to review the summaries and the opinions of trusted advisors before making decisions or presenting those decisions to the Board.

See Related: Shifting The Cyber Conversation From Technology To Risk With Admiral Michael Rogers