Cybersecurity Big Data Overload?

Overview: The Cyber Big Data Conundrum

Organizations are challenged with the spiraling overhead associated with storing and processing ever inflating sets of cybersecurity data. Many admit they are struggling to evidence improved security event detection and decision making. A growing realization is that big data must be turned into smart data: relevant, high-quality data which can be easily leveraged for advanced analytics.

Cybersecurity professionals have come to understand that ingesting data into a data lake and utilizing it to create value are two distinct challenges, particularly when there is a desire to conduct advanced analytics. After heavy investments of time and resources to collect data in bulk, many cybersecurity data analytics efforts flounder on the shoals of complexity. SAS Institute, the global leader in analytics for over 40 years, has a focused set of solutions to gain control of your cybersecurity big data.

Focused Data Management

The first step is to improve data management, particularly quality and relevance. Data feature selection indicates which data is most relevant to the inquiry at hand, reducing and optimizing the data imprint. SAS provides a powerful toolset of routines for variable and dimensionality reduction, feature engineering, and correlation analysis. Data routines produce insights into data quality lapses along with a variety of treatments to manage these lapses, including reduction, sampling, and/or guidance concerning binning. This includes improving insights into the network itself: the discovery and mapping of user, network, device, and digital assets, even the identification of unregistered or unknown entities.

Leverage Discovery Analytics to Identify Hidden Patterns

Cybersecurity professionals struggle to manage unknown-unknowns (i.e. zero-day or hitherto unidentified attacks), both unseen vulnerabilities as well as the threat of complex, evolving attacks. Data analytics empowers the discovery of hidden patterns and the detection of evolving threats. Data analytics can be applied to discover patterns regarding network assets and usage, including the nature of hidden assets, and to profile patterns in associated behaviors. By establishing a baseline concerning categories of assets and users (the ‘Norm’), a foundation is set for anomaly detection. When asset access, device behavior, and/or user behavior fall out of categorized ranges, anomalies indicative of potential incursions, misuse, or abuse surface.

Beyond Rules: Self-Reinforcing Detection

Should there be a record of known compromises, predictive detection models can quickly be tested and implemented However, given rarity and the evolving nature of attacks, such examples are not always available. Via semi-supervised machine learning, an initial detection model can be bootstrapped to detect focused statistical anomalies. The resulting model allows for targeted contextual alerts when anomalous signals suggest at-risk users and assets are potentially being compromised. The underlying anomaly detection model is refined and improved as cases are confirmed or rejected through subsequent investigations.

Cybersecurity Detection Model Management

Bound together, the combination of discovery and detection analytics approaches iterate in a cyclical fashion to refine targeted understanding. A robust environment is provided for managing, iterating, and testing detection models. A range of advanced analytics and machine learning algorithms can be tested like-to-like to select and deploy a champion model.

Empower Investigations and Workflow

A robust set of tools are available to support both cybersecurity investigators and case workers. Investigators can utilize preformatted investigator dashboards and reports, including the ability to run self-service analytics. As needed, a powerful cybersecurity case workflow platform is available to support case workers in routing triaging, and remediating alerts.

Optimize Cyber Resource Utilization

SAS offers targeted solutions to support resource workflow optimization in the context of evolving risks. As detection models improve cyclically, workflow metrics are analyzed to support intelligent routing and resource optimization. As well, an organization can pilot new workflows and observe results before implementing in full production.

Want to Know More?

The full whitepaper is available here.

Ready to go? Seeking a discussion, demo, or trial? Let us know your needs here:

Scott Mongeau is a SAS Institute Cybersecurity Data Scientist and university lecturer/researcher in applied data science


, , , , ,

About SARK7

Scott Allen Mongeau (@SARK7), an INFORMS Certified Analytics Professional (CAP), is a researcher, lecturer, and consulting Data Scientist. Scott has over 30 years of project-focused experience in data analytics across a range of industries, including IT, biotech, pharma, materials, insurance, law enforcement, financial services, and start-ups. Scott is a part-time lecturer and PhD (abd) researcher at Nyenrode Business University on the topic of data science. He holds a Global Executive MBA (OneMBA) and Masters in Financial Management from Erasmus Rotterdam School of Management (RSM). He has a Certificate in Finance from University of California at Berkeley Extension, a MA in Communication from the University of Texas at Austin, and a Graduate Degree (GD) in Applied Information Systems Management from the Royal Melbourne Institute of Technology (RMIT). He holds a BPhil from Miami University of Ohio. Having lived and worked in a number of countries, Scott is a dual American and Dutch citizen. He may be contacted at: LinkedIn: Twitter: @sark7 Blog: Web: All posts are copyright © 2020 SARK7 All external materials utilized imply no ownership rights and are presented purely for educational purposes.

View all posts by SARK7


Subscribe to our RSS feed and social profiles to receive updates.


  1. Open Call: Cybersecurity Data Scientist Practitioner Research | BAM! Business Analytics Management… - October 20, 2018

    […] Cybersecurity Big Data Overload? […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: