Op-Ed: How data visibility is the key to security

Organisations need a better data foundation to minimise damage from inevitable data breaches and comply with new regulations like the Cyber Security Act 2024. This foundation intends to provide immediate, single-pane visibility into all enterprise data across all environments. Adding AI and automation into the mix lets organisations detect anomalies and threat patterns rapidly; they can thus meet the 72-hour reporting requirement for ransomware payments, as mandated by the new legislation.

Here are some things to consider when getting started.

Taming the sprawl

The exponential data growth in modern organisations has led to unintentional sprawl. Different business divisions often harness disparate, siloed storage platforms, including on-premises systems and various public and private cloud environments. As data is scattered across numerous repositories, safeguarding against breaches becomes much more challenging, and detecting an intrusion quickly becomes highly difficult.

VIEW ALL

As a result, data breaches involving organisations with data stored across multiple on-premise and cloud environments are costlier and take longer to contain. Last year, such breaches cost Australian businesses an average of $4.4 million, posing a significant financial and operational burden.

Consequently, organisations have been working to consolidate their data storage into centralised, cloud-based repositories like data lakes. These vast “reservoirs” store enormous amounts of raw data in its original format, regardless of its structure or source. Instead of data being siloed in different departments or systems, a federated data lake allows authorised users from across the company to access and analyse data as needed. With a federated data architecture, multiple data lakes are combined to provide a single data source, easing collaboration and fostering the use of new AI applications. With a federated model, costs resulting from data duplication and high data transfer fees from cloud providers are reduced.

More importantly, unrestricted access to data enables security operations (SecOps) teams to have full visibility into aspects like network traffic, user interactions, and system logs. This capability allows SecOps teams to deploy AI and automation technologies to spot suspicious behaviour and minimise damage. In fact, the global average data breach cost for companies wielding AI or automation for breach detection was $3.26 million, 26 per cent lower than the costs incurred by companies that were not utilising these solutions. In addition, they were able to help cut the breach life cycle down by roughly 98 days, reducing the fallout of attacks.

Capacity and speed

Not all data lake architectures are the same, and many solutions compromise storage capacity and querying speed. To maximise threat detection capabilities, organisations should find a platform that scales to meet storage needs affordably and enables low-latency data querying. This data ought to include everything from neatly organised spreadsheets to unstructured data, messy log files, and documents, giving a more holistic picture of the company’s security situation.

The combination of high capacity and low latency allows for the real-time investigation of potential security incidents across an organisation’s entire digital environment, including diverse datasets from different cloud environments and on-premises applications. With quick access to all structured and unstructured data, enterprises can run AI-powered security information and event management (SIEM) tools that automatically flag anomalies and potential security risks, allowing IT staff to focus on the most critical issues.

The ability to maintain a larger repository of data – such as network activity logs dating back several years – in a quickly searchable state gives these tools a broader historical record from which to find complex patterns, detect trends, and predict future security risks. This is becoming increasingly important with the growing prevalence of advanced persistent threats (APTs), a sophisticated, long-term cyber attack carried out by patient adversaries over months or sometimes years.

Next-level insights

By gaining comprehensive visibility into enterprise data, organisations can implement diverse AI-driven SecOps solutions to enhance their ability to detect, comprehend, and respond to threats. This includes utilising generative AI (GenAI) applications that leverage large language models.

These GenAI models provide context-aware guidance to security analysts, grounding their recommendations in real-time organisational data. For example, they may consider factors such as user risk scores, which assess the potential threat level associated with individual users based on their behaviour patterns and access privileges. They can also perform an asset criticality assessment, which prioritises threats based on the importance of the target systems or data.

The GenAI-powered systems can then offer highly relevant and actionable insights by integrating this contextual information. For instance, they might correlate a suspicious login attempt with a user’s recent travel history and the criticality of the accessed system, providing risk assessment with a much deeper level of nuance than singular tools could offer.

This level of intelligent analysis enables security teams to respond to threats with unprecedented speed and precision, focusing their efforts on the most pressing threats and giving teams a better chance of catching breaches before they can be completed. As capabilities evolve, AI systems will become even more effective at identifying and mitigating threats, staying one step ahead in the ever-evolving cyber security landscape.

When utilised effectively, these advanced SecOps tools offer the most powerful means of reducing the detection window for data breaches and the cost of recovering from them. SIEM solutions like Elastic Security provide AI-driven, comprehensive threat detection, investigation, and response capabilities. It also offers extended and native protection for endpoint and cloud security. Being an open-source solution ensures that it integrates with other security and IT technologies, a critical feature that cannot be overlooked in today’s multi-cloud, multivendor IT architectures.

Security is a data problem, and for security systems to work effectively, organisations need a data platform that provides unencumbered and unrestricted visibility across all facets of enterprise operations.