Robots are taking over the world!
That’s the dystopian vision conjured in the minds of many when talking about Machine Learning.
But don’t be swayed by the hysteria. Machine Learning offers enormous potential. The key is to find ways to leverage it so it opens up new insights into patterns that we humans simply can’t detect without a bit of computer assistance.
At Shearwater, our commitment is to your security. We are constantly on the hunt for new and innovative ways to defend organisations from a broad range of cyber threats.
We believe Machine Learning can play a significant role in the quest to achieve stronger levels of cybersecurity.
The challenge of identifying threats
Network security monitoring requires a broad range of technologies and tools. To achieve a comprehensive security posture, each of them needs to work together harmoniously.
However, getting them to communicate with each other can be a major challenge.
When the plethora of tools don’t communicate with each other, there’s a risk you’ll only gain visibility into a small fraction of network access requests.
And even if you do achieve a high level of visibility, deciphering all the data requires specialist skills.
Security Information and Event Management (SIEM) provides a solution by collecting information, aggregating it and turning it into insightful, meaningful knowledge.
In theory all your network activity will be logged and you’ll have the required visibility.
Perfect! The end.
Not quite…visibility is only half the battle.
Attackers are operating under a cloak of anonymity, often disguised as day-to-day users. Significantly, the most serious threats are the ones you can’t see. Attempting to identify their activity among the troves of logs can be difficult and cumbersome.
Just imagine the vast depth of data your network security tools record each day:
· Application logs
· System logs
· Security logs
And that’s just the beginning. Consider every failed password attempt – it would also generate a log. There can be literally thousands of these logs each day. And this represents a tiny fraction of the activity being recorded.
What is Machine Learning and how can it enhance your security posture?
It’s clear the traditional approach for logging and flagging security threats in organisations is far from efficient.
However, Machine Learning can assist us in this task by providing some degree of automation.
By tapping into the potential of Machine Learning, there exists the possibility of mapping datasets, from which the computer can learn to identify and flag potential threats. Over time, the computer will learn from both its successes and failures to enhance performance automatically without the need to be explicitly programmed.
Harnessing this technology can quickly and automatically produce models that can be used to analyse even larger, more complex datasets. This in turn delivers more accurate results, more efficiently.
Enter Ken Liu, Shearwater’s latest security protégé.
Ken, a Shearwater Security Analyst and recent graduate from the Bachelor of IT (Network Security) degree at TAFE NSW, is a keen Machine Learning student.
As part of his studies, Ken’s research focused on training a computer to monitor logs of server “traffic.” By analysing extensive database logs, Ken enabled the computer to identify what appeared to be “normal” events that weren’t in fact actually “normal” at all – they were hackers in disguise!
With this level of insight, Ken was then able to factor in other known issues and feed them back into the system. For example, as the machine learnt to successfully identify and flag significant events, it remembered how to detect them in the future.
This saved Ken hours of data trawling and created a virtuous learning cycle.
Achieving accuracy was challenging due to several false negatives in which potential threats were flagged that were not in fact threats.
Yet, with ongoing support and assistance from the Shearwater team, Ken was able to overcome this challenge and achieve a much higher level of probability that only genuine threats were being highlighted.
This offers an instructive lesson: for Machine Learning to succeed in providing value, it requires an element of human experience and intuition. By combining the strengths of a computer’s analytical and pattern-matching capabilities, together with human experience and intuition, Ken was able to achieve an optimal outcome.
Humans provide the thinking.
Computers provide the horsepower.
Through his research, Ken also found that different approaches were required for analysing different database logs using Machine Learning. A one-size-fits-all strategy will not work. Every organisation requires a unique approach, as each differs vastly in terms of size, industry and their own internal IT environments.
So, what’s next?
Ken is planning additional Machine Learning research with a view to integrating his work with the advanced security monitoring systems Shearwater already uses.
By integrating the insights gained from his work, Ken believes Shearwater will be able to improve the quality and efficiency of the security services it offers clients.
How Shearwater can help you?
Do you need to get a handle on your logs? Does your organisation have a plethora of security systems that are not communicating with each other effectively?
By tapping into our expertise and innovations, you’ll enhance your organisation’s security capabilities to protect yourself from the growing range of threats.