A proof of concept research project, presented in story form. With pictures. By stack.io.
Have you heard of Falco?
Well you should. Falco is a security tool. It is a last line of defence for your cloud infrastructure. Let’s take a look at why you should consider using it.
Current security landscape
When it comes to securing cloud infrastructure, the majority of time is spent front-loading the efforts. Most of the tools in the DevOps arsenal are focused on ensuring security incidents are mitigated (with a preference towards preventing it entirely).
Take a typical web application. Once the codebase has been committed to a repo, you may decide to subject it to code quality and security analysis by the likes of SonarQube or similar.
Once it passes, you may load it into a container, and further subject it to scanning by container vulnerability scanners such as Trivy.
We’re not done yet. Once the container passes the test, you may load it into a container registry such as Docker Hub, Amazon ECR, or Google GCR, and those systems may subject your containers to further scans.
The orchestration environment may also be scanned for vulnerabilities such as Kubesec for Kubernetes.
This technique of front-loading is effective against security threats such as package vulnerabilities and unwanted intrusions from those Bad Actors. You do what you can to lock down your app, containers, orchestrators, and cloud accounts before exposing it all to production traffic.
But what happens if someone (or thing) breaks in?
It happens to the best of us - even the well intentioned and disciplined teams. If there’s one thing about Bad Actors, they are persistent. What happens if one of them manages to find a way in?
Once compromised, a Bad Actor can do many things to take advantage of this exploitation. For example, using a package manager (or equivalent), they may install a shell to gain terminal access.
They may further download a toolkit to help with the nefarious work.
At this point there’s plenty of reconnaissance that can be done. Using nmap, the network can be probed and mapped to find out what else is inside.
It’s a common misconception that containers are more secure than say VMs. This is not always the case.
It may be discovered that Kubernetes is the orchestrator, or maybe AWS ECS. APIs are queried and encoded (not encrypted) database credentials from the environment are fetched. At this point it’s trivial to use these credentials to connect to the database and dump the data.
They may further reach in their toolbox and deploy some nasty malware or more common these days - ransomware.
They may encrypt your database and demand payment before giving you the key to decrypt.
Often the Bad Actor(s) may disseminate the stolen data and sell it for pieces on the Dark Web if payment is not made (and sometimes even if payment is made).
You have now been pwned.
Based on a recent report generated by IBM, on average it takes organizations 9.5 months to even discover they have been hacked. And breaches are very costly. Costs can be contained the faster breaches are discovered and actioned.
Contrary to this, Ransomware attacks happen much quicker. A recent report by Sophos has identified that “dwell time” (the time it takes from initial breach to discovery) is decreasing, and is now hovering around 5 days! This means that from the moment a Bad Actor gets inside the network, to when they make themselves known by demanding payment is less than a week.
Further to this, it appears Ransomware Bad Actors tend to time their infiltrations to after working hours of the victims, to reduce the chances of being detected, with a preference towards Fridays and Saturdays, when Systems People may be harder to reach.
This condradictory information between the reports by IBM and Sophos does not take away from the fact that Bad Actors really only need anywhere from minutes to hours to compromise a system. Whether it takes organizations 5 days or 9.5 months, it's already too late. Catching the breaches, preferably in the act, is of the utmost importance.
Enter Falco
Falco is a real-time security tool that monitors syscalls. Syscalls are the low level system related calls to the kernel to do anything on a system such as establishing a network connection, reading a file, or launching a process such as a shell.
By monitoring these syscalls, Falco is in a unique position to access vital information.
Monitoring syscalls is like drinking from the firehose - it’s noisy, numerous, and verbose. Using pattern matching, Falco can filter out important and interesting calls, such as launching a shell or installing a package.
Falco comes with a default rule set to capture the common patterns of a Bad Actor and their commands out of the box. A web interface is also available to view the data. Once captured, Falco can trigger one or more integrations to take action, such as sending notifications to chat.
Where things get really interesting is with the Custom Action.
With some simple code, a Function-as-a-Service (FaaS) function call (such as AWS Lambda) can be called to push a network policy to cut off access and isolate the container.
A notification can then be sent informing the Systems Person of the incident, and allows them to inspect the compromised, but isolated container to figure out what went wrong.
Disclaimer
It’s probably the right time to mention that we are not affiliated with Falco. What started off as an internal R&D initiative to investigate better audit logging led us to discover Falco. We are sharing this research because we believe the community should know about this tool and how it can help.
And we like telling stories.
How much does Falco cost?
Falco is free and open source. Cloud costs are minimal - the overhead of the driver and supporting tools is very small.
Below is an example of the additional costs of running Falco on a 20-node AWS EKS setup:
Interested in trying it out?
Check out our GitHub repository which contains example Helm charts, configuration files, script files, and documentation to get Falco up and running with minimal disruption.
What does this get me?
In the box is a cloud-agnostic, highly-available, production-ready implementation of Falco.
This turnkey, time-saving configuration includes:
Falco-driver: Monitors and helps detect malicious or unexpected activity by capturing and analyzing syscalls.
Falco sidekick: An additional component that enhances the capabilities of Falco by providing a way to forward the alerts generated by Falco to various destinations or outputs such as Slack, Mattermost, Teams, Datadog, Prometheus, etc.
Redis in-memory database: Allows for the UI to query it in order for the syscall events to be displayed on the dashboard's UI.
Sidekick-UI: Allows data to be displayed on a dashboard
As Falco is cloud agnostic, it insures compatibility and easy integration with various cloud providers, offering flexibility in deployment and scalability.
This turnkey configuration provides out-of-the-box solution that help you get started quickly, reducing the time to deployment and easing the setup process.
Falco's comprehensive toolkit equips enterprises with a robust solution needed to secure Kubernetes environments.
Being a part of the CNCF, Falco benefits from a community of experts dedicated to pushing the boundaries of what is possible in security monitoring.
Falco offers a high degree of customizability, allowing for the development of tailored security solutions that meet unique business needs.
Need assistance or a quick demo?
Do you need help implementing or have questions?
We're here to help!
Get Notified
Be the first to know when we release another POC