What is Container Runtime Security?
For cybersecurity professionals, DevOps engineers, and IT managers, understanding and mitigating container runtime security issues is imperative. The ephemeral and dynamic nature of containers gives rise to complex security concerns, particularly during runtime, when containers and their contents are active and exposed to the application environment.
Get Early Access to RAD Security - Drift Detection with Cloud Native Workload Fingerprinting
In the world of containerization, we think of runtime as the period when applications are actively running; that period when a program is being executed concurrent with all of the external instructions and dependencies needed for proper execution.
Runtime is one of three distinct phases in the application lifecycle, alongside build-time and deployment. Build-time concerns the actual construction of the applications and the coding and imaging that define the application itself. Deployment, meanwhile, mainly involves the configuration and set-up of the container environment.
To really understand runtime security, it's important to note that, in a perfect world, many runtime issues can be solved in the build-time and deployment stages. Application code and container images should be made free of vulnerabilities in the build-time phase, forming a strong foundation for the overall security of the app. The same goes for setting up strong environmental guardrails and rules of engagement in the deployment stage.
But this is not a perfect world. And so the onus is on developer and security teams to be ever vigilant to the risks that emerge when containerized applications are set into motion.
The High Cost of Runtime Security Failures
Runtime security failures in container environments can and do lead to catastrophe. Critical data can be lost, stolen, or otherwise compromised. Businesses can grind to a halt. Substantial financial losses can ensue. In worst-case scenarios, cybercriminals gain control over entire systems, allowing them to manipulate or destroy data, install malware, or use compromised system as a launch pad for attacks on other systems. In a world where data is a vital asset, the implications of such runtime security failures are far-reaching and potentially devastating.
In 2021, nearly 60 percent of organizations supporting applications containerization environments reported detecting container and/or Kubernetes misconfigurations in the past 12 months. One-third said they'd experienced a security incident during application runtime, the research also found.
In 2022, security researchers discovered more than 1,600 containers available on Docker Hub that included malicious code like DNS hijackers and cryptominers. Just one of these tainted containers is enough to compromise an entire app environment.
What are Container Runtimes?
While we tend to think about runtime security issues as the collection of vulnerabilities that arise and events that occur while apps are in motion, security practitioners and app development professionals also understand that Runtimes are a real thing. Container Runtimes are the actual software that instructs containers on how to run and interact with the host system and with each.
The capital “R” Runtime plays a critical role in the container ecosystem, providing isolation and serving as the execution layer that manages the lifecycle of containers, from creation through to termination. As such, it’s Ground Zero for a number of the security concerns that arise in containerized environments. Understanding Runtime elements and getting arms around the variety of available Runtime choices and use cases are indispensable aspects of container security maturity.
High-Level Container Runtime
Docker Engine: Perhaps best-known high-level container runtime, Docker makes it easier to create, deploy, and run applications and services in a containerized environment on any Linux or Windows. It packages software into standardized units for development, shipment and deployment. Within a Kubernetes architecture, Docker containers can be managed by the Kubernetes API, allowing for scaling, load balancing, and monitoring of containerized applications. When used within a Kubernetes cluster, Docker containers are managed by Kubelet using a Docker-compatible CRI, usually containerd.
CRI-O: CRI-O is a lightweight, high-level container runtime designed specifically to work with Kubernetes. Fully CRI (Container Runtime Interface) compliant — and tightly aligned with Kubernetes releases and features — CRI-O is used in Kubernetes environments to pull images from various registries, spin up containers, manage storage, and execute user workloads in Kubernetes pods.
Podman: An alternative to Docker, Podman is an open source, Linux-native tool specializing in running containers without a daemon, making it easy to find, run, build, share, and deploy apps using OCI-compliant containers and images. Like most other common engines (Docker, CRI-O, etc.), Podman relies on an OCI-compliant runtime like runc or crun to interface with the operating system and create the running containers.
Mirantis Container Runtime (formerly Docker Engine - Enterprise): Mirantis Container Runtime extends the capabilities of the basic Docker Engine within a Kubernetes ecosystem and is designed to meet the needs of enterprise users that need advanced features like FIPS 140-2 compliance, long-term support, and robust security options.
Low-Level Container Runtime
containerd: Because it technically offers an API layer on top of an OCI-compatible runtime, containerd straddles the threshold between low- and high-level runtimes. Available as a daemon for Linux and Windows, containerd handles the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc. In a Kubernetes context, it functions under the kubelet, which communicates with containerd to run containers based on Pod specifications.
runc: This is the default runtime underpinning other higher-level runtimes like Docker and CRI-O, and other platforms. Its focus is on spawning and running containers according to OCI specifications. Within the Kubernetes context, it is primarily utilized by the aforementioned container runtimes rather than being invoked directly by Kubernetes.
crun: A low-level runtime in C that implements the OCI runtime specification, Designed for rapid container execution and optimized for performance and resource efficiency with minimal overhead, crun aims to be a simple, lightweight alternative to runc.
runhcs: Focused primarily on Windows containers, runhcs is essential for running Windows-based containers in Kubernetes environments. runhcs fits into the Kubernetes architecture by enabling nodes to manage and run Windows-based containers and it bridges the gap between Kubernetes orchestration and specific requirements for running containers on the Windows OS.
rkt (pronounced “rock-it”): Developed by CoreOS (now part of RedHat), rkt is a pod-native container engine that integrates well with existing Unix systems. It was designed for modern production cloud-native environments, with particular attention to modularity and strong security features like container signature verification and isolation mechanisms. Although rkt can be used with Kubernetes via an alternative CRI implementation, its development was discontinued in 2019.
Container Runtime Security Issues
As the use of containerized apps expands, so does the threat landscape related to them. The most prevalent security issues faced during container runtime include:
Software supply chain
Images are a core component of a containerized architecture. These static files with executable code are used to pre-configure and pre-populate containers. In theory, at least, container images are immutable, and deploying them guarantees consistent results across any host environment. In practice, container images can have vulnerabilities and malware. When exploited, these can compromise the container and even possibly the host system. Containers often use open-source components that can be outdated. Images from untrusted sources are sometimes rife with malware, backdoors, or other unpatched security holes.
Inadequate Isolation Between Containers
Because containers share the same operating system kernel as their host, insufficient isolation opens the environment to a troubling possibility: A breach in one container could allow an attacker to compromise other containers in the environment or even the host system itself, allowing for unauthorized access or code execution code at the kernel level, cascading the overall impact of an attack.
Privilege Escalation
Attackers frequently exploit container vulnerabilities to gain escalated privileges. Misconfigurations or the use of default privileges often leave elements of the containerized environment with higher-level access than they really need for routine operation. This opens many windows for attackers to dig in, pivot, and move laterally through compromised systems. In the same vein, the leakage of container secrets leakage -- API keys or database credentials, etc. -- either through inadequate encryption, or weak handling and storage policies are a frequent source of runtime compromise.
Kubernetes security
Containers often come with default configurations that prioritize ease of use over security. This problem is compounded by so-called "configuration drift," where ill-advised changes are made either through human error or automated processes. When the configuration of parameters like file permissions and network settings drifts far enough from the secure baseline, the entire container environment is put at risk at runtime. Containers not properly designed to validate inputs, for example, are a prime offender when it comes to malicious code injection that leads to malware infection and systems compromise.
Go to the Kubernetes Security Master Guide
Container Runtime Security Best Practices
A complex container threat landscape requires equally robust defensive strategies. In order to effectively mitigate risks associated with container runtime security issues, experts recommend:
Regular Container Image Updates
Keep container images updated with the latest security patches. Utilize monitoring tools that scan for vulnerabilities and configuration changes within the images and fire alerts to the security team when potential config drift is discovered. Leveraging automated updates can further boost the efficiency and effectiveness of image defense processes.
Strong Isolation Mechanisms
Leverage container-specific kernel features such as namespaces and cgroups, and solutions like SELinux and AppArmor to restrict access to kernel features, enforce stronger isolation between containers, and reduce overall risk. Use init processes within containers to prevent processes from gaining additional privileges.
Least Privilege Principles
Run containers with the least set of privileges necessary. Limit root-level access to what's strictly required for the operation of the container and restrict kernel capabilities wherever possible.
Lock Down Orchestration Platforms
Platforms like Kubernetes should be configured with security top of mind. Apply network policies, Pod Security Policies (PSPs), and Role-Based Access Control (RBAC) to secure interactions within the cluster.
Regular Security Audits
Perform routine security audits and reviews to identify misconfigurations or deviations from security best practices. This includes ensuring that Kubernetes audit logging is enabled. These assessments should inform your security policies and strengthen your posture.
Best Practices for Container Hardening
Beyond proactive mitigation efforts, adhering to fundamental security practices at all stages of the app lifecycle -- build, deploy, and runtime -- remains a paramount aspect of container ecosystem defense. Topping the list:
Implementing Image Scanning and Vulnerability Management
Security in the CI/CD pipeline starts with the integration of image vulnerability scanning at application build-time. Maintain a robust vulnerability management program to promptly address identified security issues before they can find their way into production.
Enforcing Zero Trust Design with Strong Access Controls and Authentication
Adopt stringent access control measures. Authenticate actions using mechanisms like mutual TLS (used when a server needs to identify each client, often as part of a zero-trust network). Extend such controls to APIs and other management interfaces.
Monitoring Container Runtime Activity
Implement monitoring and logging solutions that provide visibility into container activities. This visibility is vital for detecting the kinds of anomalous behavior that often serve as the first indication of a security incident underway.
Regularly Patching and Updating Container Runtime Environments
Continuous patching and updating of the container runtime environment are crucial. Stay informed about vulnerabilities in platforms like Kubernetes as well as low- and high-level engines and runtimes like Docker Engine, rkt, runC, Containerd, CRI-O, or Mirantis and patch accordingly.
Container Runtime Security Tools
The broad popularity and widespread adoption of container environments has given rise to a plethora of bespoke solutions more or less aimed at bolstering the security posture of containerized apps and the systems that support them through highly specialized monitoring, scanning, and management features.
Some of the more popular runtime security tools today (including open source tools) include:
Falco: Created by Sysdig and now a CNCF project, Falco is a runtime security tool that monitors application behavior and detects malicious activity. It taps into the Linux kernel's system calls via eBPF (Extended Berkeley Packet Filter) or the sysdig kernel module to capture real-time event data, which it analyzes against a set of rules to identify anomalous activity. Upon detecting suspicious actions, it can trigger alerts, enabling rapid response.
Tetragon: Tetragon is an eBPF-based security observability and runtime enforcement tool. It works by leveraging eBPF to provide deep visibility into system and application behavior on Linux. Tetragon monitors process executions, file system I/O, network activity, and more, allowing for a granular inspection of system events and helping prevent and detect security breaches through real-time monitoring.
Aqua Tracee: Tracee is a runtime security and forensics tool from Aqua Security. like Tetragon and Flaco, it too utilizes eBPF technology to capture system-level events, focusing on behavioral analysis to track anomalies and indications of malicious activity. Tracee collects detailed information on commands, system calls, and kernel events, providing context that can be used for threat detection and forensic analysis in a security investigation.
Sysdig Secure: An open-source runtime security agent and forensics tool, Sysdig Secure uses a system call capture mechanism to collect data from the Linux kernel. It then analyzes the information to detect and prevent security incidents. It integrates with Kubernetes, providing runtime threat detection, incident response, and compliance across cloud-native application deployment.
Microsoft (Defender for Containers): A cloud-native solution to improve, monitor, and maintain the security of your containerized assets (Kubernetes clusters, Kubernetes nodes, Kubernetes workloads, container registries, container images, and more), and their applications, across multi-cloud and on-premises environments.
Most of these tools add layers of security to runtime environments by providing visibility into system behavior at a granular level, leveraging built-in Linux capabilities, and integrating with modern orchestration tools like Kubernetes to protect against internal and external threats in real time.
While some organizations may opt to include such additional wares in their growing security stacks, others are finding success in hardening their containerized environments through extensions of their existing defenses, focusing on more common and generalized detection, response, and workload protection approaches.
In fact, a number of mainstream Endpoint Detection and Response (EDR) and Cloud Workload Protection Platforms (CWPP) are getting in on container runtime security with solutions that enhance vulnerability management, compliance assurance, and threat detection tailored to the dynamic and scalable nature of cloud-native architectures. Today there are a number of EDR solutions focused on identifying and investigating suspicious activities on endpoints, perfect in a security context where almost every container is, in fact, an endpoint. It is important to note that an EDR is not appropriate for a cloud native environment, and you shouldn't take your detection and response tools' word for it when it comes to container runtime security.
Similarly, CWPPs provide a specialized security framework designed to protect workloads in public, private, and hybrid cloud environments, the kinds of spaces where most containers live.
In the realm of container runtime security, several legacy vendors continue to innovate and expand their offerings to address the evolving security challenges of containerized deployments. Some key players gaining traction in this area include:
Palo Alto Networks (Prisma Cloud): Recognized for their comprehensive cloud security suite, which includes advanced EDR and CWPP features tailored for containerized environments.
Trend Micro (Deep Security): Offers a blend of intrusion prevention, monitoring, and real-time protection for containers, making it a considerable force in the CWPP space.
McAfee (MVISION Cloud Native Application Protection Platform (CNAPP): Known for its robust security platform that integrates EDR and CWPP capabilities, McAfee tools are now being employed to help organizations defend against sophisticated threats targeting container workloads.
VMware (Carbon Black Container and EDR for Kubernetes): Offers continuous visibility, security, and compliance for the full lifecycle of containers and Kubernetes applications from development to production. From an EDR perspective, provides scalable, automated approach to detecting and responding to threats in real-time for cloud-native applications
RAD verified runtime fingerprints reduce the overhead and risk of runtime security
Today, runtime security solutions are notorious for high overhead and breaking your application, when policies are enforced. This is because they are generally deployed with high privileges, to be able to stop behavior that appears to be malicious in real-time, and this has a high cost in terms of CPU and overall risk.
RAD Security was recently released to change this paradigm, and the way runtime security is done in cloud native environments, with a standard for creating verified cloud native workload fingerprints. Using behavioral fingerprints would allow teams to detect and alert on the drift from good behavior, versus alerting on all the kinds of activity that look like bad behavior, installing with high privileges, and running with high overhead.
A free online cloud native workload fingerprint catalog is available, and teams can sign up for early access to enabling drift detection using the RAD standard in their own environments.
Sign up for early access to RAD verified cloud native workload fingerprints
Or learn more about RAD Security fingerprinting
Difference between RAD Security runtime capabilities and other runtime security tools
Inverting the model--from writing rules to identify suspected bad behavior, to instead flagging activity that deviates from the norm--greatly simplifies and improves detection capabilities. This exception-based security model means that novel attacks become immediately apparent, as they diverge from the expected behavior of the app. No new signatures need to be written, and there's no lag in detection. Immediately, we can also see that the fingerprint-based analysis can take into account not just the properties of the activity (which is the status quo for runtime activity today), but also context, relationships with other activity, and a stateful knowledge of what activity should be expected in relation to other activity.
RAD security method vs other container runtime security methods
Conclusion
Securing container environments, particularly during runtime, is a basic requirement in today's threat landscape. Updating container images, enforcing strict isolation mechanisms, adhering to the least privilege principles, and securing orchestration tools all play a decisive role in safeguarding critical apps, data, and the development and production ecosystems where the two interact.
With containers — as with the rest of the organization's digital IT systems — cybersecurity remains a moving target; it takes continuous attention, constant updates, and a dedicated commitment to best practices to keep it all up and running safely. The consequences of neglect at any stage of defense can be catastrophic.
Rest assured, though, that with diligence and proactive attention, security-minded orgs can significantly mitigate the risks associated with container runtime security and build resilient app infrastructures that both work well and work safely. For cybersecurity practitioners and developers alike, the message is clear—stay vigilant, stay updated, and always ensure that security is accounted for at every phase of containerized deployments.