Falco Kubernetes Runtime Security: eBPF Driver, Custom Rules, and SIEM Alert Integration
Retool's new app builder is where AI-generated code ships safely
Building apps with AI is easy. Getting them to production safely is another story.
A web application vulnerability was exploited in a production Kubernetes pod running an older Node.js version. The attacker gained a foothold in the container and began exploring: ran whoami, checked /etc/passwd, tried curl to their C2 server, and attempted to read the mounted service account token to pivot to the Kubernetes API. Without Falco, this activity was invisible — the container was still running, passing health checks, serving traffic, and generating no errors in application logs. With Falco deployed, the Terminal shell in container rule fired on the first whoami, the Read sensitive file rule fired when /etc/shadow was accessed, and the outbound network connection rule fired on the curl attempt. The entire attack chain was logged and alerted within seconds of the first command.
Falco's value is precisely this: it monitors what the container is doing at the syscall level, not just what is deployed. An attacker inside a container looks very different from a legitimate web application process, and that behavioral difference is detectable. The operational challenge is tuning the default rules so that legitimate operational activities (exec into pods for debugging, configuration management that writes to /etc) do not generate noise that trains security teams to ignore Falco alerts.
Installation and baseline: eBPF driver with Sidekick for alert routing
Falco's installation involves two components that are equally important: the Falco DaemonSet that runs the eBPF probe on every node, and Falco Sidekick that receives alerts from Falco and routes them to the right destinations. The DaemonSet without Sidekick logs alerts to stdout where they are visible with kubectl logs but not actionable in a security operations workflow. Sidekick is what connects Falco's detections to Slack for immediate visibility, PagerDuty for on-call escalation, and Elasticsearch or a SIEM for long-term retention and correlation.
Measure Falco CPU overhead before production deployment using a load test on a representative node
Benchmark Falco's CPU overhead on a node representative of your production workloads before deploying cluster-wide, since the overhead is workload-dependent: a node running hundreds of short-lived batch processes generates more syscall events than a node running long-lived stateful services. Run the load test first without Falco (record baseline CPU usage), then with Falco deployed and all default rules enabled (record Falco overhead). In real production deployments, Falco with the eBPF driver adds 1-5% CPU overhead on nodes running mixed workloads — the overhead is higher on nodes running many short-lived processes and lower on nodes running long-lived services. If overhead exceeds 5%, tune the most expensive rules by restricting their scope with container.id != host and specific container.image.repository conditions that exclude high-volume system containers like the CNI plugin and monitoring agents.
Configure Falco's buffered output mode for high-throughput nodes to prevent dropped events
Enable Falco's buffered output mode in production by setting buffered_outputs: true in the falco.yaml configuration, which batches alerts before sending to Sidekick rather than sending each alert individually. On nodes with high syscall event rates (build nodes, CI/CD nodes running many container processes), unbuffered output can cause Falco to drop events when the alert output rate exceeds the Sidekick ingestion rate. Monitor dropped events in the Falco Sidekick web UI statistics page which shows the drop count per output destination — any nonzero drop count indicates the alert pipeline is saturated. Set the output_timeout to 2000ms and increase the falco_engine threads to 2 or 4 on high-CPU nodes to give Falco more rule evaluation capacity without increasing the per-event processing latency that causes drops.
Rule authoring: custom detections for application-specific threats
Falco's default rule set covers the most common container attack patterns but cannot cover the application-specific behaviors that deviate from the container's expected function. A rules engineer who knows what a container is supposed to do can write extremely high-fidelity rules that fire only on genuine threats for that specific application — rules that a generic default rule set could never match because they require knowledge of the container's normal behavior. These custom rules are often the highest-signal detections in the entire Falco deployment.
Write application-specific rules using container labels to scope detection to named services
Use Kubernetes pod labels exposed as Falco fields (container.label.app, container.label.tier, container.label.environment) to scope custom rules to specific services rather than applying them cluster-wide. A rule that detects database command execution in web-tier containers uses container.label.tier = web AND proc.name in (mysql, psql, mongo, redis-cli) as the condition — the same process execution in a database management container is expected. A rule detecting network scanning tools (nmap, masscan, nc) in any container uses spawned_process AND proc.name in (nmap, masscan, nc, ncat) with no container scope restriction since no legitimate container should run a port scanner. Combine label-scoped rules with event type filters: a rule using evt.type = openat AND fd.name contains /etc/kubernetes/ detects access to Kubernetes credential files that should never be accessed by application processes.
Use Falco macros and lists to maintain exception sets that can be updated independently of rule logic
Define Falco macros and lists for exception sets that change frequently — tool names, image repositories, authorized users — so that updating an exception does not require changing the rule logic itself. Create a list allowed_admin_images containing the image repository prefixes of containers that legitimately use privileged capabilities, then reference this list in rules: NOT container.image.repository in (allowed_admin_images). When a new authorized privileged container is deployed, add its image to the list in the ConfigMap, redeploy the rules, and the exception takes effect without reviewing or modifying any rule conditions. This separation of exception data from rule logic makes the exception management process auditable — a diff of the ConfigMap shows exactly which containers were added to or removed from exception lists, rather than burying exception logic inside complex rule conditions.
Briefings like this, every morning before 9am.
Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.
The bottom line
Falco provides kernel-level runtime security visibility that admission controllers and image scanners cannot: it sees what a container is doing inside a running pod, not just what was deployed. Install with the eBPF driver for compatibility and the DaemonSet deployment model for complete node coverage. Deploy Falco Sidekick alongside Falco to route CRITICAL alerts to PagerDuty, WARNING alerts to Slack, and all alerts to the SIEM for retention. Tune default rules with local overrides rather than disabling them to preserve coverage while eliminating environment-specific false positives. Write application-specific rules scoped to container labels to detect behavioral anomalies that generic rules miss. Enable drift detection for the highest-confidence indicator of container compromise — any new binary executed at runtime is a near-zero false positive signal that warrants immediate investigation. Falco's 1-5% CPU overhead is the cost of genuine runtime visibility; the alternative is container compromise that remains invisible until the attacker pivots to infrastructure.
Frequently asked questions
How do I install Falco on Kubernetes with the eBPF driver?
Install Falco on Kubernetes using the official Helm chart with the eBPF driver enabled for modern kernel compatibility. Add the Falco Helm repository with helm repo add falcosecurity https://falcosecurity.github.io/charts then install with helm install falco falcosecurity/falco --namespace falco --create-namespace --set driver.kind=ebpf --set falcosidekick.enabled=true --set falcosidekick.webui.enabled=true. The eBPF driver requires Linux kernel 4.14+ and loads a BPF program rather than a kernel module, avoiding the need for kernel module signing on restricted nodes. Verify the installation with kubectl get pods -n falco — all DaemonSet pods should reach Running state within 2-3 minutes. Check that the eBPF probe loaded successfully with kubectl logs -n falco daemonset/falco | grep 'eBPF' which should show probe loaded successfully. On GKE or EKS, the eBPF probe typically requires no additional node configuration; on AKS, verify that the kernel version is 4.14+ before deploying. Test rule detection with kubectl exec -it pod-name -- bash to spawn a shell in a container and verify Falco generates a Terminal shell in container alert.
How do I write custom Falco rules to detect application-specific threats?
Write custom Falco rules in falco_rules.local.yaml (mounted as a ConfigMap in the Helm deployment) using the rule, condition, desc, output, and priority keywords. A rule detecting curl or wget execution inside web application containers (which should never make outbound HTTP calls) uses: rule: outbound-download-in-web-container, condition: spawned_process AND proc.name in (curl, wget, fetch) AND container.label.app = web-server, output: Download tool executed in web container (proc=%proc.name parent=%proc.pname container=%container.name image=%container.image.repository), priority: WARNING. The condition uses Falco's macros (spawned_process is a built-in macro for process creation events) and Falco fields (proc.name, container.label.app) to narrow the detection to specific containers. Test the rule by exec-ing into the target container and running curl http://example.com — the alert should fire within one second. Add exceptions using list keywords: define a list allowed_web_tools containing the process names that are legitimately used by the web container and add NOT proc.name in (allowed_web_tools) to the condition to prevent false positives.
How do I tune Falco rules to reduce false positive alert volume?
Tune Falco false positives by overriding the default rules for your environment rather than disabling them entirely, using Falco's rule override mechanism. To modify a default rule, add a rule block in falco_rules.local.yaml with the same rule name and append: true to extend the condition with additional exclusions. For the Terminal shell in container rule that fires on every exec into a pod (legitimate in dev/test environments), append: AND NOT container.label.environment = development to limit firing to production containers. For the Write below etc rule that fires on configuration management tools, append: AND NOT proc.name in (chef-client, puppet, ansible-playbook). Deploy the local rules file as a Kubernetes ConfigMap and mount it in the Falco DaemonSet: the helm chart supports falco.rules_file configuration that specifies additional rule files to load. Monitor alert volume per rule using Falco Sidekick's built-in web UI (available at the sidekick-ui service port 2802) which shows alert counts by rule, priority, and container, identifying which rules generate the most noise relative to their security value.
How do I configure Falco Sidekick to route alerts to Slack and PagerDuty?
Configure Falco Sidekick for multi-destination alert routing during Helm installation by passing configuration values for each output target. For Slack, set falcosidekick.config.slack.webhookurl to the Slack webhook URL and falcosidekick.config.slack.minimumpriority to WARNING to send all warnings and above to Slack. For PagerDuty, set falcosidekick.config.pagerduty.routingkey to the Events API v2 routing key and falcosidekick.config.pagerduty.minimumpriority to CRITICAL to page only on critical detections like privilege escalation or container escape indicators. For SIEM integration, set falcosidekick.config.elasticsearch.hostport to the Elasticsearch endpoint and falcosidekick.config.elasticsearch.index to falco-alerts to forward all alerts regardless of priority for retention and correlation. Pass all secrets (webhook URLs, API keys) as Kubernetes Secrets and reference them in the Helm values using falcosidekick.config.slack.webhookurlsecret: secret-name rather than placing them in plain-text Helm values files. Verify routing by checking the Sidekick web UI's health page at /healthz and the output statistics at /stats which shows alert counts per output destination.
How do I detect container drift with Falco?
Detect container drift — new binaries or libraries executed inside a running container that were not present in the original image — using Falco's built-in drift detection capability available from Falco 0.35+ with the falcoctl artifact install falco-rules command loading the drift rules. Enable the Container Drift Detected rule in your rules configuration, which fires when a process is executed from a file that was created or modified after the container started (detectable via the container start time and file creation time fields in the Falco event). In environments that need drift detection without the Falco 0.35+ drift plugin, write a custom rule using the condition: spawned_process AND NOT proc.is_container_healthcheck AND container.id != host AND fd.name startswith /proc indicating process execution from paths that were not in the read-only image layer. Drift detection is one of Falco's highest-fidelity signals because legitimate containerized applications never execute new binaries at runtime — any new binary execution indicates either a misunderstood deployment pattern or an active compromise where an attacker dropped and executed a tool inside the container.
What are the most important Falco default rules to enable for Kubernetes security?
The highest-value Falco default rules for Kubernetes runtime security in order of detection priority: Terminal shell in container fires when an interactive shell is spawned inside any container, which should never happen in production containers that are not debug targets — any shell access indicates either an attacker using an exploit or a developer bypassing normal deployment procedures. Detect privilege escalation using ptrace fires when a process uses ptrace system calls that are used by both debuggers and memory-reading malware. Write below binary dir fires when any file is created in /bin, /sbin, /usr/bin indicating a binary was dropped into the container at runtime (container drift). Read sensitive file trusted after startup fires when files like /etc/shadow, /etc/sudoers, or SSH private keys are read by non-privileged processes outside the container startup period. Contact K8S API Server from Container fires when a process inside a container makes a direct call to the Kubernetes API server, which indicates either a pod with excessive permissions being exploited or a malicious container trying to enumerate or modify cluster resources. Enable these five rules at priority WARNING or higher and route them to on-call alerting as the baseline container runtime security detection layer.
How does Falco compare to cloud-native runtime security tools like GCP Container Threat Detection?
Falco and cloud-native tools like GCP Container Threat Detection (CTD) or AWS GuardDuty EKS Runtime Monitoring serve complementary purposes with different deployment models and detection coverage. Falco is infrastructure-agnostic, deploys to any Kubernetes cluster (on-premises, any cloud, bare metal), and gives full control over rule customization, rule priorities, and alert routing — critical for organizations with specific detection requirements or multi-cloud environments. Cloud-native tools like CTD integrate with the cloud provider's security platform automatically without requiring a separate DaemonSet, update their detection rules without operator action, and correlate container runtime events with cloud API events and network flows that Falco cannot see. For GKE environments using SCC Premium, CTD provides baseline runtime detection as an additional layer alongside Falco rather than a replacement — CTD catches container escape and reverse shell events that Falco rules may not cover, while Falco catches application-specific behavioral anomalies that CTD's generic rules will not detect. For non-GCP environments, Falco is the primary runtime security tool; for GCP environments with SCC Premium, run both.
Sources & references
Free resources
Critical CVE Reference Card 2025–2026
25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.
Ransomware Incident Response Playbook
Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.
Get threat intel before your inbox does.
50,000+ security professionals read Decryption Digest for early warnings on zero-days, ransomware, and nation-state campaigns. Free, daily, no spam.
Unsubscribe anytime. We never sell your data.
Founder & Cybersecurity Evangelist, Decryption Digest
Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals every morning.
Win a $2,495 Black Hat pass.
Full-access to Black Hat USA 2026 in Las Vegas. Subscribe free to enter.