September 14, 20269 min read

Datadog Agent High Memory Usage: Causes, Diagnostics, and Fixes

Sources:Datadog Docs - High CPU or Memory Consumption|Datadog Docs - Solve Memory Leaks with Profiling|Datadog Blog - Investigate memory leaks and OOMs with Datadog's guided workflow|DataDog/datadog-agent GitHub - Cluster Agent memory leak issue|DataDog/datadog-agent GitHub - ECS Fargate memory leak issue

Eric Bang

Founder & Cybersecurity Evangelist

core Datadog diagnostic commands for memory issues: datadog-agent status and datadog-agent flare --profile

seconds a flare --profile capture takes to collect profiling data, per Datadog's own documentation

7.67

Agent version that added an Agent Profiling check able to auto-generate a flare on a configurable memory threshold

seconds, a commonly recommended min_collection_interval floor for a noisy check contributing to memory growth

Reach 50,000+ security professionals

Sponsor a newsletter slot and put your brand in front of CISOs, engineers, and IT decision-makers.

info@decryptiondigest.com View Sponsorship Options

Summarize with:

ChatGPT

Claude

Google AI

The Datadog Agent is designed to run quietly in the background collecting metrics, logs, and traces, which is exactly why a memory problem with it tends to go unnoticed until it becomes an outage: a Kubernetes DaemonSet pod that keeps getting OOM-killed and restarted, a host-level Agent process that has crept from a few hundred megabytes to several gigabytes over a few weeks, or node memory pressure that Kubernetes resolves by evicting other workloads. None of this is usually a sign that the Agent itself is broadly broken. Datadog's own troubleshooting documentation and its public GitHub issue tracker both point to a small, identifiable set of causes: a specific check integration returning far more metrics or running far more instances than expected, tag cardinality that has grown unbounded, Autodiscovery patterns that are too broad for a dynamic container environment, or log and trace buffering that grows when the intake connection is slow or unreachable. This guide walks through those causes, the Agent's own commands for finding which one applies to you, concrete fixes for hosts and Kubernetes, how to confirm the fix holds under real sustained load rather than just after a restart clears the symptom temporarily, and when the right move is filing a support ticket with a memory profile instead of continuing to tune configuration blind.

Causes: What Actually Drives Agent Memory Growth

Datadog's own troubleshooting documentation for high CPU or memory consumption identifies integration behavior and runtime load as the primary drivers, and the Agent's public GitHub issue tracker shows that specific versions and specific components have shipped with genuine memory leaks that were later fixed. Treat these as the starting checklist rather than assuming every case is a mysterious leak:

An integration returning excessive metrics or running too many check instances

Datadog's documentation names this as a leading cause: a check configured against a target that returns thousands of metrics per run, or a check instance count that has grown alongside the environment (one instance per database, per queue, per service) without anyone revisiting whether all of them are still needed.

Unscoped Autodiscovery patterns in dynamic container environments

A broad wildcard Autodiscovery template that matches far more containers than intended causes check instance churn as containers are created and destroyed, particularly in an environment with frequent deploys or autoscaling, where each newly matched container spins up another check instance the Agent has to track.

High-cardinality tags inflating the metrics pipeline

Tags that include unbounded values (a raw user ID, a request ID, a pod name that changes on every restart) multiply the number of unique metric series the Agent has to buffer and forward, growing memory usage in proportion to cardinality rather than to actual event volume.

APM trace buffer growth under load

Under sustained high request volume, the Agent's trace buffering can grow if traces are being generated faster than they can be processed and forwarded, particularly when sampling is not configured to bound the volume the Agent has to hold in memory at once.

Log collection buffering when the intake is slow or unreachable

If the Agent's connection to Datadog's log intake is degraded or blocked (a network issue, a proxy misconfiguration, a firewall change), the Agent buffers logs waiting to forward them, and that buffer grows for as long as the intake stays unreachable and log volume keeps arriving.

A specific Agent version with a known, since-fixed leak

Datadog's own GitHub issue tracker documents concrete version-specific leaks reported by users, including a Cluster Agent v7.46.0 slow memory leak and an ECS Fargate memory reporting issue tied to Agent 7.48.1. Check your running Agent version against the changelog and open issues before assuming a leak is unique to your environment; it may already be a known, fixed problem in a newer release.

Subscribe to unlock Remediation & Mitigation steps

Free subscribers unlock full IOC lists, Sigma detection rules, remediation steps, and every daily briefing.

Diagnostics: Finding the Responsible Check or Component

Datadog ships diagnostic tooling directly in the Agent specifically so this doesn't have to be guesswork. Work through these before changing any configuration.

Start with datadog-agent status

Run `datadog-agent status` and review the Collector section, which lists every running check, its instance count, and its last run duration. An unexpectedly large instance count for one check, or a check whose instance count keeps climbing over successive checks of status, points directly at the responsible integration.

Profile a specific check's memory use directly

Datadog's documentation describes running `sudo -u dd-agent -- datadog-agent check <check_name> -m -t 10` to get memory profiling output for a single Python-based check, which isolates whether that specific integration is the source before you go further.

Capture a profiling flare

Run `sudo datadog-agent flare --profile 30` to collect a 30-second profile alongside the normal flare bundle. This is the artifact Datadog support will ask for if the issue needs to be escalated, so capturing it while the problem is actively occurring is more useful than capturing it after a restart has already cleared the symptom.

Use the Agent Profiling check on 7.67 and later for hard-to-reproduce spikes

Starting with Agent version 7.67, an Agent Profiling check can be configured to automatically generate a flare once memory or CPU crosses a configured threshold, which is the right tool for a leak that builds slowly over days rather than one you can catch by manually running a command at the right moment.

In Kubernetes, correlate with kubectl top pod and describe

Run `kubectl top pod` against the DaemonSet's pods to see actual memory usage trend over time relative to the configured limit, and `kubectl describe pod` on a recently OOM-killed pod to confirm it was in fact killed for exceeding its memory limit rather than for an unrelated node-level pressure event.

Check tag cardinality directly

Review which tags are attached to the highest-volume metrics coming from the affected host or pod, looking specifically for any tag whose value is effectively unique per event (an ID, a timestamp-derived value, a per-pod-restart identifier) rather than a bounded category.

Subscribe to unlock Remediation & Mitigation steps

Free subscribers unlock full IOC lists, Sigma detection rules, remediation steps, and every daily briefing.

Free daily briefing

Briefings like this, every morning before 9am.

Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.

Fixes: Reconfiguring Rather Than Just Restarting

A restart clears symptoms temporarily but does not address the underlying driver identified in diagnostics. Match the fix to what the diagnostics actually showed.

Increase the collection interval on a noisy check

For a check identified as returning excessive metrics per run, raise `min_collection_interval` in that integration's conf.yaml (commonly to 60 seconds or higher, well above the 10 to 15 second default for many checks) to reduce how often it runs and how much data it buffers between collection cycles.

Scope Autodiscovery templates to specific patterns

Replace a broad wildcard Autodiscovery match with a narrower pattern (specific image names, specific labels) so the Agent only spins up check instances for containers that genuinely need monitoring, cutting the instance churn that drives memory growth in a dynamic environment.

Apply tag cardinality limits

Exclude or transform any tag identified as effectively unbounded before it reaches the Agent's metrics pipeline, either at the application instrumentation level or through Agent-side tag exclusion configuration, so the number of unique series the Agent has to track stops scaling with request or event volume.

Adjust APM sampling to bound trace volume

Configure trace sampling rates so the volume of traces the Agent buffers under peak load stays within a predictable ceiling, rather than letting the buffer grow unbounded during a traffic spike.

Add log processing rules to reduce volume at the source

For log buffering caused by high volume compounding a slow intake connection, add exclusion or sampling rules for high-volume, low-value log sources so the buffer has less to hold onto even if the intake connection degrades again.

Set Kubernetes resource requests and limits deliberately, not as an afterthought

Size the DaemonSet's memory limit based on observed steady-state usage plus headroom for legitimate spikes, rather than an arbitrary default. A limit set too low turns an ordinary spike into a recurring OOM kill even after the underlying check or cardinality issue is fixed.

Upgrade to a patched Agent version

If diagnostics point to a version with a documented, since-fixed leak (verified against the Agent's release notes and GitHub issues rather than assumed), upgrading is the fix, not further configuration tuning around a bug that a newer release already resolved.

Subscribe to unlock Remediation & Mitigation steps

Free subscribers unlock full IOC lists, Sigma detection rules, remediation steps, and every daily briefing.

Validation: Confirming Memory Stabilizes Under Sustained Load

Memory usage that looks fine for the first hour after a restart or a config change has not been validated. Growth patterns caused by cardinality or buffering issues often take hours or days to reappear.

Watch the memory trend over days, not minutes

Chart Agent memory usage (via `kubectl top pod` history, host-level monitoring, or the Agent's own reported metrics) across at least several days after the fix, since a slow leak or a cardinality-driven growth pattern that took a week to become visible the first time will take a comparable amount of time to reappear if the fix didn't address the actual cause.

Re-run datadog-agent status periodically to confirm instance counts are stable

Confirm the previously offending check's instance count in the Collector section stays flat over time after an Autodiscovery scoping fix, rather than slowly climbing again as new containers get matched by a pattern that is still broader than intended.

Validate under peak traffic, not average traffic

For APM buffer and log buffering fixes specifically, confirm behavior during the highest-traffic period the environment actually sees (a batch job window, a traffic peak), since a sampling or buffering fix validated only against average load can still fail during the exact conditions that caused the original problem.

Confirm OOM kill events actually stop, not just become less frequent

In Kubernetes, check pod restart counts and events over the following week. A reduction in OOM kill frequency without full resolution usually means the fix addressed part of the cause (for example, the Autodiscovery scoping) but not all of it (for example, cardinality still growing more slowly).

Subscribe to unlock Remediation & Mitigation steps

Free subscribers unlock full IOC lists, Sigma detection rules, remediation steps, and every daily briefing.

Failure Cases: What Still Leaks Despite These Fixes

Configuration tuning solves the common cases Datadog documents. It does not solve every case, and it's worth naming where it falls short.

A genuine unpatched Agent-version leak with no available fix yet

If diagnostics point to memory growth that correlates with a specific Agent version and matches an open (not yet fixed) issue on Datadog's public tracker, no amount of check or Autodiscovery tuning resolves a leak inside the Agent binary itself; the fix has to come from Datadog as a patched release.

Cardinality growth driven by application-side changes outside the Agent's control

If a development team adds a new high-cardinality tag to application instrumentation faster than the platform team can catch and exclude it, tag limits become a reactive patch rather than a durable fix; the actual fix requires a tagging standard enforced at the instrumentation level, not just Agent-side filtering.

Sustained load that genuinely exceeds the provisioned resource limit

A workload whose legitimate steady-state telemetry volume has grown past what the DaemonSet's memory limit was originally sized for will keep hitting that ceiling regardless of check tuning, because the limit itself, not a leak, is now the binding constraint.

Log or trace buffering during a genuinely extended intake outage

If Datadog's intake is unreachable for an extended period due to a network path issue outside the Agent's control, buffering will grow for as long as that outage lasts no matter how well log or trace volume is tuned; the fix there is resolving the connectivity issue, not further reducing telemetry volume.

Subscribe to unlock Remediation & Mitigation steps

Free subscribers unlock full IOC lists, Sigma detection rules, remediation steps, and every daily briefing.

Escalation Criteria: When to File a Datadog Support Ticket

Escalate to Datadog support with a memory profile attached once any of the following is true, rather than continuing to tune configuration:

Diagnostics rule out every documented cause and growth continues

If `datadog-agent status` shows stable check instance counts, cardinality has been reviewed and bounded, Autodiscovery is scoped narrowly, and the Agent is on a current version with no matching known issue, and memory still grows steadily, that pattern warrants a support case with the profiling data attached rather than more speculative configuration changes.

The growth pattern matches a suspected but unconfirmed Agent-level leak

Capture a `flare --profile 30` (or let the Agent Profiling check on 7.67+ auto-generate one at a memory threshold) while the issue is actively occurring, and file it with Datadog support. A profile captured live is far more useful to their engineering team than a description of symptoms after a restart already cleared them.

The same symptom matches an open GitHub issue on DataDog/datadog-agent

If your version and symptom pattern match an already-reported issue on the public tracker, reference it directly in your support case, since that gives Datadog a known thread to correlate your report against rather than starting the investigation from scratch.

OOM kills are causing telemetry gaps that affect incident response

If repeated OOM kills on monitoring infrastructure are creating blind spots during actual incidents, that operational impact justifies treating this as a priority support case rather than a background tuning task, since the monitoring gap compounds the original problem.

Subscribe to unlock Remediation & Mitigation steps

Free subscribers unlock full IOC lists, Sigma detection rules, remediation steps, and every daily briefing.

Related Monitoring and Agent Resource Issues

Agent memory growth is one instance of a broader pattern in security and observability tooling: an agent or collector consuming more host resources than expected. Our guide to CrowdStrike Falcon sensor high CPU usage covers the same class of problem for an EDR sensor rather than a monitoring agent, and the diagnostic discipline (isolate the responsible component before changing configuration, validate under sustained load rather than after a restart) carries over directly. If Agent memory pressure is part of a broader conversation about telemetry volume and cost, our cloud log ingestion cost optimization guide covers reducing volume at the source, which often reduces both cost and the buffering pressure that drives memory growth. And if the growth pattern you're chasing turns out to be API-side rather than Agent-side, our VirusTotal API 429 troubleshooting guide covers a related but distinct failure mode: a pipeline hitting an external rate limit rather than a local resource ceiling.

The bottom line

Datadog Agent high or growing memory usage almost always traces back to one of a small set of documented causes: a specific check integration returning too many metrics or running too many instances, an unscoped Autodiscovery pattern churning through containers, unbounded tag cardinality, or buffering that grows when APM, log, or trace volume outpaces what can be forwarded. Use the Agent's own status, per-check profiling, and flare --profile commands to identify the actual responsible component before changing any configuration, since a restart or a resource limit bump treats the symptom without addressing the cause. Validate the fix over days of real sustained load, not minutes after a restart. When diagnostics rule out every documented cause and growth continues, or the pattern matches a known but unresolved issue on Datadog's own GitHub tracker, capture a live memory profile with flare --profile and escalate to Datadog support rather than continuing to tune configuration against a problem that may be inside the Agent binary itself.

Frequently asked questions

What is the most common cause of Datadog Agent high memory usage?

Datadog's own troubleshooting documentation names a specific integration returning too many metrics per run, or running an excessive number of check instances, as a leading cause. Unscoped Autodiscovery patterns in dynamic container environments and unbounded tag cardinality are the other most common documented causes.

How do I find which Datadog check is causing high memory usage?

Run `datadog-agent status` and review the Collector section for an unusually large or growing instance count, then profile a suspected check directly with `datadog-agent check <check_name> -m -t 10` to confirm its memory footprint before changing configuration.

Does restarting the Datadog Agent fix high memory usage?

A restart clears the symptom temporarily but does not address the underlying cause. If the driver is an unscoped Autodiscovery pattern, unbounded tag cardinality, or a version-specific leak, memory usage will climb again on the same trajectory after the restart unless the actual cause is fixed.

Why does the Datadog Agent keep getting OOM-killed in Kubernetes?

Usually because the DaemonSet's memory limit is set below what the Agent's actual usage grows to under a documented cause (a noisy check, cardinality growth, or Autodiscovery churn), rather than because of a random or unfixable issue. Use `kubectl top pod` and `kubectl describe pod` to confirm the OOM kill is memory-limit related, then fix the underlying driver and size the limit to match validated steady-state usage plus headroom.

What is flare --profile used for in Datadog Agent troubleshooting?

Running `datadog-agent flare --profile 30` captures a 30-second memory and CPU profile alongside the standard flare bundle. It is the diagnostic artifact Datadog support needs to investigate a suspected Agent-level leak, and it is most useful when captured while the problem is actively occurring rather than after a restart has already cleared it.

When should I escalate Datadog Agent memory issues to Datadog support?

Escalate once diagnostics have ruled out every documented cause (stable check instance counts, bounded cardinality, scoped Autodiscovery, a current Agent version) and memory still grows steadily, or once the symptom pattern matches an open issue on Datadog's public GitHub tracker. Capture a live flare --profile before filing the case.

Sources & references

Summarize with:

ChatGPT

Claude

Google AI

Free resources

Free download

Critical CVE Reference Card 2025–2026

25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.

Free download

Ransomware Incident Response Playbook

Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.

Follow Decryption Digest

Daily briefings + Google Top Stories in one step.

Enter your email and we'll subscribe you to the daily briefing and add Decryption Digest to your Google Search feed at the same time.

Author

Eric BangCISSP

Founder & Cybersecurity Evangelist, Decryption Digest

Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals every morning.

View profile →LinkedIn

Back to all briefings

Subscribe for Updates

datadog agent memory leak high memory usage kubernetes oom kill datadog troubleshooting observability agent monitoring agent performance daemonset resource limits

Datadog Agent High Memory Usage: Causes, Diagnostics, and Fixes

Causes: What Actually Drives Agent Memory Growth

Diagnostics: Finding the Responsible Check or Component

Fixes: Reconfiguring Rather Than Just Restarting

Validation: Confirming Memory Stabilizes Under Sustained Load

Failure Cases: What Still Leaks Despite These Fixes

Escalation Criteria: When to File a Datadog Support Ticket

Related Monitoring and Agent Resource Issues

The bottom line

Frequently asked questions

What is the most common cause of Datadog Agent high memory usage?

How do I find which Datadog check is causing high memory usage?

Does restarting the Datadog Agent fix high memory usage?

Why does the Datadog Agent keep getting OOM-killed in Kubernetes?

What is flare --profile used for in Datadog Agent troubleshooting?

When should I escalate Datadog Agent memory issues to Datadog support?

Sources & references

Critical CVE Reference Card 2025–2026

Ransomware Incident Response Playbook

Daily briefings + Google Top Stories in one step.

Get briefings like this every morning