SIEM Log Ingestion Cost Optimization: Cut Your Bill Without Creating Blind Spots
Retool's new app builder is where AI-generated code ships safely
Building apps with AI is easy. Getting them to production safely is another story.
SIEM log costs scale quadratically with ambition: every new cloud workload, every additional application, and every 'let's collect everything just in case' decision compounds monthly. Organizations that started with a reasonable SIEM budget find themselves paying 5-10x the original estimate within two years because no one audited log sources by security value, and the default configuration of most log shippers sends everything.
The optimization path is a structured audit, not arbitrary cuts. The goal is to identify log sources where the security value does not justify the cost, implement tiering or filtering for those sources, and validate that you have not created detection gaps in the process. Organizations that do this correctly often find 40-60% cost reduction with no meaningful loss in detection capability.
The log value audit: measure before you cut
Before reducing any log collection, you need hard numbers on both cost and security value for every source. This section walks through building a log source cost-value matrix using queries available in Splunk, Microsoft Sentinel, and Elastic, and then identifying filterable event types within sources you want to keep. The matrix drives data-backed decisions rather than gut-feel cuts, and reveals which specific event IDs or request patterns within a valuable log source are responsible for most of the volume without contributing any detection signal.
Build the log source cost-value matrix
For your top 20 log sources by volume: create a spreadsheet with columns: log source name, GB/day, estimated monthly cost (volume x your SIEM's per-GB rate), detection rules that query this source (count them), number of true positive alerts generated from this source in the past 90 days, and whether this source is required by a specific compliance framework and for what retention period. Calculate cost per true positive alert for each source: a source costing $500/month that generates 20 true positive alerts per month has a $25 cost per detection. A source costing $3,000/month that generates 0 true positive alerts in 90 days is a strong candidate for cold tiering or filtering. This matrix drives data-backed conversations about log reduction priorities.
Identify filterable event types within valuable log sources
Some high-value log sources have internal noise that can be filtered without losing security signal. Windows Security Event Logs: filter Event ID 4634 (account logoff) at the shipper — logoff events rarely contribute to detection but can represent 20-30% of Windows event volume. Keep 4624 (logon success) and 4625 (logon failure). Firewall logs: filter ALLOW events for established connections to known-good internal infrastructure (server-to-server communication within your data center) — keep all DENY events and all events involving external IPs. Web server access logs: filter health check requests (known monitoring IP to /health or /ping endpoints) and static asset requests for clearly non-security-relevant extensions (.css, .ico, .woff). Each filtered event type reduces volume without eliminating a detection source.
Implementation: tiering and filtering by SIEM platform
Once the cost-value matrix identifies which sources belong in cold or warm storage, the next step is implementing tiering and pre-ingestion filtering using the specific tools available in your SIEM. This section covers Microsoft Sentinel's Basic Logs tier and archive configuration, and Splunk SmartStore with Heavy Forwarder null-queue filtering. Both approaches let you retain the log data for compliance or investigation purposes at a fraction of hot-tier cost, while keeping only detection-relevant sources in the real-time analytics tier. Apply changes incrementally, one source at a time, and run detection validation tests after each change before moving to the next source.
Microsoft Sentinel: use Basic Logs and archive tier
Migrate high-volume, low-detection-value tables to Basic Logs: in the Log Analytics workspace, go to Tables, select the table (e.g., AzureDiagnostics for non-security-critical services), and change the Table Plan to Basic. Verify that no active Analytic Rules reference this table before changing — Basic Logs tables cannot be used in scheduled query rules. For compliance archive: configure long-term retention on tables via the Retention setting — data past the interactive retention period moves to archive storage at a fraction of the analytics cost. Query archive-tier data using search jobs when needed for investigations, at a per-GB search cost rather than the per-GB retention cost.
Splunk: implement SmartStore and index tiering
Splunk SmartStore moves indexed data to S3-compatible object storage (remote storage) while keeping frequently-accessed buckets in local SSD cache. Configure SmartStore for high-volume, low-query-frequency indexes: add [index-name] to the indexes.conf SmartStore stanza pointing to your S3 bucket. SmartStore transparently moves data to S3 after the cache age threshold and retrieves it on demand for searches — the storage cost drops from SSD pricing to S3 object storage pricing. For event filtering before ingestion: use Splunk Heavy Forwarders with transforms.conf routing rules to null-queue (drop) matching events before they enter the indexer. The null queue does not count dropped events against your license.
Briefings like this, every morning before 9am.
Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.
The bottom line
SIEM log cost optimization is a revenue-neutral security project: the cost savings pay for additional security investments or offset SIEM cost growth from a growing environment. The methodology is log source audit by cost and security value, followed by tiering decisions that move high-volume, low-security-value data to cheaper storage tiers, followed by pre-ingestion filtering of events with no security signal. Validate after every change that detection rules still have data sources and that Atomic Red Team tests still fire alerts. The target state is a SIEM where every log source is either: required by a specific detection rule and justified by that rule's true positive rate, or required by a specific compliance framework at a documented retention period. Everything else is a candidate for cold archiving or pre-ingestion filtering.
Frequently asked questions
How do I identify which log sources are driving the most SIEM cost?
Most SIEM platforms provide log volume breakdown by source. In Microsoft Sentinel: run KQL query Usage | where TimeGenerated >= ago(30d) | summarize TotalGB = sum(Quantity)/1024 by DataType | sort by TotalGB desc to see log volume by data type. In Splunk: index=_internal source=*license_usage.log | stats sum(b) as bytes by pool | sort -bytes. In Elastic: use the Index Management page or query .monitoring-es* for index sizes. After identifying the top 5-10 volume sources by GB: estimate the security value of each. Routine web access logs from a public-facing CDN: low security value (IP-based detection is done more effectively at the WAF), high volume. Windows Security Event Logs from domain controllers: high security value, should be retained in hot tier. This cost-per-security-value analysis drives your tiering decisions.
Which log sources are typically safe to filter or tier to cold storage?
Lower value sources typically safe to move to cold tier (compliance archive, not real-time analysis): CDN access logs (web traffic is better analyzed at the WAF or network edge), application debug and trace logs (high volume, no security signal — security-relevant application events should be logged separately at WARN or ERROR level), load balancer access logs (health check traffic constitutes 50-90% of these logs and carries zero security value — filter health check events before ingestion), and verbose cloud service logs for services with no user data (S3 server-side encryption logs, CloudFront access logs for public static assets). Never move to cold tier: authentication logs (AD, Okta, Entra ID sign-in logs), DNS query logs (critical for C2 detection), EDR telemetry, network flow data (even compressed NetFlow), and security control logs (WAF, IDS/IPS blocks).
How do I implement pre-ingestion filtering to reduce log volume at the shipper level?
Pre-ingestion filtering drops events at the log collection agent before they are sent to the SIEM, eliminating them from the billable ingestion quota. Implementation by agent: (1) Logstash: use a conditional filter block: if [message] =~ /health.*check/ { drop { } } to drop health check events. (2) Fluentd: use the grep filter plugin to drop events matching a pattern. (3) Vector: use a filter transform with a VRL (Vector Remap Language) condition. (4) AWS Kinesis Firehose: use data transformation via Lambda to filter events before delivery to the SIEM. (5) Elastic Agent: use ingest pipelines with drop processors. Before implementing filters: document which event patterns you are dropping and why, and verify by replaying known-malicious events against the filter logic to confirm security-relevant events are not inadvertently dropped. Save filter configuration to version control so changes are tracked.
What is the Microsoft Sentinel auxiliary log tier and when should I use it?
Microsoft Sentinel's Basic Logs (auxiliary tier) ingests data at approximately 1/8 the cost of the Analytics tier, with the tradeoff that Basic Logs data cannot be used in real-time detection rules (Analytic Rules, Scheduled query rules). Basic Logs can be used for manual investigation queries (Log Analytics query editor) and are subject to an 8-day retention period by default before archiving. Use cases for Basic Logs: verbose web server access logs you need for forensic investigation but not real-time detection (the WAF alerts on attacks; the access logs provide context for investigation), application performance logs that may be relevant in a breach investigation but are not security-controlled events, and cloud resource operation logs for services with no direct security detection use case. Keep in Analytics tier: all log sources that feed real-time detection rules — authentication, process execution, DNS, network connections.
How do I validate that my log reduction changes did not eliminate critical detection coverage?
Validation sequence after implementing log tiering or filtering: (1) Map your SIEM detection rules to their data sources: for each active detection rule, identify which log source(s) it queries. If any rule queries a log source you just moved to cold tier or filtered, the rule no longer functions — either restore that log source to hot tier or rebuild the detection using an alternate log source. (2) Run detection validation tests: use Atomic Red Team to simulate the attack techniques your rules are designed to detect and confirm alerts still fire after the log changes. (3) Check for silent rule failures: some SIEMs do not alert when a rule queries a non-existent or empty data source — review rules that have had zero alerts since your log changes (they may have been silently broken). (4) Review detection coverage dashboard: if you use a MITRE ATT&CK coverage visualization, verify that technique coverage did not decrease after the optimization.
How do I optimize AWS CloudWatch Logs costs specifically?
CloudWatch Logs costs have three components: ingestion (per GB), storage (per GB-month), and query (per GB scanned by CloudWatch Logs Insights). Optimization by component: (1) Ingestion: set log retention periods per log group (default is never expire — set 30-90 day retention for non-compliance logs, 1 year for security logs). Export logs to S3 using subscription filters and a Lambda function or CloudWatch Logs subscription before the retention period ends (S3 is 10-100x cheaper than CloudWatch storage). (2) Storage: enable S3 Intelligent-Tiering for exported logs (auto-moves infrequently accessed logs to cheaper tiers). (3) Queries: use log groups and timeframe filters in CloudWatch Logs Insights queries — scanning 1 TB of logs is expensive; scanning a specific 1-hour window in a specific log group is cheap. Move from CloudWatch to OpenSearch or Amazon Security Lake for security-relevant logs that require frequent querying.
How do I get stakeholder agreement to reduce log collection that was previously required for compliance?
Many log sources that are over-retained are kept because someone once said 'we need this for compliance' without specifying which regulation and what retention period it requires. Reframe the conversation with specific evidence: (1) Present actual cost data: '$2,400/month for 6 months of CDN access logs that have never been queried in any security investigation.' (2) Identify the specific compliance requirement and retention period: PCI DSS 3.4 requires 12 months of audit log retention, but CDN access logs are not audit logs — they are application performance data. Match each log source to the specific compliance requirement or security use case that justifies its cost. (3) Propose a tiered solution: retain all logs, but move low-value high-volume data to archive storage where it remains available for compliance purposes at 10% of the current cost. This satisfies the auditor while reducing operational cost.
Sources & references
Free resources
Critical CVE Reference Card 2025–2026
25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.
Ransomware Incident Response Playbook
Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.
Get threat intel before your inbox does.
50,000+ security professionals read Decryption Digest for early warnings on zero-days, ransomware, and nation-state campaigns. Free, daily, no spam.
Unsubscribe anytime. We never sell your data.
Founder & Cybersecurity Evangelist, Decryption Digest
Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals every morning.
Win a $2,495 Black Hat pass.
Full-access to Black Hat USA 2026 in Las Vegas. Subscribe free to enter.