Multi-Cloud Logging Strategy: Controlling SIEM Costs Without Detection Gaps

CloudTrail management events are the highest-signal AWS log source; ingest all of them into the SIEM, organization-wide, with a 12 month minimum retention. Data events (S3 object-level, Lambda invocations, DynamoDB) are 10 to 100 times the volume and should be selective: enable S3 data events only on buckets tagged sensitive=true or in production, enable Lambda data events only for functions with IAM roles granting cross-account or sensitive permissions. Use CloudTrail Lake for the long-tail data events at $2.50/GB ingested but $0.005/GB scanned, which is far cheaper than SIEM hot storage for events you query monthly rather than daily. AWS Config is low volume and high signal for resource configuration drift; ingest the configuration item snapshots and rule compliance events. GuardDuty findings are pre-correlated detections and should go straight to the SIEM as alerts, not as raw logs. VPC Flow Logs are where most teams hemorrhage budget: a busy production VPC can emit 500 GB per day. Send VPC Flow Logs to S3 (not directly to SIEM), use Athena or a Kinesis Firehose with a Lambda transform to filter to only REJECT records plus ACCEPT records to/from external IPs, and only ingest the filtered subset. Keep the raw logs in S3 with a 90 day lifecycle to Glacier for incident response.

Azure diagnostic settings are configured per-resource and per-category, which means an Azure environment with 5000 resources has roughly 25,000 logging decisions to make. Use Azure Policy to enforce diagnostic settings by resource type so the decision is made once. The non-negotiable categories: Azure Activity Log (all subscriptions, all categories, 100% to Sentinel; this is your CloudTrail equivalent), Entra ID Sign-in Logs and Audit Logs (these are not in Activity Log, configure under Entra ID > Diagnostic settings), Microsoft 365 audit log via the Office 365 connector (UAL), Key Vault AuditEvent, Storage Account StorageRead/StorageWrite for sensitive accounts only, SQL Database SQLSecurityAuditEvents. Network Security Group flow logs and Azure Firewall logs are the volume problem; route them to a Log Analytics workspace in the Basic Logs tier ($0.65/GB ingested, 8 day retention, limited query) and only summarize aggregated denials into the analytics workspace. For Defender for Cloud, alerts should go to Sentinel; the underlying recommendations and assessments should stay in Defender and be queried via the API rather than ingested. Use Sentinel's commitment tiers (100 GB/day, 300 GB/day, etc.) for a 20 to 50% discount once volume is stable.

GCP Cloud Audit Logs split into four streams: Admin Activity (always on, free), System Event (always on, free), Data Access (off by default, billable), and Policy Denied (off by default, billable). Admin Activity is the equivalent of CloudTrail management events and must be exported to your SIEM via a Pub/Sub log sink for all projects in the organization. Configure the sink at the organization or folder level, not per-project, or you will miss new projects. Data Access logs are off by default for good reason: enabling them on BigQuery, Cloud Storage, or Cloud SQL can produce tens of TB per day. Enable Data Access DATA_READ and DATA_WRITE only on projects tagged sensitive, and within those projects only on the specific services where you need read auditability. For Cloud Storage, prefer Storage Audit Logs filtered to administrative actions plus reads of buckets with specific labels. VPC Flow Logs in GCP are configured per-subnet and sampled by default (0.5 to 1.0 sample rate); keep the sample rate low (0.1) for non-sensitive subnets and 1.0 for DMZ or PCI subnets. Pipe everything through a Pub/Sub topic with a Dataflow filter job before sending to the SIEM.

The cheapest log is the one you never ingest. In AWS, the pattern is CloudWatch Logs > Kinesis Firehose > Lambda transform > S3 or HTTP endpoint to SIEM. The Lambda transform receives a batch of records, applies filters, and returns only the matching records. Typical filters: drop AWS service principals from CloudTrail (eventSource ends with .amazonaws.com and userIdentity.type = AWSService) except for assumeRole events, drop healthcheck and metadata service calls (userAgent contains 'aws-internal'), drop expected scheduled events (eventName = DescribeInstances from known automation roles). In Azure, use an Event Hub between the diagnostic setting and Sentinel for high-volume sources, with an Azure Function consumer that filters before forwarding via the Sentinel REST API. In GCP, Pub/Sub with a Dataflow streaming pipeline does the same job; the Apache Beam SDK supports stateful filtering, which is useful for deduplicating retry storms. Document every filter rule in source control (Terraform module or similar) and tag with a justification and a review date. A filter that drops 95% of volume and saves $50k a month is also a single line of code an attacker would love to add.

Adopt a three-tier model. Hot tier: 30 to 90 days in the SIEM, fully indexed, used for active detection and recent incident response. This is the most expensive storage and should only contain logs that drive real-time detections or are queried multiple times a week. Warm tier: 6 to 24 months in a data lake (Snowflake, AWS Security Lake, Azure Data Explorer, Google BigQuery, or Splunk Smart Store). Queries are slower (seconds to minutes) but cost is 5 to 20 times lower per GB. Use the warm tier for retrospective hunts, compliance queries, and incidents that turn out to have a long dwell time. Cold tier: 2 to 7 years in object storage with Glacier or Archive storage class, compressed and partitioned by date. Cost is roughly $0.001/GB/month but retrieval takes hours. Use cold for compliance retention floors (SOX, HIPAA, PCI 1 year minimum, varies by jurisdiction). The transitions are not just storage tier changes; design the schema so the warm and cold tiers are queryable by the same field names as the hot tier (use OCSF or ECS as the common schema). Otherwise an analyst pivoting from a SIEM alert to a 6 month lookback hits a schema wall.

Some events must reach the SIEM no matter what the volume looks like, because they are the only record of the highest-impact attacker behaviors. IAM events: every IAM policy change, role creation, access key creation, and cross-account assumption in AWS; every Entra ID role assignment, application consent grant, and conditional access policy change in Azure; every IAM policy binding and service account key creation in GCP. Authentication events: every sign-in success and failure for human and service identities, including the IP, user agent, and authentication method. Cross-account, cross-tenant, and cross-project calls: anything that crosses a trust boundary is high-signal and low-volume relative to total log volume. KMS and Key Vault operations: encryption key access, deletion, and rotation events; these are the canary for ransomware in cloud environments. Resource deletion events: any DeleteBucket, DeleteVM, DeleteDatabase, especially when issued by a non-automation principal. Build a quarterly review where the security team and the cloud platform team confirm these categories are still flowing end-to-end with a synthetic test event in each cloud.