Security Log Retention: Balancing Compliance Requirements With Detection Needs

The four most common frameworks that specify log retention requirements are PCI-DSS, HIPAA, SOC 2, and ISO 27001. Their requirements are not identical, and organizations subject to multiple frameworks must satisfy the most stringent requirement for shared log sources.

PCI-DSS v4.0 (Requirement 10.7) requires at least 12 months of audit log retention with three months immediately available for analysis. Immediately available means queryable within your primary log analysis tool, not in archival storage requiring a restoration process. This is the most specific and prescriptive requirement among common frameworks.

HIPAA Security Rule requires covered entities and business associates to retain documentation of security policies and procedures for six years from creation or last effective date. For audit logs specifically, HIPAA does not specify a minimum retention period but states logs must be maintained for breach investigation and audit purposes. Most healthcare organizations apply a six-year retention period to security logs based on the broader HIPAA documentation retention requirement, which is the conservative and defensible position.

SOC 2 (Type II) does not specify a minimum log retention period. The auditors assess whether your actual retention practice is consistent with your stated policy and whether it supports the trust services criteria you are being audited against. For availability and security criteria, 12 months of retention is the typical expectation auditors apply in practice.

ISO 27001 Annex A 8.15 (Logging) requires that logs be protected from modification and unauthorized access and retained in line with the organization's documented retention requirements. It does not specify a minimum period. Your documented policy creates the compliance obligation, so be deliberate about what you document.

General regulatory guidance from NIST SP 800-92 recommends a minimum of one year for security event logs and longer retention for logs relevant to investigations of significant security incidents. For organizations subject to no specific framework, this provides a defensible baseline.

Paying SIEM ingestion and storage prices for 12-plus months of logs is one of the most common places organizations overspend on security operations. SIEM platforms charge premium prices for interactive query capability; that premium is justified for recent, actively-investigated data and unjustified for archival data that needs to be accessible but not frequently queried.

A three-tier architecture aligns cost with access frequency:

Hot storage (0 to 90 days): The primary SIEM or detection platform. Full interactive query capability, real-time alert correlation, high I/O performance. This is where active detection happens and where analysts investigate current incidents. Retain all ingested log sources here. Cost: premium (typically $0.25 to $1.50 per GB per month for managed SIEM).

Warm storage (90 days to 12 months): A queryable log archive at significantly lower cost than hot storage. Cloud-native options include AWS OpenSearch with Ultrawarm or Glacier Instant Retrieval, Azure Data Explorer, or dedicated security data lake solutions. Query performance is slower than hot storage but acceptable for investigation use cases. Cost: significantly reduced (typically $0.03 to $0.10 per GB per month).

Cold storage (12 months and beyond): Object storage (AWS S3 Glacier, Azure Blob Archive) for compliance archival. Not interactively queryable without restoration, but meets the retention obligation for audit purposes. Retrieval for forensic investigation requires a restoration job that may take hours. Cost: minimal (typically $0.001 to $0.004 per GB per month).

The transition from hot to warm at 90 days aligns with PCI-DSS's three-month immediate availability requirement while dramatically reducing storage cost for months 4 through 12. The transition to cold at 12 months satisfies PCI-DSS total retention while maintaining archival access for HIPAA six-year obligations at near-zero cost.

Not all log sources require the same retention period or the same access tier. Applying a uniform retention policy to all logs regardless of type is both a cost problem and a compliance risk: you may be paying to retain low-value logs for years while deleting high-value investigation logs that a future breach inquiry will require.

Authentication and authorization logs are the highest-priority retention category. Attacker dwell times measured in weeks or months mean that the initial access event for a breach you discover today may have occurred 60 to 90 days ago. Authentication logs should remain in hot storage for 90 days and warm storage for 12 months. For HIPAA-covered environments, retain for six years in cold storage.

Endpoint process execution logs (EDR telemetry, Sysmon) have a shorter investigation relevance window for most use cases. 30 to 90 days in hot storage is sufficient for detection; 12 months in warm storage covers extended investigation needs. Volume tends to be high, so the cost impact of warm storage for this category is significant.

Cloud control plane logs (CloudTrail, Activity Log) should follow the same retention schedule as authentication logs given that they record the administrative actions taken by compromised cloud credentials. These are compact in volume and easy to retain in cold storage for multi-year compliance.

Firewall and network flow logs are high-volume and lower-priority for long-term retention. 30 to 90 days in hot storage for detection correlation; 6 to 12 months in warm storage for investigation. Cold storage beyond 12 months is rarely required unless a specific compliance obligation or active investigation mandates it.

Retention is not only about access duration. Compliance frameworks and incident response require that logs have not been modified since they were created. A security log that a threat actor could have tampered with has reduced evidentiary value in a breach investigation or regulatory audit.

The minimum standard for log integrity is immutability at rest. Cloud object storage services (S3 with Object Lock, Azure Immutable Blob Storage) provide write-once, read-many storage that prevents modification or deletion for a defined retention period. Enable immutability with a retention lock that matches your compliance obligation before writing logs to the archive.

Cryptographic integrity verification provides a stronger guarantee than immutability alone. Some SIEM platforms and log management tools sign each log batch with a hash or digital signature at ingestion time. The integrity of the archive can be verified by recomputing the hash and comparing against the stored value. This creates a chain of custody that demonstrates to auditors that logs have not been altered.

Access controls are the third layer. Restrict write access to log archives to only the systems that forward logs and the administrative accounts needed for retention management. No analyst or investigator should have write access to the archive. Access to read logs for investigation should be logged and attributed, and those access logs should themselves be retained.

The investigation that forces you to confront a retention gap will always involve logs that were just outside your retention window. This is not bad luck: attackers with dwell times of 30 to 90 days before detection are the norm, and a 30-day retention policy is practically a guarantee of incomplete forensic reconstruction.

When an incident requires accessing data from warm or cold storage, your incident response process needs to account for restoration time. Cold storage retrieval can take four to 12 hours. Warm storage queries may be slower than hot storage by an order of magnitude. Building test scenarios where your IR team practices pulling warm and cold storage data reduces the friction during an actual investigation.

Legal hold procedures are a separate consideration from regular retention policy. When a security incident triggers a legal or regulatory proceeding, legal hold requirements may extend retention beyond your normal schedule. Have a documented process for legal counsel to place a hold on specific log sources that suspends the normal deletion schedule. This process must be executable quickly: the window between incident discovery and evidence preservation is short.