How to Implement Security Logging: What to Log, What Not to Log, and How Long to Keep It
Retool's new app builder is where AI-generated code ships safely
Building apps with AI is easy. Getting them to production safely is another story.
The question is not 'should we log?': it is 'what should we log, and how long should we keep it?' Logging everything is expensive, generates unmanageable data volumes, and buries the signal in noise. Not logging enough means critical evidence is unavailable when an incident occurs.
A targeted security logging strategy focuses on the events that generate detection value and investigation evidence: authentication events, process execution, network connections, DNS queries: while excluding the high-volume noise that generates cost without value.
Tier 1 Log Sources: The Non-Negotiables
These are the sources that, if missing, fundamentally limit your ability to detect and investigate incidents:
1. Windows Security Event Log (Domain Controllers)
Domain controllers see every authentication event in the environment:
Priority Windows Security Event IDs:
4624 - Successful logon (Logon Type 3=network, 10=RDP, 7=unlock, 2=interactive)
4625 - Failed logon attempt
4634 - Logoff
4648 - Logon with explicit credentials (RunAs, PsExec -u)
4662 - Operation performed on Active Directory object
4663 - Attempt to access object (requires SACL)
4672 - Special privileges assigned to new logon (admin-equivalent logon)
4688 - Process creation (enable command-line logging!)
4698 - Scheduled task created
4702 - Scheduled task modified
4720 - User account created
4726 - User account deleted
4728 - Member added to security-enabled global group
4732 - Member added to security-enabled local group
4740 - User account locked out
4756 - Member added to security-enabled universal group
4768 - Kerberos ticket (TGT) requested
4769 - Kerberos service ticket requested
4776 - NTLM authentication attempt
7045 - New service installed
Enable audit policies for the above (via Group Policy):
Computer Configuration > Windows Settings > Security Settings > Advanced Audit Policy
- Account Logon: Audit Credential Validation - Success, Failure
- Account Management: All - Success, Failure
- Logon/Logoff: Audit Logon - Success, Failure
- Logon/Logoff: Audit Special Logon - Success
- Object Access: Audit File Share - Success, Failure (for sensitive shares)
- Policy Change: All - Success, Failure
- Privilege Use: Audit Sensitive Privilege Use - Success, Failure
- System: Audit Security State Change - Success, Failure
- Detailed Tracking: Audit Process Creation - Success (and enable command-line)
2. Firewall / NGF Logs
High-value firewall events:
- Deny events (blocked connection attempts): inbound and outbound
- Allow events for sensitive ports (22, 3389, 443 from unexpected sources)
- New allow rules created
- Traffic to/from threat intel blacklists
NOT worth logging:
- Allow events for high-volume benign traffic (HTTP to CDNs, DNS)
- Health check traffic between load balancers and backend servers
3. DNS Query Logs
DNS is the most underlogged high-value source. Every DNS query reveals which hosts are communicating with which domains: including C2 beaconing that evades firewall logs because it uses legitimate-looking HTTPS traffic:
# Windows DNS Server: enable diagnostic logging
Set-DnsServerDiagnostics -All $true -EventLogLevel 7
# Or collect DNS from the network layer via Zeek or Suricata
# Zeek extracts DNS from packet captures and logs all queries
# Pi-hole or Infoblox: can centralize DNS and forward query logs to SIEM
# What to look for in DNS logs:
# - High-frequency queries to random-looking domains (DGA malware)
# - DNS queries to newly registered domains (<30 days old)
# - TXT record queries (often used for C2 data exfiltration)
# - Very long subdomain queries (DNS tunneling)
# - Internal hosts querying domains not on your organization's baseline
4. Authentication Logs (IdP / SSO)
For Azure AD / Entra ID:
- Sign-in logs (all authentication attempts)
- Audit logs (role assignments, MFA changes, app consent grants)
- Risk detection logs (impossible travel, unfamiliar location, leaked credentials)
For Okta:
- System Log (all events)
- Priority: auth.login.fail, user.session.start, user.mfa.factor.deactivate,
policy.evaluate_sign_on, application.user_membership.add
Tier 2 Log Sources: High Value for Specific Use Cases
Web Application Logs (WAF + Application)
From WAF:
- Blocked requests (SQL injection, XSS, path traversal attempts)
- Rate-limit triggers
- GeoIP blocks
From application server (Nginx/Apache/etc):
- 400-series errors at unusual rates (scanning/fuzzing)
- POST requests to unexpected endpoints
- Long URLs (potential injection)
NOT worth logging:
- Every 200 OK to static assets (CSS, JS, images)
- Health check requests from load balancers
Endpoint/EDR (Sysmon or Native EDR)
Sysmon provides process-level telemetry not available from Windows Event Log alone:
<!-- Sysmon events to prioritize -->
<!-- Event 1: Process Create (with hash, command line, parent) -->
<!-- Event 3: Network Connection (outbound connections from all processes) -->
<!-- Event 7: Image Loaded (DLL load events: detect DLL injection) -->
<!-- Event 8: CreateRemoteThread (process injection) -->
<!-- Event 11: FileCreate (file creation events) -->
<!-- Event 12/13: Registry Create/Modify (persistence) -->
<!-- Event 22: DNS Query (all DNS queries from all processes) -->
Cloud Provider Logs (AWS, Azure, GCP)
# AWS CloudTrail: ALL management events (free tier covers this)
# Priority event types:
# - ConsoleLogin (authentication to AWS console)
# - AssumeRole (who is assuming which roles)
# - CreateUser, DeleteUser, AttachRolePolicy (IAM changes)
# - PutBucketPolicy, DeleteBucket (S3 changes)
# - AuthorizeSecurityGroupIngress (firewall changes)
# - RunInstances, TerminateInstances (compute changes)
# - GetSecretValue (secrets access)
# Enable S3 Data Events only for sensitive buckets
# (cost: $0.10/100K events: expensive at scale, worth it for sensitive data stores)
aws cloudtrail put-event-selectors \
--trail-name MyTrail \
--event-selectors '[{"ReadWriteType":"All","IncludeManagementEvents":true,"DataResources":[{"Type":"AWS::S3::Object","Values":["arn:aws:s3:::sensitive-bucket/"]}]}]'
Briefings like this, every morning before 9am.
Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.
What NOT to Log: The Cost-Noise Problem
SIEM ingest cost scales with data volume. Many organizations spend the majority of their SIEM budget on high-volume, low-security-value logs:
Stop logging these (or heavily filter before sending to SIEM):
# High volume, low security value:
- Windows Event ID 4656 (handle requested): extremely noisy, rarely actionable
- Windows Event ID 4658 (handle closed): companion to 4656, equally noisy
- Successful authentication to benign services (LDAP health checks, etc.)
- DNS queries to CDN domains (akamaiedge.net, cloudfront.net, etc.) from trusted sources
- Load balancer health checks (every 5-30 seconds to every backend)
- Antivirus successful scans (log only detections and quarantines)
- IDS informational alerts with no associated context
- Full packet capture of encrypted TLS traffic (uninspectable, just metadata)
- Application INFO-level logging in production (DEBUG logs are for development)
# Log only the interesting subset:
# Windows: Use Advanced Audit Policy for surgical event selection
# Firewall: Log blocks + specific allow categories, not all allows
# Web: Log 4xx and 5xx, sample high-volume 200 traffic
Log tiering: not all logs need to be in your SIEM:
Hot tier (SIEM / high-cost storage, 90 days):
- Authentication events (IdP, Windows, VPN)
- Process creation (Sysmon Event 1 or Windows 4688)
- DNS queries
- Firewall deny events
- EDR alerts and events
- Cloud management events (CloudTrail, Azure Activity Log)
Warm tier (object storage + query capability, 1 year):
- Full firewall logs (allow + deny)
- Web access logs
- VPN connection logs
- NetFlow/IPFIX
Cold tier (compressed archive, 3-7 years for compliance):
- Compliance-required logs
- Audit trails for regulated systems
Log Retention: How Long Is Long Enough
Regulatory minimums by framework:
PCI DSS: 12 months minimum, 3 months immediately available
HIPAA: 6 years (for security documentation)
SOC 2: Typically 90 days for operational evidence, 1 year for audit
FedRAMP: 90 days local, 1 year in cloud
GDPR: No specific requirement, but must demonstrate ability to investigate
CISA recommendation: 90 days hot, 12 months cold (minimum)
Why dwell time determines retention requirements:
If the median time to breach detection is 212 days (IBM estimate), a 90-day log retention means the initial intrusion event is already gone by the time you detect the breach. The actual investigation needs:
- At minimum: the entire intrusion timeline from initial access to detection
- In practice: 12 months of authentication and network logs to capture the full attacker dwell period
Cost-efficient retention strategy:
# AWS S3: tiered storage for log archives
# Standard (hot): $0.023/GB/month: use for 90 days of SIEM-indexed logs
# Glacier Instant Retrieval (warm): $0.004/GB/month: 90 days to 1 year
# Glacier Deep Archive (cold): $0.00099/GB/month: 1-7 years
# Set lifecycle policy on your log bucket
aws s3api put-bucket-lifecycle-configuration \
--bucket my-security-logs \
--lifecycle-configuration '{
"Rules": [{
"ID": "log-tiering",
"Status": "Enabled",
"Transitions": [
{"Days": 90, "StorageClass": "GLACIER_IR"},
{"Days": 365, "StorageClass": "DEEP_ARCHIVE"}
],
"Expiration": {"Days": 2555} # 7 years
}]
}'
Log integrity: tamper evidence:
# Forward logs to an immutable destination immediately
# AWS: Enable CloudTrail log file validation
aws cloudtrail update-trail \
--name MyTrail \
--enable-log-file-validation
# Centralized syslog with WORM (write once, read many) storage
# AWS CloudWatch Logs: Enable log group protection
aws logs put-data-protection-policy \
--log-group-name SecurityLogs \
--policy-document '{"Statement":[{"Effect":"Deny","Action":["logs:DeleteLogGroup"]}]}'
The bottom line
Security logging value comes from selecting the right sources and tuning out noise: not from logging everything. The highest-value sources are Windows Authentication events (4624, 4625, 4648, 4688 with command-line) from domain controllers, DNS query logs, authentication logs from your IdP, and cloud management events (CloudTrail, Azure Activity Log). Do not log Windows Event IDs 4656/4658, load balancer health checks, or AV successful scans: they add cost without detection value. Retain hot logs for at least 90 days in your SIEM and 12 months in cold storage; most incidents are discovered after the initial compromise event has aged past 30-day retention limits.
Frequently asked questions
What should you log for security monitoring?
The highest-value sources: Windows Security events (4624 logon success, 4625 logon failure, 4648 explicit credentials, 4688 process creation with command line) from domain controllers, DNS query logs from all internal resolvers, authentication events from your IdP or SSO, firewall deny events, EDR alerts, and cloud management logs (AWS CloudTrail, Azure Activity Log). Do not log high-volume low-signal sources like load balancer health checks, successful AV scans, and Windows Event IDs 4656/4658.
How long should you keep security logs?
At minimum 90 days immediately queryable (hot) and 12 months in cold storage. CISA recommends this minimum; PCI DSS requires 12 months with 3 months immediately available. The driver is dwell time: attackers often have months of access before detection, so logs must cover the full intrusion period. Use tiered storage (SIEM for 90 days, object storage like S3 Glacier for up to 7 years) to balance cost with retention requirements.
What is the most important log source for security monitoring?
Authentication logs are the highest-value source across all environments because they are present everywhere, cover the most common attack vector (credential compromise), and have a high signal-to-noise ratio. For Windows environments: Active Directory authentication events (Event ID 4624, 4625, 4648, 4768, 4769) are essential. For cloud environments: Azure Active Directory sign-in logs or AWS CloudTrail IAM events. For SaaS: Microsoft 365 audit logs or Google Workspace audit logs. If you can only send one log source to a SIEM, authentication events will catch more active intrusions than any other single source.
How do I build security alerting without drowning in false positives?
False positive rate is the primary reason security teams disable alerts. Build alerts in three tiers: Tier 1 (near-certain indicators) — always page immediately: VSS deletion, EDR detection of known malware, OFAC-sanctioned IP connections, domain admin account creation. Tier 2 (high-signal behavioral) — alert to a queue for analyst review: brute force after X failed attempts, new privileged account creation, large outbound data transfers. Tier 3 (low-signal environmental) — report weekly, do not alert in real-time: compliance drift, patch SLA violations, expired certificates. Start with 3-5 Tier 1 rules you have high confidence in rather than 50 rules generating constant noise. Add rules only when you have tuned existing ones.
What is UEBA and how does it improve security logging value?
UEBA (User and Entity Behavior Analytics) uses machine learning to baseline normal behavior patterns for each user and system, then generates alerts when behavior deviates significantly from the baseline. Unlike rule-based detection (which fires on fixed thresholds), UEBA adapts to organizational context: an analyst downloading 500 files before a holiday weekend may be normal for them specifically, while the same action from a developer account that never accesses that folder is anomalous. UEBA is particularly effective for insider threat and compromised credential detection. Most enterprise SIEM platforms (Splunk UEBA, Microsoft Sentinel UEBA, Exabeam) offer UEBA as an add-on. It requires 2-4 weeks of baseline data before generating meaningful alerts.
What log sources must be centrally collected to meet the minimum requirements for effective security monitoring and incident response?
Minimum log sources for effective security monitoring, in priority order: Authentication logs (Active Directory Security event log Event IDs 4624, 4625, 4648, 4720, 4728, 4756; Entra ID sign-in logs for cloud; VPN authentication logs) -- without these, you cannot detect credential compromise. Endpoint process creation logs (Windows Event ID 4688 with command line enabled, or Sysmon Event ID 1; macOS Unified Logs for process execution on Mac endpoints) -- required for detecting malware execution and LOLBin abuse. DNS query logs (Windows DNS debug logging; firewall DNS inspection or Pi-hole/Infoblox logs) -- DNS is the only visibility into many malware C2 channels. Firewall and proxy logs (outbound connection attempts with destination IP/domain, bytes transferred) -- required for detecting data exfiltration and C2 communications. Email security logs (rejected/quarantined messages, attachment scanning results, URL clicks) -- required for detecting phishing as initial access. Cloud platform logs (AWS CloudTrail, Azure Activity Log, GCP Cloud Audit Logs) -- required for any cloud environment. Do not collect these in isolation: logs must be centralized and correlated across sources to be useful for investigation. A SIEM that ingests only endpoint logs misses the authentication and network context that turns a process creation event into a confirmed lateral movement indicator.
Sources & references
Free resources
Critical CVE Reference Card 2025–2026
25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.
Ransomware Incident Response Playbook
Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.
Get threat intel before your inbox does.
50,000+ security professionals read Decryption Digest for early warnings on zero-days, ransomware, and nation-state campaigns. Free, daily, no spam.
Unsubscribe anytime. We never sell your data.
Founder & Cybersecurity Evangelist, Decryption Digest
Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals every morning.
Win a $2,495 Black Hat pass.
Full-access to Black Hat USA 2026 in Las Vegas. Subscribe free to enter.