Cloud Incident Response: AWS CloudTrail Forensics, Containment, and Evidence Preservation
Retool's new app builder is where AI-generated code ships safely
Building apps with AI is easy. Getting them to production safely is another story.
Cloud incidents fail in predictable ways: the analyst looks at the CloudWatch dashboard instead of CloudTrail, containment happens via network isolation instead of IAM policy revocation, and evidence is destroyed when the compromised instance is terminated before a snapshot is taken.
This guide covers the cloud IR workflow that matches how cloud attacks actually happen: starting with IAM principal identification, moving to blast radius scoping, then containment that preserves evidence, and recovering with verified least-privilege credentials.
Initial Triage: Identify the Compromised Principal and Scope the Blast Radius
When a cloud security alert fires, the first question is which IAM principal is compromised and what it can access. Run aws sts get-caller-identity using the suspected credentials to confirm the identity. Query CloudTrail for the last 90 days of activity for that principal: aws cloudtrail lookup-events --lookup-attributes AttributeKey=Username,AttributeValue=<username> --start-time <90-days-ago>. Look for CreateAccessKey, AttachUserPolicy, AssumeRole, and PutRolePolicy events that indicate privilege escalation. Run aws iam list-attached-user-policies and aws iam list-user-policies to map what the principal is currently authorized to do. Check for newly created access keys and new role trust policy modifications. For scope assessment, query CloudTrail for all API calls from the same sourceIPAddress as the compromised principal to identify all affected resources.
Containment: Disable Credentials Without Destroying Evidence
Contain the compromised principal without deleting evidence. Disable IAM access keys with aws iam update-access-key --access-key-id <key> --status Inactive rather than deleting them; deletion removes the audit trail. Attach an explicit deny-all inline policy to the compromised user or role: create a policy with Effect: Deny, Action: *, Resource: * and attach it with aws iam put-user-policy. This overrides all other policy grants without requiring knowledge of what policies are currently attached. If the principal has assumed a role, wait for existing sessions to expire after applying the deny policy. For SCP-level containment in AWS Organizations, attach a deny-all SCP to the affected account to prevent any principal in that account from making API calls while the investigation proceeds.
Briefings like this, every morning before 9am.
Threat intel, active CVEs, and campaign alerts, distilled for practitioners. 50,000+ subscribers. No noise.
Forensic Evidence Collection and Preservation
Preserve CloudTrail logs before they roll or are tampered with. Enable CloudTrail log file integrity validation before an incident so logs include SHA-256 digests for tamper detection. Export all CloudTrail logs for the affected account and time range to an isolated S3 bucket in a separate account with Object Lock enabled (WORM mode) to prevent modification: aws s3 sync s3://cloudtrail-bucket/ s3://forensic-evidence-bucket/ --sse aws:kms. Capture VPC Flow Logs for the affected subnets. For EC2 instances involved in the incident, take EBS snapshots before terminating: aws ec2 create-snapshot --volume-id vol-<id>. Do not terminate instances until snapshots complete. For compromised Lambda functions, download the deployment package via aws lambda get-function --function-name <name>. Document all containment actions with timestamps.
Recovery and Post-Incident Hardening
Recovery in cloud incidents requires rotating all credentials that the compromised principal had access to, not just the compromised credential itself. Enumerate all secrets the principal could access: AWS Secrets Manager secrets with GetSecretValue permission, SSM Parameter Store parameters with GetParameter permission, and environment variables in EC2 or Lambda. Rotate each one. Create replacement credentials with least-privilege policies derived from AWS IAM Access Analyzer or CloudTrail event history rather than copying the compromised principal's existing permissions. Review all IAM changes made during the incident window and revert unauthorized policy attachments, new users, and trust policy modifications. Enable AWS Config rules for continuous compliance monitoring: IAM_PASSWORD_POLICY, ACCESS_KEYS_ROTATED, MFA_ENABLED_FOR_IAM_CONSOLE_ACCESS, and ROOT_ACCOUNT_MFA_ENABLED.
The bottom line
Cloud incident response quality is determined by what you enabled before the incident. CloudTrail without data events cannot show what was read from S3. GuardDuty disabled means no automated compromise detection. IMDSv2 not enforced means SSRF leads directly to credential theft. Enable logging, enable GuardDuty, enforce IMDSv2, and pre-build a deny-all containment policy for every IAM role tier before you need it. For consolidating GuardDuty and Security Hub findings into a single workflow, see the AWS Security Hub guide. For the on-premises ransomware IR playbook that complements this cloud guide, see the ransomware incident response playbook.
Frequently asked questions
How do I tell if an AWS IAM access key has been compromised?
Indicators of compromised IAM access keys: API calls from unfamiliar IP addresses or geographies (check CloudTrail sourceIPAddress), calls at unusual hours outside the principal's normal working pattern, CreateAccessKey or AttachUserPolicy events from the principal (indicates self-escalation), high-volume API calls consistent with enumeration (ListBuckets, DescribeInstances, ListFunctions called in rapid succession), and GuardDuty findings in the UnauthorizedAccess or Recon category. Run aws cloudtrail lookup-events filtered on the access key ID to get the full activity timeline. Any API call from a Tor exit node, VPN provider, or cloud provider IP range the organization does not use is a strong indicator.
What should I check in CloudTrail during a cloud incident?
Priority CloudTrail event types to investigate during a cloud incident: CreateAccessKey and DeleteAccessKey (credential management), AttachUserPolicy, AttachRolePolicy, PutUserPolicy, PutRolePolicy (privilege escalation), UpdateAssumeRolePolicy (trust policy modification for pivot), ConsoleLogin with MFA false (console access without MFA), AssumeRoleWithWebIdentity and AssumeRole (cross-account access), CreateUser and CreateRole (new principal creation), GetSecretValue and GetParameter (credential access from Secrets Manager and SSM), and PutObject or GetObject on sensitive S3 buckets (data access or exfiltration). Sort by eventTime to reconstruct the attack timeline.
How do I contain a compromised AWS IAM role without disrupting legitimate workloads?
Attach an explicit deny inline policy to the role rather than deleting it or removing its trust policy. Create a policy with Effect: Deny, Action: *, Resource: * and attach it with aws iam put-role-policy. This blocks all new API calls while preserving the role configuration for forensic review. For roles used by running EC2 instances or Lambda functions, the deny policy takes effect immediately for new API calls; existing in-progress calls complete first. If the role is used by critical production workloads, create a replacement role with the same permissions, update the workloads to use the new role, and then apply the deny policy to the compromised role.
What is IMDS credential theft in AWS and how do I prevent it?
The EC2 Instance Metadata Service (IMDS) provides temporary IAM role credentials to code running on the instance via http://169.254.169.254/latest/meta-data/iam/security-credentials/. If an attacker achieves SSRF or code execution on the instance, they can retrieve these credentials and use them remotely outside AWS. Prevention: require IMDSv2 (which uses a PUT request session token and blocks simple SSRF) on all EC2 instances by setting --metadata-options HttpTokens=required at instance launch or via AWS Config rule EC2_IMDSV2_REQUIRED. For containers on ECS, disable task role credential access when not required. Enforce this org-wide with an SCP denying RunInstances when metadata options do not enforce IMDSv2.
What logging should I enable before a cloud incident to support forensics?
Enable before any incident occurs: AWS CloudTrail multi-region trail with log file integrity validation and delivery to a dedicated S3 bucket in a separate account, AWS Config recording for all resource types and regions, VPC Flow Logs on all VPCs with delivery to S3 or CloudWatch Logs, S3 server access logging and S3 data events in CloudTrail for buckets containing sensitive data, AWS GuardDuty in all regions and accounts with S3 protection, Lambda data events in CloudTrail, and Route 53 DNS query logging. Without pre-incident logging, CloudTrail is limited to 90 days of management events and no data events, making forensic reconstruction incomplete.
How is cloud incident response different from on-premises IR?
Cloud IR differs in speed, evidence volatility, and attack surface. Credentials compromised in the cloud can be used immediately from any geography; there is no network-level chokepoint to slow an attacker. Evidence (logs, instance state) can be deleted by the attacker if they have sufficient IAM permissions, so preservation is time-critical. The attack surface is the IAM control plane, not just the network: privilege escalation via role chaining, cross-account access, and STS credential generation replace lateral movement via pass-the-hash. Cloud IR requires IAM forensics skills in addition to traditional IR skills, and containment uses IAM policy controls rather than firewall rules.
Sources & references
Free resources
Critical CVE Reference Card 2025–2026
25 actively exploited vulnerabilities with CVSS scores, exploit status, and patch availability. Print it, pin it, share it with your SOC team.
Ransomware Incident Response Playbook
Step-by-step 24-hour IR checklist covering detection, containment, eradication, and recovery. Built for SOC teams, IR leads, and CISOs.
Get threat intel before your inbox does.
50,000+ security professionals read Decryption Digest for early warnings on zero-days, ransomware, and nation-state campaigns. Free, daily, no spam.
Unsubscribe anytime. We never sell your data.
Founder & Cybersecurity Evangelist, Decryption Digest
Cybersecurity professional with expertise in threat intelligence, vulnerability research, and enterprise security. Covers zero-days, ransomware, and nation-state operations for 50,000+ security professionals every morning.
Win a $2,495 Black Hat pass.
Full-access to Black Hat USA 2026 in Las Vegas. Subscribe free to enter.