How to Validate Your SIEM Is Actually Detecting Attacks (Not Just Ingesting Logs)

Before testing detection rules, verify that log sources are actually sending current events. A rule cannot fire if the events are not arriving.

What to check for each log source:

In your SIEM's search interface, query each critical log source for events in the last 15 minutes. If any source shows no events in 15 minutes, ingestion is either broken or the source is genuinely quiet (which is itself suspicious for high-volume sources like domain controller authentication logs).

Expected event volumes for reference:

• Windows domain controller Security log: Multiple Event ID 4624 (logon) events per minute during business hours. Zero events in 15 minutes during work hours means the log forwarder is broken.
• Windows endpoint Security log: At least one Event ID 4624 per login session. If a logged-in machine shows no Security events in an hour, the agent is dead.
• Firewall/network device: At minimum, deny events every few minutes (background internet noise is constant). A firewall with zero events means the syslog forwarding is broken.
• DNS resolver log: Continuous queries during business hours. Zero DNS events means the resolver is not forwarding.

How to test log forwarder health without generating noise:

On a Windows machine you are monitoring, open Event Viewer and manually create a test event:

eventcreate /ID 999 /L APPLICATION /T INFORMATION /SO SIEM-TEST /D "SIEM ingestion test event"

Search your SIEM for this event within 2 minutes. If it appears, the forwarding pipeline for that machine is functioning. If it does not appear within 5 minutes, the forwarder is broken or the pipeline has an upstream gap.

Once log ingestion is confirmed, test whether your detection rules are actually triggering. Use built-in Windows tools to simulate the attack behaviors your rules claim to detect: no additional software required.

Test 1: PowerShell download cradle (MITRE T1059.001)

Run this on a monitored test endpoint:

powershell -ExecutionPolicy Bypass -Command "Invoke-WebRequest -Uri 'http://169.254.169.254/test' -TimeoutSec 2" 2>&1

This uses the AWS metadata IP as the target (returns immediately with an error on non-cloud systems, generates no external traffic). A properly configured SIEM with PowerShell logging enabled should alert on:

• PowerShell Operational log Event ID 4104 (Script Block Logging) showing Invoke-WebRequest with an external URI
• Or your EDR should flag the PowerShell execution pattern

If no alert fires, either PowerShell Script Block Logging is not enabled (HKLM:\SOFTWARE\Policies\Microsoft\Windows\PowerShell\ScriptBlockLogging) or your SIEM has no rule for this technique.

Test 2: Suspicious process spawned by Office application (MITRE T1566.001)

This simulates a macro-delivery attack pattern:

cmd /c whoami > C:\Users\Public\test_output.txt

This by itself is benign. The detection test is to run it from a parent process that should not spawn cmd.exe. If you are testing in a controlled environment, you can script this from a PowerShell parent:

Start-Process cmd.exe -ArgumentList "/c whoami > C:\Users\Public\test_output.txt"

A SIEM with process creation logging (Sysmon Event ID 1 or Windows Event ID 4688) should generate an alert when cmd.exe is spawned by an unusual parent.

Test 3: Local account creation (MITRE T1136.001)

This is a common attacker persistence technique:

net user SIEM-TEST-ACCOUNT P@ssw0rdTest123 /add

This should generate Windows Security Event ID 4720 (user account created). Your SIEM should alert on new local account creation, and the alert should fire within 5 minutes.

Clean up immediately:

net user SIEM-TEST-ACCOUNT /delete

The final tier confirms that triggered alerts actually reach the right people in a usable form. An alert that fires in the SIEM console but routes to an unmonitored queue provides zero defensive value.

Check alert routing for each test in Tier 2:

1. Did the alert appear in the SIEM console within the expected time window (under 5 minutes for real-time rules)?
2. Did it route to the correct analyst queue, email distribution list, or ticketing system?
3. Does the alert contain enough context for a first-responder with no prior knowledge of the event to begin triage? Required context: hostname, username, process name, command line, source IP, and timestamp.
4. Is the alert severity calibrated correctly? A PowerShell download cradle alert should not be severity Low.

The enrichment gap test:

For the local account creation test (Event ID 4720), check whether your alert includes:

• The new account name
• The user or service account that created it
• The hostname where it was created
• The source IP of the creating session

If the alert says 'New user account created' with no additional context, your enrichment pipeline is not functioning. Analysts will require manual investigation to answer basic questions that should be in the alert.

Scheduling validation runs:

Run Tier 1 (ingestion check) monthly via an automated script that posts results to your SOC channel. Run Tier 2 and Tier 3 quarterly or after any SIEM rule or forwarding configuration change. Document the results each time: this documentation becomes your detection program's evidence of due care.

The three built-in tests above cover the most common detection gaps. For broader coverage of MITRE ATT&CK techniques, Atomic Red Team by Red Canary provides over 1,000 safe test cases that simulate specific attack techniques without causing damage to your environment.

Install in a controlled environment:

Install-Module -Name invoke-atomicredteam -Scope CurrentUser
Import-Module invoke-atomicredteam

Run a specific atomic test:

Invoke-AtomicTest T1059.001  # PowerShell execution
Invoke-AtomicTest T1003.001  # LSASS credential dumping
Invoke-AtomicTest T1136.001  # Local account creation

Each test maps to a specific MITRE ATT&CK technique ID. Run the test, then search your SIEM for the resulting events. Missing alerts identify specific detection gaps, which become specific rule development backlog items.

For organizations that recently covered the Slopoly AI-generated malware techniques or the LameHug APT28 LLM campaign, running the LSASS credential dumping and PowerShell obfuscation Atomic tests is specifically relevant to validating those attack patterns are detectable in your environment.