SIEM Detection Library Version Control: Detection as Code at Scale

The pathologies of ad-hoc SIEM rule management compound silently. Rules created directly in the console have no commit history, so when a rule starts misfiring at 3 a.m. nobody can identify what changed or who changed it. There is no diff between the version that worked yesterday and the version that doesn't work today. Duplicate rules proliferate because there is no searchable index: the same brute-force detection ends up implemented three times by three engineers with slight variations in threshold. Rules that have not fired in eighteen months sit in production because deleting them is risky and nobody owns the decision. False positive tuning happens in place, with no record of why a particular IP range was excluded or who approved the exception. Performance degrades as expensive queries accumulate because nobody profiled them before deployment. The SOC manager has no way to answer the simple question 'what is our detection coverage for credential access on Windows endpoints?' because there is no machine-readable mapping from rules to ATT&CK techniques. Each of these problems is solvable individually with discipline, but collectively they require a system, and that system is detection as code.

Sigma is a YAML-based detection format designed to be portable across SIEMs. A Sigma rule has a defined anatomy: title, id (UUID), status (experimental, test, stable, deprecated), description, references, author, date, modified, tags (commonly MITRE ATT&CK technique IDs), logsource (product, service, category), detection (selection criteria and condition logic), falsepositives (documented known FP sources), and level (informational through critical). The detection block uses selection blocks and a condition expression that combines them with boolean logic and aggregation functions. pySigma is the conversion library that translates Sigma rules into the query language of your specific platform: Splunk SPL, Microsoft Sentinel KQL, Elastic ES Query DSL, CrowdStrike LogScale, Sumo Logic, Chronicle YARA-L, and others. The SigmaHQ repository contains roughly three thousand community-maintained rules covering most ATT&CK techniques, providing a substantial starting library. SOC Prime extends this with curated and commercial content. Writing in Sigma rather than native query language preserves portability when you change SIEM vendors and forces the engineer to think in terms of log sources and selection logic rather than vendor-specific syntax tricks.

A working detection-as-code repository typically splits rules by source platform under directories like rules/windows, rules/linux, rules/cloud/aws, rules/network, and rules/application. Each rule has a sibling tests directory containing at least one true-positive log sample and at least one documented false-positive scenario. A separate directory holds custom field mappings, log source schemas, and conversion overrides for your environment. Lifecycle state is tracked via the Sigma status field combined with branch and tag conventions: a new rule starts on a feature branch with status experimental, moves to test when merged to a staging branch and deployed to a non-production SIEM workspace, moves to stable when promoted to production after a defined soak period, and moves to deprecated when scheduled for retirement. The retire-don't-delete principle keeps deprecated rules in the repository with a tombstone tag, preventing accidental recreation and preserving the historical context for incident investigations that reference old alerts. Promotion to stable requires a CODEOWNERS approval from a senior detection engineer, and only the CI deployment service account has write access to the production SIEM API.

The CI pipeline is where detection as code earns its keep. Stage one runs sigma-cli or the equivalent linter to validate YAML syntax and Sigma schema conformance. Stage two converts every changed rule via pySigma to the target SIEM dialect, catching conversion failures and unsupported constructs early. Stage three runs the rule against the bundled true-positive log samples, asserting that the rule fires; this requires a small test harness, often built on the SIEM vendor's local engine or a Sigma evaluation library. Stage four runs the rule against the documented false-positive samples and against a corpus of benign baseline logs, asserting that the rule does not fire on known-good activity. Stage five estimates query cost: for Splunk this means dispatching against a small index and capturing scan_count and run_duration; for Sentinel it means a getschema query plus a small-window execution; rules above a threshold trigger mandatory performance review. Stage six validates that every rule has at least one ATT&CK technique tag in the tags field and that the technique exists in the current ATT&CK matrix. Stage seven enforces documentation: description must exceed 50 characters and falsepositives must not be empty. Pull requests that fail any gate are blocked from merge.

Once every rule is tagged with the ATT&CK techniques it detects, coverage reporting becomes a query rather than a project. A nightly job parses every Sigma file, extracts tags, and produces a coverage matrix joined against the ATT&CK STIX bundle. The output is a heatmap published to a Confluence page or internal dashboard showing every technique colored by detection count and rule maturity. Techniques with zero rules become the prioritized backlog for the detection engineering team. Techniques with only experimental rules become candidates for promotion testing. The same data feeds a monthly report to the CISO showing trend over time: techniques covered, rules promoted to stable, rules retired, mean time from threat intel to deployed detection. This visibility transforms the detection team from a reactive ticket-driven function into a measurable product team with a published roadmap. It also makes capability claims defensible during board reporting and during sales-engineering conversations with customers in regulated industries who ask 'what is your detection coverage for ransomware techniques?'

Log source schemas change. Windows Security event field names occasionally shift between OS versions. Microsoft 365 unified audit log renames properties roughly once per quarter. AWS CloudTrail introduces new event names as services launch. Detection as code makes this drift visible. The repository includes a daily job that runs a sample of every production rule against current log data; if a rule's selection criteria no longer match the underlying schema, the job opens a GitHub issue with the offending rule, the failed selection, and a suggested fix from the diff between the cached old schema and the current schema. Schema files live in the repository under schemas/ and are themselves versioned, so a schema update is a reviewable pull request. SIEM vendor query language updates follow the same pattern: pySigma backend updates land as dependency bumps with a CI run that re-converts the entire library, surfacing any rules that fail to compile against the new backend version. The result is that the detection library degrades gracefully under change rather than silently breaking, which is the difference between a SOC that knows its detection posture and one that thinks it does.