Sigma Rules to Production: Building a CI/CD Pipeline for Detection-as-Code

Start with a clear directory layout. Keep Sigma YAML files under a rules/ directory, organized by MITRE tactic: rules/initial-access/, rules/persistence/, rules/lateral-movement/, and so on. This mirrors the ATT&CK taxonomy and makes coverage gap analysis trivial later.

Branch strategy mirrors a standard GitFlow. The main branch represents what is currently deployed to production. A staging branch represents what is deployed to your staging SIEM workspace. Feature branches follow the pattern feat/rule-name-technique-id. All new rules start on a feature branch and open a pull request to staging first.

Never merge directly to main. The pipeline enforces this via branch protection rules requiring at minimum two reviewers and all status checks passing. Require that at least one reviewer holds the detection-engineer role in your GitHub team to prevent analysts from bypassing peer review on high-noise rules.

Install the core toolchain with: pip install sigma-cli pySigma-backend-splunk pySigma-backend-microsoft365defender. For Sentinel you will use the microsoft365defender backend, which targets the Kusto Query Language used by Sentinel Analytics rules.

Basic conversion for a single rule: sigma convert -t splunk rules/lateral-movement/pass-the-hash-net.yml. For Sentinel KQL: sigma convert -t microsoft365defender -p sysmon rules/lateral-movement/pass-the-hash-net.yml.

In the pipeline you will batch-convert all rules in a changed directory. Use git diff --name-only origin/main HEAD -- rules/ to get the list of modified YAML files, then pass that list to sigma convert. This avoids reconverting the entire ruleset on every commit.

Backend pipelines matter. The -p sysmon flag applies the Sysmon field-mapping pipeline, which translates Sigma's generic field names (CommandLine, Image, ParentImage) to the names your data source actually uses. If you ingest Windows events via a different source, create a custom pipeline YAML under pipelines/ and reference it with -p pipelines/your-pipeline.yml. Commit pipeline configs to the repo alongside rules so field mappings are versioned.

A rule that converts successfully is not a rule that works correctly. You need to validate that the converted query actually matches known-malicious log samples and does not match known-benign ones.

Create a tests/ directory with subdirectories matching your rules/ layout. Each rule gets a YAML test file: tests/lateral-movement/pass-the-hash-net.yml. The test file specifies true-positive events (log lines that should trigger the rule) and true-negative events (log lines that should not).

For Splunk, spin up a free Splunk developer instance or use Splunk Attack Range for automated testing. The sigma-cli test command runs these validations: sigma test --backend splunk --pipeline sysmon rules/lateral-movement/pass-the-hash-net.yml.

For Sentinel, use the Kusto Explorer desktop client or az monitor log-analytics query CLI to run KQL against a test workspace populated with sample data. The EVTX Samples repository on GitHub provides thousands of Windows event log files organized by attack technique. Import the relevant EVTX files into your test workspace using the Sentinel Data Connector or a custom ingestion script.

Store test results as pipeline artifacts. If the true-positive rate for any rule drops below 100% or the false-positive rate exceeds a configurable threshold, fail the pipeline and block the merge.

The workflow file lives at .github/workflows/detection-pipeline.yml. It triggers on pull_request to staging and push to staging and main.

The PR validation job runs on pull_request. Steps: checkout code, set up Python, install dependencies, identify changed rule files with git diff, run sigma convert on each changed file, run sigma test against sample data, upload conversion artifacts.

The staging deployment job runs on push to staging after PR merge. Steps: convert all rules in the repository, authenticate to your staging SIEM workspace using repository secrets (SENTINEL_STAGING_WORKSPACE_ID, SENTINEL_STAGING_CLIENT_SECRET, SPLUNK_STAGING_HEC_TOKEN), deploy rules via API, tag the commit with the deployment timestamp.

The production promotion job runs on push to main and requires a manual approval step via GitHub Environments. Configure the production environment to require one designated approver. The job deploys to production and writes a deployment manifest JSON file recording which rule versions are live. That manifest is the foundation for your rollback procedure.

Secret management: store all API keys in GitHub Actions Encrypted Secrets. Never hardcode workspace IDs or tokens in workflow files. Use environment-scoped secrets so staging keys cannot be used by production jobs.

For Sentinel, use the Analytics Rules REST API. Endpoint: PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/providers/Microsoft.SecurityInsights/alertRules/{ruleId}. Authenticate with a service principal that has Microsoft Sentinel Contributor role on the staging workspace only.

Construct the request body by reading the converted KQL and wrapping it in the ScheduledAlertRule schema. Set severity, query frequency, query period, trigger threshold, and suppression settings. Store these operational parameters as YAML front matter in each Sigma rule file so they travel with the rule and do not require manual entry in the API call script.

For Splunk, use the savedsearches endpoint: POST https://splunk-staging:8089/servicesNS/nobody/search/saved/searches. Set alert_type, cron_schedule, dispatch.earliest_time, and actions.email or actions.notable as appropriate for your environment. Authenticate with a token stored in GitHub Secrets.

After deployment, wait 30 minutes and then query each SIEM for rule execution errors. For Sentinel, check the SecurityAlert table for alerts with ProviderName == "ASI Scheduled Alerts". For Splunk, check the _internal index for search scheduler errors. Fail the deployment job if any rule reports an execution error.

The deployment manifest written during each production deployment contains the previous rule version hash. The rollback script reads the manifest, fetches the previous rule version from git history, converts it, and redeploys it to production via the same API paths.

Automate rollback triggering by adding a post-deployment monitoring job that runs 2 hours after each production push. The job queries Sentinel using KQL and Splunk using SPL for alert volume per rule compared to the 7-day baseline.

Sentinel KQL for false positive spike detection: SecurityAlert | where TimeGenerated > ago(2h) | where ProviderName == "ASI Scheduled Alerts" | summarize AlertCount = count() by AlertName | join kind=inner (SecurityAlert | where TimeGenerated between (ago(9d) .. ago(2d)) | summarize BaselineAvg = avg(count()) by bin(TimeGenerated, 1d), AlertName | summarize BaselineAvg = avg(BaselineAvg) by AlertName) on AlertName | where AlertCount > BaselineAvg * 3.

Splunk SPL equivalent: index=notable earliest=-2h | stats count as AlertCount by search_name | join search_name [search index=notable earliest=-9d latest=-2d | timechart span=1d count by search_name | stats avg(*) as BaselineAvg by search_name] | where AlertCount > BaselineAvg * 3.

If the job finds any rule exceeding three times its baseline alert volume, it opens a GitHub Pull Request from a rollback/ branch targeting main with the previous version of that rule and assigns it to the on-call detection engineer. The engineer reviews and merges to trigger the rollback deployment.