How to Score, Tune, and Retire Noisy SIEM and EDR Detection Rules

The alert fatigue numbers from 2026 are not abstract. Microsoft and Omdia's State of the SOC report found that 46% of all alerts prove to be false positives and 42% go uninvestigated entirely, meaning close to half of what a SIEM generates never gets looked at by a human. The SANS 2025 Detection and Response Survey found that 73% of security teams name false positives their single biggest detection challenge, with more than 60% encountering them frequently or very frequently. Vectra AI's 2026 State of Threat Detection report puts the daily alert count at an average of 2,992 per organization, with 63% going unaddressed.

Those numbers describe a downstream symptom. The upstream cause, in most environments that are not running a mature, staffed detection engineering function, is that detection rules get written once and then never revisited. A rule ships with a vendor's default content pack, or gets built during an incident response and left running afterward, or arrives from an MSSP's baseline configuration, and then sits in production indefinitely because removing a rule feels riskier than leaving it, even when the rule has not produced a real finding in over a year. Nobody owns the decision to retire it, so nobody makes that decision.

This is different from a log source coverage problem, though the two compound each other. A rule can be perfectly well-written and still be worthless if it is built against a source that only has partial coverage, which is why confirming the underlying telemetry is intact, covered in our SIEM log source coverage checklist, has to happen before a noisy rule gets blamed on its logic rather than its data. Once coverage is confirmed, the remaining question is purely about the rule's own track record, and that is what a scoring framework answers.

Each active rule gets scored 0 to 4 on five criteria, multiplied by a weight, and summed into a composite out of 100 (the maximum possible weighted sum of 4.0 multiplied by 25). The weights below are a reasonable starting allocation for a team without a dedicated detection engineer; adjust them once you have a full scoring pass or two under your belt and a sense of which factor actually drove your best keep and cut decisions.

Most of the inputs above already exist somewhere in your stack, they are just never assembled in one place. Here is the pull, then a worked example against three representative rules so the arithmetic is concrete rather than abstract.

Pulling the data. True-positive rate and mean triage time come from your case management or ticketing system, filtered to the rule's alert source over the last 90 days and grouped by disposition (true positive, false positive, benign true positive) and by the timestamp gap between creation and closure. In Microsoft Sentinel, this typically means querying the SecurityIncident table joined against the analytics rule name, which is exactly the kind of query covered in our guide to writing KQL queries for Microsoft Sentinel. In Splunk, the equivalent lives in the notable event index with a disposition field; in an OpenSearch-based stack, it is whatever case management index your SOAR or ticketing tool writes to. ATT&CK mapping comes from whatever field your rule already tags (Sigma rules carry this in a tags: attack.txxxx field natively), cross-referenced against every other active rule's tags to count overlap. Asset criticality comes from your CMDB or asset inventory's existing tiering, not a new exercise built for this framework.

Worked example. Three rules, scored against the rubric above using 90 days of data.

Rule A, "Suspicious PowerShell Encoded Command Execution": 400 alerts fired in 90 days, 14 confirmed true positives (3.5% TP rate, score 0). It maps to ATT&CK T1059.001 and overlaps with four other active PowerShell-related rules already in production (score 1). Mean triage time is 8 minutes, since analysts have learned to check the parent process and dismiss quickly (score 3). It covers general endpoints, not crown-jewel assets (tier 2, score 2). Volume is roughly 31 alerts per week, in the 20-to-50 band (score 1). Weighted composite: (0 x 0.35) + (1 x 0.20) + (3 x 0.20) + (2 x 0.15) + (1 x 0.10) = 1.20, scaled by 25 = 30 out of 100.

Rule B, "New Service Installation on a Domain Controller": 6 alerts in 90 days, 5 confirmed true positives during red-team and audit validation (83% TP rate, score 4). It maps to T1543.003 with no other active rule covering Windows service installation on tier-0 assets specifically (score 4). Mean triage time is 25 minutes, since each alert requires checking the service binary, its signer, and install context (score 2). It covers domain controllers, a tier-0 asset class (score 4). Volume is under 1 per week (score 4). Weighted composite: (4 x 0.35) + (4 x 0.20) + (2 x 0.20) + (4 x 0.15) + (4 x 0.10) = 3.60, scaled by 25 = 90 out of 100.

Rule C, "Password Spray Pattern (Multiple Failed Logins Followed by Success)": 85 alerts in 90 days, 20 confirmed true positives (23.5% TP rate, score 2). It maps to T1110.003 and overlaps partially with one other rule (an impossible-travel detection), score 3. Mean triage time is 12 minutes (score 3). It covers a mix of standard and privileged service accounts, tier 1 (score 3). Volume is about 6.6 per week, in the 5-to-20 band (score 2). Weighted composite: (2 x 0.35) + (3 x 0.20) + (3 x 0.20) + (3 x 0.15) + (2 x 0.10) = 2.55, scaled by 25 = 64 out of 100.

The composite score maps to four action tiers. None of these are meant to be applied mechanically without a human sanity check, which is exactly what the failure cases section below covers.

A scoring framework is only as good as the judgment applied around its edges. These are the ways teams misuse a rubric like this in practice.

Retiring a rare-but-severe rule because it rarely fires. A rule watching for a specific lateral-movement technique against tier-0 infrastructure might fire twice a year and be a true positive both times. Scored purely on volume or even on a shallow reading of the composite, it can look like a retirement candidate. It is not. Business criticality exists in the rubric precisely to counterweight this, but a low-frequency, high-severity rule deserves a manual override that keeps it regardless of its raw composite score. Treat any tier-0 asset rule as exempt from automatic retirement, full stop.

Survivorship bias in what gets measured. A scoring pass only evaluates rules that exist and fire. It says nothing about the attack techniques you have no detection for at all, and a team that spends a review cycle purely trimming existing rules without asking what is missing ends up with a smaller but no more complete ruleset. Behavioral baselining approaches, covered in our UEBA guide, catch a category of anomaly that static threshold rules structurally cannot, and a scoring exercise on your existing static rules is not a substitute for asking whether behavioral coverage is missing entirely.

Tuning or retiring based on too short a data window. Thirty days of data will misrepresent a rule tied to a quarterly or seasonal pattern, a tax-season phishing detection, an annual audit-driven access review rule, or anything tied to a business cycle that does not repeat monthly. Use a full 90-day window at minimum, and for genuinely infrequent techniques, look back further before making a retirement call.

Gaming the triage-time metric. If analysts learn that fast dismissals make a rule look efficient, some will start closing tickets without real investigation just to keep the number low, which quietly erodes the value of the true-positive rate you are also measuring from the same disposition data. Spot-check a sample of fast closures periodically rather than trusting the aggregate blindly.

Treating the composite score as a substitute for judgment. The score is an input to a decision, not the decision itself. A rule scoring 26 might still deserve to stay if a recent incident postmortem specifically called for it, and a rule scoring 78 still deserves scrutiny if the underlying log source, per the coverage checklist above, turns out to be incomplete.

Scoring the rule instead of the pipeline feeding it. Sometimes a rule's poor score is really a data quality problem: duplicate events, unparsed fields, or noisy enrichment happening upstream of the SIEM. Before committing to a rewrite of the detection logic itself, it is worth checking whether a security data pipeline tool is filtering, deduplicating, and enriching that telemetry before it ever reaches the rule; our comparison of Cribl, DataBahn, and Observo AI for security data pipelines covers that layer as a complementary noise-reduction lever that operates upstream of any individual rule's logic.

It is worth being precise about what this framework is not. Elastic's Detection Engineering Behavior Maturity Model describes five stages, Foundation, Basic, Intermediate, Advanced, and Expert, that assess the maturity of a team's processes and behaviors: how rules get documented, how threat modeling feeds rule creation, how telemetry gets validated, how a program improves over time. Ryan Stillions' Detection Maturity Level model takes a different axis entirely, describing nine levels (0 through 8) of what kind of intelligence an organization can actually detect against, from raw atomic indicators at the bottom up through tactics, techniques, and adversary goals at the top. The Detection Engineering Maturity Matrix at detectionengineering.io adds a third lens, three stages (Defined, Managed, Optimized) aimed specifically at helping a team figure out how to build or grow a detection engineering function as an organizational capability.

All three of those are program-level assessments. They answer "how mature is our detection function as a whole" and "what kind of intelligence can we act on." None of them hand a two-person security team a way to look at the 600 rules already running in production this afternoon and decide which ones are worth keeping. That is the gap this scoring framework fills, and it is a deliberately narrow, tactical complement rather than a competing maturity model. A team sitting at DEBMM's Foundation stage or DML level 1 can still run this scorecard against its existing ruleset today; doing so, and documenting the retire and tune decisions that result, is itself evidence of the kind of rule-management discipline that DEBMM's own criteria reward on the path toward its later stages. Use the maturity models to plan the program. Use this scorecard to clean up what the program has already accumulated. The two are not in tension, and a mature detection engineering practice, the kind that also runs a structured threat hunting program, typically ends up feeding new hypothesis-driven rules from hunting straight into this same scoring cycle once they graduate to production.

A minimal, copyable set of fields to track per rule. A spreadsheet is sufficient to start; you do not need a dedicated tool to run the first scoring pass.