How the reliability score is computed

Sample

Reliability v1 uses every article from a given outlet that we have ingested and analyzed. Outlets with fewer than 5 analyzed articles are marked as insufficient sample. The source set is the outlet's public RSS feed; sections we don't ingest (video, paywalled content) are not included.

Formula

reliability =
   50 × confidence_norm
 + 30 × success_rate
 + 20 × volume_norm

confidence_norm = avg(confidence)                  ∈ [0, 1]
success_rate    = analyzed / (analyzed + failed)
volume_norm     = min(1, log10(analyzed + 1) / log10(100))

Three components, linearly weighted: average self-rated model confidence (cap 50), AI analysis success rate (cap 30, reflects how well-formed the HTML is and how often analysis fails), and sample volume (cap 20, log-scale, saturates at 100 articles). No hidden parameters; the formula is fully reproducible.

Tiers

• EXCELLENTHigh quality, sufficient sample, stable analysis85-100
• GOODGenerally stable with minor flags70-84
• FAIRSample is sufficient but some metrics need attention55-69
• POORSample is sufficient but composite is low; pair with other outlets0-54
• INSUFFICIENTNot enough analyzed articles yet (< 5); will accumulate—

Known limits

v1 only measures analysis stability + sample volume — not real credibility dimensions like content bias, citation quality, or correction history. The full Spec 13 release will add five dimensions: emotional neutrality, citation credibility, speculation restraint, headline honesty, and multi-perspective coverage. Until then this score is a baseline only. Also: confidence is the AI model's self-rating, not a verdict on the article's truth.

Disagree with a score?

If you represent an outlet we score, or you're a reader who thinks a score is clearly off, email [email protected]. We publish all reasonable appeals together with our raw data.