Alle berichten

How the Agnes score works

Huub, founder of Agnes ·

Agnes gives every frontend repository a health score from 0 to 100, every day there are new commits. A single number is easy to put on a dashboard or a README badge, and just as easy to distrust if you can't see how it was made. So this post explains exactly how the score is calculated, with the real formula and the real numbers. If you think one of them is wrong, I'd like to hear why.

I built Agnes because more and more of our apps and services run entirely on frontend technologies, from the interface down to the API. We needed one system that keeps those codebases healthy and catches security issues, and a score you can follow from day to day tells you more than yet another list of warnings.

What gets analysed

Agnes looks at repositories that are mostly frontend code: at least 60% of the non-blank lines are HTML, CSS (including SCSS, Sass and Less), JavaScript, TypeScript, Vue, Svelte or Astro. Server-side templates (Blade, Twig, Nunjucks, Handlebars) don't count towards that share either way, but the HTML in them is checked too. Generated and vendored folders such as node_modules and dist don't count, and you can leave out more with .gitignore-style rules per project.

The analysis runs the open-source tools frontend teams already use, inside a container with no network access: Oxlint, Biome, ESLint (with the Vue, Svelte, Angular, no-unsanitized and trust-boundary plugins), Stylelint, html-validate, the Svelte compiler, FTA, Knip, jscpd and Secretlint, plus axe-core when the accessibility checks are switched on. Dependencies are read from the lockfile and checked against OSV and the npm registry. Agnes never runs your build, your tests or your scripts, and the code is deleted when the run ends. The Tools page lists them all.

Each tool's findings are sorted into categories, and each category gets its own score.

The categories and their weights

CategoryWeightWhat it measures
Code smells35%Lint findings: error-prone patterns, complexity, unused code, CSS and HTML problems
Dependencies35%Known vulnerabilities, deprecated and outdated packages in the lockfile
Security15%Committed secrets, HTML injection and similar findings
Duplication15%The share of the code that is copy-pasted
Accessibilityabout 10%Optional, off by default: static accessibility checks
Tests15%Only with a committed coverage report: measured line coverage

The overall score is the weighted average of the categories that could be analysed. A category that can't be analysed (for example dependencies, when there is no lockfile) is left out and the other weights are scaled up to match. Missing data never counts as zero, and it never counts as perfect either.

Tests deserve a note. Without a coverage report, Agnes only knows which source files have a test file next to them. That is shown on the project page, but it doesn't count towards the overall score, because “a test file exists” says very little about whether the code is tested. Once you commit a coverage report (lcov, Istanbul's json-summary, Cobertura or Clover), measured line coverage joins the score at 15%, reaching 100 at 80% coverage.

From findings to a number

Every finding has a severity and a confidence, and each gets a weight:

  • severity: error 3, warning 1, info 0.25;
  • confidence: high 1, medium 0.5, low 0.25.

A finding's penalty is the two multiplied, so a high-confidence error weighs 3 and a low-confidence info weighs 0.0625. A category's penalty is the sum of its findings. Findings a tool reports beyond what the report lists still count, otherwise a repository with more problems would score better.

The penalty then becomes a score with a decay curve:

score = 100 × e^(−penalty / decay)

At zero penalty the score is 100. At a penalty equal to the category's decay value it is about 37. It never goes below 0, and each extra finding costs a little less than the one before. The first problems are the ones that matter most, and a large legacy codebase doesn't end up at a meaningless zero.

How the penalty is measured depends on the category:

  • Code smells are counted per thousand lines (KLOC), because a bigger codebase naturally has more of them. The decay is 40 weighted findings per KLOC. Repositories under 1,000 lines count as 1 KLOC, so a tiny project isn't punished for one warning.
  • Accessibility works the same way, also with 40 per KLOC.
  • Dependencies and security are counted in absolute terms, with decays of 15 and 6. One vulnerable package is one risk whether the project has 2,000 lines or 200,000.
  • Duplication uses the duplicated percentage that jscpd reports, with a decay of 10%.

A worked example

Take a 20,000-line project with:

  • 10 errors and 120 warnings from the linters, all high confidence: a penalty of 10 × 3 + 120 × 1 = 150, or 7.5 per KLOC. Code smells: 100 × e^(−7.5/40) = 83.
  • Dependency findings adding up to a penalty of 5. Dependencies: 100 × e^(−5/15) = 72.
  • One committed secret, an error: a penalty of 3. Security: 100 × e^(−3/6) = 61.
  • 4% duplicated code. Duplication: 100 × e^(−4/10) = 67.

The overall score is the weighted average: 0.35 × 83 + 0.35 × 72 + 0.15 × 61 + 0.15 × 67 = 73.

Scores are shown in four bands: 80 and up is healthy, 60 to 79 fair, 40 to 59 needs work, and under 40 poor. This project is fair, and the single committed secret costs it more than you might expect. That's on purpose.

Which fix is worth the most

A score is only useful if it tells you what to do next. For each rule, Agnes recalculates the score as if all of that rule's findings were fixed, and ranks the rules by the points they would gain. A rule with 40 warnings in a small project can be worth more than one error in a big one, and the ranking shows that directly. AI agents connected to Agnes's MCP server get the same ranking, so they can plan the fixes with the most effect first.

What the score can't tell you

The score is static analysis of the code in your repository. It doesn't run your application, so it can't judge colour contrast, focus handling or anything that only happens in a browser. It doesn't run your tests. Accessibility findings are listed per WCAG success criterion, but they are not an accessibility audit or a compliance statement. A high score means the tools found little to complain about, not that the code is good. A low score means there is work to do, but not that the developers did a bad job: it says nothing about how much was shipped, or under what deadline.

What it is good for is direction: whether a repository is getting healthier or not, and what to fix first.

Free for open source

Agnes is built on open-source tools, so open source gets it for free. On every plan, up to 10 public repositories per team are analysed without counting towards the plan, with up to 100,000 lines each. A public repository can also get a public report page and a README badge that shows its score. Agnes's own html-validate-templates carries one.

The numbers will change

All of these values (the weights, the decays, the severity and confidence weights) sit together in one file in the Agnes codebase, so they can be tuned against real repositories. When they change, the changelog will say so, and scores may move without any change in your code. If a score looks wrong for your repository, I'd like to hear about it: support@agnes.dev.

Curious what your own repository scores? Connecting one takes a minute, and if the number stings a little, at least you'll know which fix to start with.