Catch Core Web Vitals regressions in CI

Performance regressions do not arrive as a project. They arrive one pull request at a time: a font added here, a widget dropped into a layout there, a component that was on the server quietly moved to the client. Each one is small, none of them is anybody's fault, and six weeks later the page is a second slower with no single commit to blame.

The fix is unglamorous. Measure on every pull request, and fail the ones that make it worse.

What CI can and cannot measure

Be precise about this, because a check that claims more than it measures loses the team's trust the first time it disagrees with reality.

CI answers one question well: did this change make the page worse than the last build? It does not answer are my users having a good time, because it is one machine on a simulated network. That second question belongs to field data, as I argued in the LCP guide — the lab tests a change, the field decides whether it mattered.

A workflow that is worth having

Build the app the way production builds it, serve it, run Lighthouse several times, assert:

name: Performance
on: pull_request

jobs:
  lighthouse:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }

      - run: npm ci
      - run: npm run build

      - name: Lighthouse CI
        run: |
          npm install -g @lhci/cli
          lhci autorun

And the configuration that makes it mean something:

module.exports = {
  ci: {
    collect: {
      startServerCommand: "npm run start",   // the production build, never the dev server
      url: [
        "http://localhost:3000/",
        "http://localhost:3000/blog",
        "http://localhost:3000/blog/some-representative-post",
      ],
      numberOfRuns: 3,                        // median of three; one run is noise
    },
    assert: {
      assertions: {
        "largest-contentful-paint": ["error", { maxNumericValue: 2500 }],
        "cumulative-layout-shift":  ["error", { maxNumericValue: 0.1 }],
        "total-blocking-time":      ["warn",  { maxNumericValue: 300 }],
        "resource-summary:script:size": ["error", { maxNumericValue: 200000 }],
        "unused-javascript":        ["warn",  { maxLength: 1 }],
      },
    },
    upload: { target: "temporary-public-storage" },
  },
};

Four details in there do most of the work.

Test the production build. A dev server is unminified, unbundled and carries a refresh runtime. Numbers from it are not wrong so much as unrelated.

Three runs, median. Lighthouse varies between runs on shared CI hardware. One run against a tight threshold produces a check that fails for no reason, and a check that fails for no reason gets re-run until it passes, which is the same as not having it.

A script size budget. resource-summary:script:size is the most stable assertion on the list — bytes do not vary with CPU contention. It also catches the most common regression: a dependency that quietly joined the client bundle. That is the number to watch if you only watch one, and finding what moved it is a separate exercise.

Pick representative URLs. Three pages that represent the real templates beat twenty that repeat one. A content page, an index, the home page.

Set thresholds you will actually honour

The common mistake is setting every budget at the "good" threshold on day one, watching the pipeline go red, and adding continue-on-error: true by Friday.

Set them from where you are:

1. Run Lighthouse on main and record the medians.
2. Set each threshold slightly above the current number — enough headroom that normal variance passes.
3. Mark the two or three you would genuinely block a merge for as error. Everything else warn.
4. Tighten the thresholds as the numbers improve, so they never drift back.

That last step is what makes this a ratchet rather than a dashboard. The budget follows the build down and never lets it climb back.

What to do when it goes red

A failing performance check should tell you what regressed, and the assertion name usually does. The common cases:

• CLS jumped. Something renders without reserved space — an image without dimensions, a lazily-loaded block, a late-arriving banner. The causes are enumerated in how to fix Cumulative Layout Shift.
• Script size jumped. A new dependency reached the client. Open the analyzer and find out which.
• LCP jumped with no size change. Usually the element changed, or the render path did — a hero moved into a client component, a font stopped being preloaded.

Each of those is a ten-minute fix in the pull request that caused it, and a half-day investigation three months later. That gap is the entire argument for the check.

The takeaway

Core Web Vitals work tends to be done in bursts: a bad report, a week of fixes, a quiet regression back to where it started. CI replaces the bursts with a ratchet.

Keep it honest and it stays useful: production build, several runs, median, a handful of budgets you would genuinely block a merge for, and the rest as warnings. Then treat the field data — Search Console and CrUX — as the thing that decides whether any of it reached your users, because the lab never gets to say that.