Skip to content
Sandeep Kumar
Updated 5 min read

What I check before a traffic spike

The list I go through in the weeks before a sale or launch: load tests that mean something, cache hit rates, connection limits, and the bill.

Sandeep Kumar, Full-stack EngineerSandeep KumarFull-stack Engineer
What I check before a traffic spike: Fixed limits that run out before CPU does
On this page
  1. 1. Load test with production data and the real URL mix
  2. 2. Find the limit that isn't CPU
  3. 3. Check the cache hit rate, then test it cold
  4. 4. Check memory graphs a week out
  5. 5. Read the bill and agree on one dashboard
  6. Conclusion

Will your site hold on sale day? "We autoscale" is not an answer, because autoscaling adds more of what you already have.

If the database has a hundred connection slots and the payment partner allows fifty requests a second, more app servers only reach those limits sooner. Most spike outages come from a fixed limit like that, one that always existed and never mattered on a normal day.

This is the five-step checklist I run in the two and a half weeks before a sale or launch: a load test that copies production, the limits that are not CPU, the cache when it is cold, memory, and the bill.

1. Load test with production data and the real URL mix

The most common load test I see is a tool pointed at the homepage with five hundred virtual users. It always passes, because the homepage sits in the CDN and the origin never notices.

A load test is a prediction about sale day, and it is only as good as what it copies from production:

  • Use a recent, scrubbed copy of the production database. Ten thousand rows in staging keep every index in memory and every plan cheap. I wrote up how I do that in syncing production to local.
  • Run once with a warm cache and once right after a flush.
  • Spread the test across a couple of regions with some think time. Real shoppers on patchy 4G hold connections open for seconds and fill pools that a fast test machine never touches.

Most important, copy the URL mix. Pull a day of access logs, group by route, and replay routes in the same proportions.

Search, filtered listings and logged-in pages are where the origin does real work. A k6 script for that is short:

js
// k6 run spike.js  (paths.json: [{ "path": "/search?q=...", "weight": 12 }, ...])
import http from "k6/http";
import { check } from "k6";

const paths = JSON.parse(open("./paths.json"));
const total = paths.reduce((s, p) => s + p.weight, 0);

export const options = {
  stages: [
    { duration: "2m", target: 200 },
    { duration: "5m", target: 1000 },   // the spike
    { duration: "2m", target: 0 },
  ],
  thresholds: { http_req_duration: ["p(95)<800"], http_req_failed: ["rate<0.01"] },
};

function pick() {
  let r = Math.random() * total;
  for (const p of paths) if ((r -= p.weight) <= 0) return p.path;
  return paths[0].path;
}

export default function () {
  check(http.get(`${__ENV.BASE_URL}${pick()}`), { ok: (r) => r.status < 400 });
}

Watch the database while it runs. The k6 summary tells you something got slow; the database tells you why.

2. Find the limit that isn't CPU

When a site falls over under load, CPU is usually fine. Something with a fixed number of slots ran out first, and requests queued behind it until they timed out.

The database connection limit is the usual one. Postgres defaults to max_connections = 100 and MySQL to 151.

Ten app instances with a pool of twenty each want two hundred, and under normal traffic nobody notices the arithmetic is wrong. Count it on paper: instances at peak times pool size, plus workers, cron and the admin panel.

If the total is over the limit you need a pooler or smaller pools, which I cover in connection pooling with PgBouncer.

The others are quieter:

  • Redis maxclients defaults to 10,000, which runs out fast if every serverless instance opens its own client.
  • Every socket is a file, so a Node process with the common soft limit of 1,024 descriptors fails with EMFILE, an error that looks nothing like a capacity problem.
  • Every payment gateway, SMS provider and shipping API on the checkout path has a rate limit in a PDF nobody on the team has read.
100Postgres max_connectionsdefault
151MySQL max_connectionsdefault
10,000Redis maxclientsdefault
1,024open files per Node processcommon soft limit
Fixed limits that run out before CPU does
bash
# During the load test: connections in use vs allowed (Postgres)
psql -c "select count(*) as in_use, current_setting('max_connections') as max from pg_stat_activity"

# Inside the app container, not on your laptop
ulimit -n

Each one has a different fix. Raising the instance count fixes none of them.

3. Check the cache hit rate, then test it cold

A healthy cache hides how expensive the site is. At a 98% hit rate the database sees two requests in a hundred.

At 80% it sees twenty: ten times the load, from a change that looks small on a graph.

Get the steady-state number from INFO stats in Redis (keyspace_hits against keyspace_misses) or your CDN analytics. Then find out what the origin does in the first five minutes after the cache is empty, because it empties at the worst time.

Here's the catch: someone ships a fix for a wrong discount badge at 10:40 on sale day, the deploy changes a cache key prefix, and every page misses at once. Hot keys rebuild under full load, which is the pattern in cache stampedes.

So run the load test against a cold cache on purpose, and write a rule for the sale window: no deploy that changes cache keys. If one is unavoidable, warm the top keys from a script first.

4. Check memory graphs a week out

A process that grows 20 MB an hour looks fine on Tuesday and gets killed on Friday afternoon, at peak.

Look at fourteen days of memory graphs for every long-running process. A sawtooth that resets on each deploy is a leak hidden by your release cadence, and a deploy freeze gives it a full week to grow.

You can fix it with a heap snapshot and patience (finding a Node.js memory leak), or schedule rolling restarts through the event and fix it in December. I would take the restarts over an untested fix shipped two days before a sale.

5. Read the bill and agree on one dashboard

Go through each cost that grows with traffic and multiply it by five:

  • egress
  • NAT gateway processing (easy to multiply by accident, see the NAT gateway bill trap)
  • the autoscaling maximum, which is also a spending limit
  • APIs billed per call

If anyone is surprised, have that conversation now rather than in December, when finance asks why the bill tripled.

Agree on one dashboard before the day too:

  • p95 and p99 per route
  • error rate
  • database connections against the limit
  • hit rate
  • queue depth
  • process memory

Agree who watches it and who can roll back.

The worst incident call has six people building six graphs while the fix is a config change nobody is allowed to make. I also keep Core Web Vitals checks in CI, so the front end does not get heavier in the weeks before.

Conclusion

Replace "we autoscale" with an answer you have earned: "Tested at five times last year's peak. The connection pool broke first at about 3.5x; we fixed it and retested. No deploys from the 25th to the 30th. Here is the dashboard we will all watch at 9am."

If you do one thing this week, count your database connections on paper against max_connections, then read Postgres connection pooling with PgBouncer.

Which limit do you think your site hits first?

FAQs

How far ahead should I prepare for a traffic spike?

Two to three weeks is enough for most sites. That leaves time to run a realistic load test, fix the first limit it finds, run it again, and then freeze risky deploys in the final week. Starting the day before only gives you time to raise instance counts, which rarely fixes the real bottleneck.

Why does my site pass load tests but fail under real traffic?

Usually because the test was too kind. It hit a handful of URLs that stayed in cache, used a small staging database, or ran from one region with no slow clients. Real traffic spreads across thousands of URLs, misses the cache more often and holds connections open longer, so the database and connection pools see far more work.

Should I scale up servers before a sale?

Scale what is actually limited. Adding application servers multiplies database connections, so if the database is the bottleneck more app servers make it worse. Find the first limit with a load test, then raise that limit, and pre-warm autoscaling rather than waiting for it to react.

What should I monitor during a traffic spike?

p95 and p99 latency per route, error rate, database connections in use against the maximum, cache hit rate, queue depth for background jobs and the memory of each process. Put them on one screen before the event so nobody is building dashboards while the site is slow.

Found this useful?Share on XShare on LinkedIn

Related reading