Will your site hold on sale day? "We autoscale" is not an answer, because autoscaling adds more of what you already have.
If the database has a hundred connection slots and the payment partner allows fifty requests a second, more app servers only reach those limits sooner. Most spike outages come from a fixed limit like that, one that always existed and never mattered on a normal day.
This is the five-step checklist I run in the two and a half weeks before a sale or launch: a load test that copies production, the limits that are not CPU, the cache when it is cold, memory, and the bill.
1. Load test with production data and the real URL mix
The most common load test I see is a tool pointed at the homepage with five hundred virtual users. It always passes, because the homepage sits in the CDN and the origin never notices.
A load test is a prediction about sale day, and it is only as good as what it copies from production:
- Use a recent, scrubbed copy of the production database. Ten thousand rows in staging keep every index in memory and every plan cheap. I wrote up how I do that in syncing production to local.
- Run once with a warm cache and once right after a flush.
- Spread the test across a couple of regions with some think time. Real shoppers on patchy 4G hold connections open for seconds and fill pools that a fast test machine never touches.
Most important, copy the URL mix. Pull a day of access logs, group by route, and replay routes in the same proportions.
Search, filtered listings and logged-in pages are where the origin does real work. A k6 script for that is short:
// k6 run spike.js (paths.json: [{ "path": "/search?q=...", "weight": 12 }, ...])
import http from "k6/http";
import { check } from "k6";
const paths = JSON.parse(open("./paths.json"));
const total = paths.reduce((s, p) => s + p.weight, 0);
export const options = {
stages: [
{ duration: "2m", target: 200 },
{ duration: "5m", target: 1000 }, // the spike
{ duration: "2m", target: 0 },
],
thresholds: { http_req_duration: ["p(95)<800"], http_req_failed: ["rate<0.01"] },
};
function pick() {
let r = Math.random() * total;
for (const p of paths) if ((r -= p.weight) <= 0) return p.path;
return paths[0].path;
}
export default function () {
check(http.get(`${__ENV.BASE_URL}${pick()}`), { ok: (r) => r.status < 400 });
}Watch the database while it runs. The k6 summary tells you something got slow; the database tells you why.
2. Find the limit that isn't CPU
When a site falls over under load, CPU is usually fine. Something with a fixed number of slots ran out first, and requests queued behind it until they timed out.
The database connection limit is the usual one. Postgres defaults to max_connections = 100 and MySQL to 151.
Ten app instances with a pool of twenty each want two hundred, and under normal traffic nobody notices the arithmetic is wrong. Count it on paper: instances at peak times pool size, plus workers, cron and the admin panel.
If the total is over the limit you need a pooler or smaller pools, which I cover in connection pooling with PgBouncer.
The others are quieter:
- Redis
maxclientsdefaults to 10,000, which runs out fast if every serverless instance opens its own client. - Every socket is a file, so a Node process with the common soft limit of 1,024 descriptors fails with
EMFILE, an error that looks nothing like a capacity problem. - Every payment gateway, SMS provider and shipping API on the checkout path has a rate limit in a PDF nobody on the team has read.
# During the load test: connections in use vs allowed (Postgres)
psql -c "select count(*) as in_use, current_setting('max_connections') as max from pg_stat_activity"
# Inside the app container, not on your laptop
ulimit -nEach one has a different fix. Raising the instance count fixes none of them.
3. Check the cache hit rate, then test it cold
A healthy cache hides how expensive the site is. At a 98% hit rate the database sees two requests in a hundred.
At 80% it sees twenty: ten times the load, from a change that looks small on a graph.
- 98% hit rate2 in 100
- 80% hit rate20 in 100
Get the steady-state number from INFO stats in Redis (keyspace_hits against keyspace_misses) or your CDN analytics. Then find out what the origin does in the first five minutes after the cache is empty, because it empties at the worst time.
Here's the catch: someone ships a fix for a wrong discount badge at 10:40 on sale day, the deploy changes a cache key prefix, and every page misses at once. Hot keys rebuild under full load, which is the pattern in cache stampedes.
So run the load test against a cold cache on purpose, and write a rule for the sale window: no deploy that changes cache keys. If one is unavoidable, warm the top keys from a script first.
4. Check memory graphs a week out
A process that grows 20 MB an hour looks fine on Tuesday and gets killed on Friday afternoon, at peak.
Look at fourteen days of memory graphs for every long-running process. A sawtooth that resets on each deploy is a leak hidden by your release cadence, and a deploy freeze gives it a full week to grow.
You can fix it with a heap snapshot and patience (finding a Node.js memory leak), or schedule rolling restarts through the event and fix it in December. I would take the restarts over an untested fix shipped two days before a sale.
5. Read the bill and agree on one dashboard
Go through each cost that grows with traffic and multiply it by five:
- egress
- NAT gateway processing (easy to multiply by accident, see the NAT gateway bill trap)
- the autoscaling maximum, which is also a spending limit
- APIs billed per call
If anyone is surprised, have that conversation now rather than in December, when finance asks why the bill tripled.
Agree on one dashboard before the day too:
- p95 and p99 per route
- error rate
- database connections against the limit
- hit rate
- queue depth
- process memory
Agree who watches it and who can roll back.
The worst incident call has six people building six graphs while the fix is a config change nobody is allowed to make. I also keep Core Web Vitals checks in CI, so the front end does not get heavier in the weeks before.
Conclusion
Replace "we autoscale" with an answer you have earned: "Tested at five times last year's peak. The connection pool broke first at about 3.5x; we fixed it and retested. No deploys from the 25th to the 30th. Here is the dashboard we will all watch at 9am."
If you do one thing this week, count your database connections on paper against max_connections, then read Postgres connection pooling with PgBouncer.
Which limit do you think your site hits first?