Shrinking a 14 GiB Redis cache to 1.6 GiB, no downtime

A group of high-traffic content sites I work on share one Yii2 codebase and one production Redis node for page and query caching. Over time the keyspace grew to 13.97 GiB. Memory pressure meant evictions, evictions meant cache misses, and every miss was a MySQL round trip.

After adding transparent zlib compression the same keyspace measured ~1.56 GiB — a 9.14× reduction — with no downtime and no flag day. This post covers how.

The idea: compress inside the Redis serializer

Yii's cache component lets you swap the serializer pair. Instead of serialize() / unserialize() we plug in a codec with encode() / decode(). Every app — web, API, backend, console — gets it from one place, RedisCache::init(), so there is no config file that can be forgotten on one server class.

The important design rule: reads must always understand both formats. A value written before the rollout and a compressed value written after it must both decode.

An envelope you can recognise

Compressed values get a 4-byte header:

byte 0     0xC7   magic — never the first byte of PHP serialize() output
byte 1     0x01   compressor id: 1 = zlib
byte 2-3   0x0000 reserved: dictionary id
<payload>         gzcompress(serialize([$value, $dependency]), level)

PHP's serialize() output always starts with a type letter (a, O, s, i, d, b, N), so 0xC7 can never collide with a legacy value. decode() checks byte 0: magic means decompress, anything else falls through to plain unserialize().

Byte 1 is the future-proofing. Adding zstd later is a new compressor id, not a migration — values from both codecs decode side by side.

public function decode(string $raw): mixed
{
    if ($raw === '' || $raw[0] !== self::MAGIC) {
        return unserialize($raw); // legacy, uncompressed
    }
    $payload = substr($raw, 4);
    return unserialize(gzuncompress($payload));
}

Small values stay uncompressed

Below about 512 bytes the envelope and zlib framing stop paying for themselves, so CACHE_CODEC_MIN_SIZE keeps them as plain serialize() output.

Level 6, not level 1

The usual instinct is "level 1 for speed". For a read-heavy cache that is backwards. Higher levels produce less data to inflate, so decoding is faster:

Level 1 is worse on ratio and worse on the hot path. We use 6 and never go below 3.

Why it made things faster, not slower

• Decode costs less than the network. Shrinking a 44 KB value to ~3 KB saves ~338 µs on the wire at 1 Gbps, against ~37 µs of decode.
• Redis does less work. Compression runs in PHP on horizontally scaled app servers; Redis's single thread just moves fewer bytes in writeToClient.
• Evictions stop. This is the big one. With the keyspace at a ninth of its size, memory pressure disappears, the hit rate rises, and every recovered hit is a DB query that never happens.

Rolling it out across the Redis cache

Compression is controlled by a percentage dial, CACHE_CODEC_WRITE_PCT.

1. Preflight (read-only). A console script samples real values with SCAN + GET, checks for magic-byte collisions and reports the ratio and CPU cost. Proceed only on PREFLIGHT PASSED.
2. Readers only — WRITE_PCT=0. encode() returns byte-for-byte what Yii wrote before. Deploy everywhere and bake for 24 hours. Every host must be on this release before step 3, because an old host cannot read compressed values.
3. Ramp writes — 1%, 10%, 50%, 100% — watching the codec stats and Redis used_memory.
4. Rollback is setting the dial back to 0. Compressed values stay readable and age out naturally.

The same before-and-after discipline applies on the front end, where the levers are different but the method is identical: see Next.js performance.

Takeaways

• Put compression where every reader gets it, and ship readers before writers.
• Make formats self-describing with a magic byte and a version/compressor byte.
• Measure on real production values before choosing levels or thresholds.
• On a memory-bound cache, the hit-rate gain dwarfs the CPU cost.