358 MB resident, 52 MB in use: where the rest went

Certificate Transparency logs are append-only records of every TLS certificate a public CA issues, and I run a server that reads all of them and streams the certificates to whoever is listening. It had been sitting around 130 MB resident for months. I finally profiled it properly, left it running for two hours, and watched resident memory climb to 682 MB without levelling off.

The workload does not accumulate anything by design. There is a dedup cache with a fifteen minute window, a thousand-entry cache for the REST endpoint, and a broadcast channel holding a thousand messages. Add those up generously and you get maybe 60 MB.

So: 682 MB resident, 60 MB of explainable data structures, and a curve that has not flattened. That is what a leak looks like.

What resident memory actually measures

Before hunting for the leak it is worth being precise about what the number on that graph is. RSS is what the kernel charges the process for: pages that are mapped and backed by physical memory. It is not what the program is holding. An allocator that has freed an object internally still owns the page it sat on, and that page counts toward RSS until the allocator hands it back to the kernel.

Which means the graph cannot distinguish between two very different situations, and I had no way to tell them apart. So I stopped guessing and asked the allocator.

Asking the allocator

The server uses jemalloc, which tracks exactly how much memory the application holds and will tell you, if something asks. I wired six of its counters into the Prometheus endpoint:

certstream_jemalloc_allocated_bytes   // live heap: what the app holds
certstream_jemalloc_active_bytes
certstream_jemalloc_resident_bytes    // pages backed by physical memory
certstream_jemalloc_mapped_bytes
certstream_jemalloc_retained_bytes
certstream_jemalloc_metadata_bytes

One detail will bite you here: jemalloc caches these numbers and only refreshes them when you advance its epoch. Read them without that and you get the values from process start, which looks like a beautifully flat heap.

The figures below are from a shorter run on the same host once the counters were in, so resident is at 358 MB on its way up rather than the 682 MB the long run ended at. The ratio is the point.

Measure	Bytes
RSS, as the kernel sees it	358 MB
jemalloc resident	95 MB
jemalloc allocated, the live heap	52 MB

52 MB live, right where the arithmetic said it should be. Nothing was leaking. But the allocator only claimed 95 MB of resident pages while the kernel charged the process for 358 MB, and jemalloc's own accounting cannot explain that gap.

Where the memory was

/proc/pid/smaps_rollup answers this in one line:

Rss:              358240 kB
AnonHugePages:    206848 kB

206 MB of the 358 was transparent huge pages. The host had THP set to always:

$ cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

In that mode the kernel promotes anonymous mappings into 2 MiB pages wherever it can. jemalloc allocates and frees at 4 KiB granularity, and hands memory back a page at a time. A 2 MiB huge page with a single 4 KiB object still live in it cannot be returned. Under a workload that churns short-lived buffers across many threads, you accumulate huge pages that are mostly free and entirely unreturnable.

Two other defaults compounded it. jemalloc sizes its arena count as four times the CPU count, and inside a container that reads the host CPU count rather than the cgroup quota: 72 arenas on an 18 core machine, each with its own dirty page pool and decay clock, serving a runtime with four worker threads. And decay only advances when an arena is touched, so arenas belonging to bursty tasks sat on their dirty pages indefinitely.

The check that settled it

The same binary runs in production on a Linux box that had been up for twenty hours:

$ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never

$ grep -E '^(Rss|AnonHugePages)' /proc/$pid/smaps_rollup
Rss:              137436 kB
AnonHugePages:         0 kB

Zero huge pages, 137 MB resident, twenty hours of uptime. Same code, same workload, different THP setting.

That is also why nobody had reported it. Debian and Ubuntu ship THP in madvise mode, so their users never see it. Docker Desktop's Linux VM, RHEL derivatives and a fair number of cloud images use always, and their users see a process that appears to use four times the memory it does.

The fix

Five settings, embedded in the binary so they apply however it is installed:

thp:never,narenas:4,background_thread:true,dirty_decay_ms:5000,muzzy_decay_ms:5000

thp:never makes jemalloc mark its own mappings MADV_NOHUGEPAGE, opting the allocator out without touching anything else on the host. narenas:4 matches the runtime's actual thread count instead of the host's core count. background_thread:true gives idle arenas a purger, since the decay timer alone only fires when someone allocates.

In Rust with tikv-jemallocator, the way to ship a default is an exported symbol, remembering that the crate builds jemalloc with a _rjem_ prefix:

#[cfg(all(not(target_env = "msvc"), target_os = "linux"))]
#[used]
#[unsafe(export_name = "_rjem_malloc_conf")]
pub static MALLOC_CONF: &[u8] = b"thp:never,narenas:4,...\0";

Three things about that snippet. #[used] is load bearing: nothing in the crate reads the symbol, jemalloc looks it up at startup, and fat LTO will otherwise drop it. The Linux guard matters because macOS has no THP and no pthread background thread support, so passing those options there prints two warnings to stderr on every start. And jemalloc reads the _RJEM_MALLOC_CONF environment variable after the symbol, so operators can still override any of it without rebuilding.

What it did

Measure	Before	After
RSS, THP always host	358 MB	83 MB
RSS, THP madvise host	88 MB	77 MB
AnonHugePages	206 MB	6 MB
jemalloc retained	116 MB	34 MB
Arena metadata	11.8 MB	4.8 MB
Live heap	52 MB	49 MB

The live heap did not move, which is the point: nothing about how the program uses memory changed. The pages just started going back. The madvise host improved by about 11 MB too, which is the arena and decay settings doing their smaller share.

What to take from this

RSS answers what a process is charged for, not what it holds, and on a host with THP enabled those two can differ fourfold with nothing wrong. If your allocator reports its own live heap, export it next to RSS and put both on one graph.

Check /sys/kernel/mm/transparent_hugepage/enabled before believing a memory graph, especially one from a container on a developer laptop. A problem that only reproduces on always hosts looks intermittent and machine-specific, which is the worst kind to chase.

And if you swap allocators, configure it in the same commit. jemalloc had been the global allocator here for two releases with no configuration at all, running with an arena count derived from whatever machine it landed on and a decay policy that never ran on idle arenas. The swap was the right call; leaving it at defaults quietly undid most of the benefit.

The code is certstream-server-rust. The allocator settings and the jemalloc metrics landed in v1.5.5.