Sleep mode: scale to zero
An application nobody is using should cost next to nothing, and be back on the next request. Sleep mode does that: an idle application is first frozen, then stopped, and the next request wakes it. It is off by default and chosen application by application.
Three states
Active
The application runs normally and serves requests.
Frozen
The process is paused by the Linux cgroup freezer. Its memory stays resident and its TCP connections stay open; waking it is a thaw.
Stopped
The systemd unit is stopped: memory, private network and user ID are given back. The instance stays desired and placed in the cluster (its host port is kept); waking it starts the unit again, so the application restarts from scratch.
How it behaves
- Opt-in per application, through the sleep setting, off by default. An application without it is never touched: no Raft write, no activity tracking.
- While an application wakes, the ingress holds the request and forwards it exactly once. If the wake takes longer than wakeTimeoutMs (5000 ms by default), the client gets a 503 with Retry-After and the start carries on, so the next request succeeds.
- Two idle timers: freezeAfterSeconds (60 s by default) and stopAfterSeconds (900 s by default). The effective freeze delay is never shorter than twice the activity-report period (5 s by default).
- With several Higgs orchestrators, an application sleeps only if every ingress sees it idle: the cluster keeps the most recent activity reported by all of them, and only the Raft leader decides, with one write per transition and never one per request.
- One request wakes one replica, a frozen one first. Other replicas wake only when load calls for it, following the sleep.wake policy: requests in flight per replica, a slow wake with requests waiting, or a request rate. Other nodes are used first.
- Photon webhooks and service-to-service calls (the GLUONIFY_SERVICE_<APP>_URL variable) go through the internal ingress and wake the application too. Health probes (/q/...) never do.
- A stopped application can only be observed through Strange: logs, last cpu and memory values, state and since when. Nothing runs that could be scraped. Calorimeter shows it as asleep, not down, so there is no false availability alert.
- Not for applications with background duties (scheduled jobs, queue consumers, outgoing WebSockets): a frozen or stopped application does nothing. The container runtime containerd cannot freeze: the instance stays active and the failure is reported.
The figures
One machine: an Apple M4 Max, running the Linux VM of Docker Desktop (kernel 7.0.14, arm64, privileged container). Application: gluonify-probe, a minimal native Quarkus application without database. Reference run (the latest): 10 wakes per case through the ingress, a new connection for each request, the application frozen after 3 s and stopped after 8 s of idleness. Times come from one clock, from the sending of the request to the first byte of the answer. The host was busy: macOS load average 12.54 12.34 12.65 before the run, 6.96 9.16 11.09 after.
A burst of 50 requests on a stopped application causes a single start. The whole burst is served in 244.3 ms and the slowest first byte arrives at 240.5 ms (earlier run: 186.9 ms and 183.3 ms).
Extra time added by the ingress on a cold start, before and after the ready signal is pushed at once (median, 20 runs each; p95 4,851.1 ms then 23.2 ms). Before, the ingress learned that the application was ready only at the next 5-second cycle of the agent.
The first wake of the reference run, frozen then stopped: a single measurement each, not a median. In the earlier run the first stopped wake took 187.6 ms: single wakes vary a lot.
Where it started
Before sleep mode, a stopped application was only restarted at the next 5-second cycle of the agent: from the stop to the first 200 through the ingress took a median of 6,978.9 ms (20 runs). Pushing the ready signal at once brought it to 2,113.8 ms, which still includes the wait for that cycle. Sleep mode starts the unit as soon as a request arrives: 271.4 ms median in the reference run. A frozen application thawed by the bench itself answered in 5.7 ms (p95 6.1 ms), with no ingress able to wake it. The three bars come from different benches (20 and 10 runs, first 200 and first byte): they show orders of magnitude, not a controlled comparison.
What a start costs
A stopped application restarts after about 155 ms of preparation (median 154.4 ms, 20 runs), plus the start of the application itself. Two steps dominate: creating the private network namespace (netns, 90.64 ms) and launching the unit (systemd-run, 52.62 ms). Getting configuration and secrets from the top costs about 1 ms (0.8 ms, plus 0.0 ms for the Charm token, with 10 settings and 5 secrets). The steps are medians: they do not add up to the total.
Memory while frozen
A frozen application keeps all its memory: its resident size (RSS), proportional size (PSS) and the cgroup memory are the same as at rest. Asking the kernel to push its pages out with memory.reclaim, to compressed RAM (zswap) or to the swap disk of the VM, gives back only part of it and makes the resume slower. This is off by default (gluonify.strange.process.reclaim-on-freeze).
kB for memory, milliseconds for the resume (request to 200), 20 runs per row. Freshly started: all pages resident before each run; successive runs: only the pages touched by the request came back, which is closer to a long-idle application. The swap disk is the virtual disk of the VM, probably served by the cache of the VM: the latency of a real node disk is not measured. Swapped pages are application memory in clear text on disk.
Saving to disk: what was evaluated
CRIU, which checkpoints a process to disk, does not work in the lab: the kernel of the Docker Desktop VM lacks userfaultfd (so no lazy pages) and refuses the socket options of a listening TCP server, the dump of the real unit failed (inherited mounts, then a crash of the injected helper), and the restore of a simple process hung for more than 15 s (one try). MicroVMs (Firecracker, QEMU) and gVisor checkpointing were not tested. CRaC does not apply to native images. With a restart of about 0.2 s for a light native application, none of them is pursued; a complete kernel or a heavier application could change that, which is not measured.
On this setup, a frozen application answers its first request in a median of 8.8 ms (p95 19.6 ms) and a stopped one in 271.4 ms (p95 472.4 ms), against 0.3 ms for a request to an application that is awake. Freezing keeps the memory; stopping gives it back for the price of a restart. The earlier run gave 8.2 ms and 211.6 ms: the spread between runs is real.
gDown, the graph database of the platform, does the same for its databases: an idle database gives its memory back (UNLOAD DATABASE, or an idle delay, 900 s advised on the platform) and is reloaded on its first request, in 82 ms for 99 MB of files and 549 ms for 982 MB on a Mac (arm64). Sleeping databases in gDown
Limits: one machine (Apple M4 Max), inside the Linux VM of Docker Desktop and not on bare metal; a minimal native application only (no JVM, no database, no migrations), so a real application will start more slowly, which is not measured; a busy host (see the load averages above) where other sessions and Docker builds may have been running; 10 wakes per case in the sleep bench, 20 runs elsewhere; one Higgs orchestrator with a local Raft state, so the Raft round trip is that of a one-member cluster. The rule for several Higgs orchestrators ("every ingress agrees") is covered by unit tests only, and the per-ingress load counters that wake extra replicas are not tested end to end with several Higgs orchestrators (the replica policy itself is verified on two nodes by e2e-sleep.sh). The containerd runtime cannot freeze. No extrapolation to other applications, disks or bare metal.
Source: gluonify-chamber, scripts/bench-coldstart.sh, bench-sleep.sh and e2e-sleep.sh; raw results in gluonify-codex, bench/sleep-impl-2026-10-06-after-ingress-uses.json (reference run), sleep-impl-2026-10-06.json, sleep-2026-10-06.json, coldstart-2026-10-06.json and coldstart-2026-10-06-ready-push.json; design in ARCHITECTURE.md and operations in docs/OPERATIONS.md.