GKE Pod snapshots restore a loaded 70B model in 37 seconds, Google says
Victoria Shinder2 min readGoogle has introduced Pod snapshots for Google Kubernetes Engine, a feature that saves the running state of a workload - including CPU and GPU memory - and restores it on demand. In its announcement, Google says the approach cuts AI inference start-up by as much as 89%, loading a 70-billion-parameter model in 37 seconds and an 8-billion-parameter model in 15 seconds in its benchmarks. InfoQ reported on 27 September that the feature has reached general availability.
The target is the cold start. A new inference replica normally has to start its server, download model weights and load them into accelerator memory, which for large models can take minutes and pushes operators to keep spare capacity running. With Pod snapshots, that initialisation is done once; GKE captures the fully loaded state and stores it in Cloud Storage, and new replicas restore directly from it. The same mechanism is offered for agent sandboxes: capture a prepared sandbox once and start new ones from it, or suspend an idle sandbox and resume it when needed. Snapshots are configured declaratively through custom resources that define which Pods to snapshot and where to store the data, can be taken at start-up or on demand, and have retention settings.
Google quotes one customer, the photo-editing platform Retake by Codeway, which says it replaced a custom caching layer and cut start-up latency on H100 GPUs from about a minute to 8 seconds.
What it means
Checkpoint and restore of a whole process is an old idea; applying it to GPU memory inside a managed Kubernetes service is what makes it practical for inference. The economics are simple: if a replica can be ready in seconds rather than minutes, autoscaling can follow demand instead of anticipating it, and less expensive accelerator capacity sits idle.
The caveats are the ones that come with any snapshot. A snapshot captures state at a point in time, so anything that must be fresh - credentials, connections, per-request data - has to be handled on restore, and restoring many replicas from one image means they start identical. The benchmark figures are Google's own and depend on model size, hardware and storage throughput. The feature is also specific to GKE; teams that want portability across clouds are adopting a platform capability, not a Kubernetes standard.