What is garbage collection?

Garbage collection is the mechanism by which a managed runtime, such as the JVM, Go, .NET, or Python, automatically frees memory occupied by objects the program can no longer reach, without the developer calling an explicit deallocation function. The point that confuses observers most often is the difference between reachability and usefulness: the collector does not decide what the program still "needs", it decides only what is still referenced from somewhere reachable from the roots. A large object can stay alive for months simply because some forgotten list still references it: this is a memory leak even in a garbage-collected language. The families in one line each: reference counting frees an object as soon as its count hits zero, but it does not resolve cycles on its own; mark-and-sweep walks the graph of reachable objects and frees the rest; generational approaches separate young objects from long-lived ones; concurrent collectors move most of the work onto threads alongside the application.

Pauses matter more than average throughput

The point that actually moves an architecture decision is not how much memory the collector frees on average, it is how long it stops the program to do it. A 200 millisecond pause does not show up in an average computed over thousands of fast requests, but it shows up in full in the tail of the latency distribution: it lands squarely in the p99, exactly the metric an SLO is built to defend. A well-instrumented observability setup shows the telltale signature: flat latency interrupted by periodic spikes that line up exactly with collection cycles, a pattern that a dashboard of averages alone hides completely. That is the real trade-off, and the three quantities do not maximize together: program throughput (how much useful work it does per second, net of time spent collecting), pause latency (how long, and how often, the program stops), and memory footprint (how much heap is needed to collect rarely instead of constantly). An interactive service that meets its SLO only on average does not meet it at all.

An enterprise example

A real-time pricing service runs in a container with a memory limit imposed by the Kubernetes cgroup, say two gigabytes. The runtime does not always know: the JVM and .NET read the cgroup limit by default and size the heap accordingly, while Go's runtime does not: without an explicit GOMEMLIMIT its collection threshold is just a multiple of the live heap, and there is no memory ceiling the collector accounts for. The result is not a more aggressive collection but one that does not speed up at all as the limit approaches: the process keeps allocating past the real threshold, and the kernel invokes the OOM killer, which terminates the container instantly, with no application log, no stack trace, and no chance for the collector to ever step in. From the dashboards it looks like a random crash; it is instead a collector that never saw the real constraint it was running under. The fix is not adding memory to the container, it is telling the runtime the limit it runs under. Reducing pauses is a separate goal, and it is addressed by the choice of collector, not by the memory limit.

Why it matters for decision makers

Garbage collection has become an architectural topic again, not a textbook footnote, precisely because of containers: a runtime designed for a dedicated physical machine now runs inside tight, shared memory limits. Whoever owns the architecture of a low-latency service should treat garbage collector configuration as part of the performance contract, not as a tuning knob to fix after the first incident: a p99 inflated by collection pauses stays invisible until someone goes looking for it with the right tool.