What is a service mesh and when is it actually worth it?

A service mesh is a dedicated infrastructure layer that manages communication between the services of a microservices architecture with well-drawn domain boundaries: it routes calls, encrypts traffic with mTLS, applies retries and timeouts, and collects telemetry on every exchange, without that logic living inside application code, and without every team having to rebuild the same plumbing on its own. It has historically been implemented with a lightweight proxy (a sidecar) next to every service instance, intercepting inbound and outbound traffic; since 2024 the ambient alternative, without a sidecar, moves the same function to node-level proxies built on eBPF, cutting the resource overhead and operational complexity the sidecar model carried. The mesh governs east-west traffic between internal services, while an ingress gateway governs north-south traffic coming in from outside the system: two distinct layers of the same network, frequently mixed up in the same conversation.

From sidecar to ambient

The sidecar pattern, popularized by Istio and Linkerd, injects a proxy into every pod: mature and battle-tested, but with a per-service cost in CPU, memory and latency that grows linearly with the number of instances, plus a proxy upgrade cycle to coordinate on every release. The ambient model exists to remove exactly that: a single per-node proxy handles traffic for multiple services, with an optional second layer for more advanced L7 features only where they are actually needed. The result is a lighter resource footprint and incremental adoption, since a service can join the mesh without a redeploy. It is not a free lunch, though: the control plane is still there, and with it the surface to understand, monitor and keep current with every new release of the chosen project.

An enterprise example

A bank breaking up a monolithic core banking system into around thirty microservices (payments, accounts, customer master data) introduces a service mesh to meet a specific compliance requirement: every internal call must be encrypted and authenticated, not only the ones exposed externally. The mesh applies end-to-end mTLS and produces a log of who called whom, with what outcome, useful both for audit and to quickly isolate a degrading service during an incident. The same mesh also applies circuit breaking: if the customer master data service responds slowly, the mesh throttles incoming requests before the whole payment flow cascades into failure. None of these rules live in the code of the individual services, which is precisely why the bank chose the mesh instead of implementing them thirty times over, once per service.

Why it matters for decision makers

A service mesh introduces an additional control plane, per-hop latency and operational skills to maintain over time, and this cost is fixed: it is paid whether there are ten stable services or a hundred. Below a certain scale, or when services change rarely and are owned by few teams, the operational cost of the mesh outweighs the benefit: mTLS between two or three services can be had with certificates managed directly inside the application, and retries can be written into a shared library. The question to ask before adopting one is not "does a service mesh solve this problem?", the answer is almost always yes, but "does the number of services and the teams that own them justify a dedicated control plane instead of a shared library?". It is the question no mesh vendor writes in its own documentation, because the honest answer rules out a good part of the market it sells to.