What is the difference between Lambda and Kappa architecture?

Lambda architecture and Kappa architecture are two ways of solving the same problem: having both an accurate historical view and a real-time view of data, without sacrificing one for the other. Lambda, proposed by Nathan Marz in 2011, runs two parallel paths: a batch layer that recomputes everything from scratch for maximum consistency, and a speed layer that processes only the recent increment at low latency; a serving layer merges the two views on read. Kappa, proposed by Jay Kreps in 2014, removes the batch layer: everything runs through a single stream, and when history needs recomputing you replay the log from the beginning instead of maintaining a second pipeline. The real fork between the two choices is not which one is more modern, but how many distinct distributed systems you are willing to keep aligned to get the same result, and how much historical recomputation matters relative to the volume of data your log can retain and replay.

See also: Batch vs streaming · Batch processes data in scheduled chunks, streaming as it arrives: the choice depends on how much data freshness is worth.

The real cost of each choice

Lambda's cost is not the infrastructure itself, it is the dual codebase: the same business logic has to be written and maintained twice, once in the batch engine and once in the streaming one, often in two different languages or frameworks. Jay Kreps put this in writing in his 2014 article "Questioning the Lambda Architecture": maintaining code that needs to produce the same result in two complex distributed systems is exactly as painful as it seems like it would be. Drift between the two implementations is nearly guaranteed over time. Kappa solves this duplication problem, but introduces its own: replaying the log from the beginning is realistic only if the log has been retained for the whole horizon you need (days, typically, not years) and if the data volume allows reprocessing it in an acceptable time. On Apache Kafka, where Kappa originated, this means setting a long retention, now less costly with tiered storage, or leaning on open table formats to keep history outside the log itself.

An enterprise example

A typical case: a logistics company computes average delivery times per route both in real time, to flag a delay while the truck is still on the road, and historically, for the commercial team's monthly report. With Lambda, a nightly batch job recomputes the exact averages across every event in the month, while a separate streaming job updates an estimate over the recent window that late-arriving events can only correct afterwards; both run the same calculation logic, written twice, with the concrete risk that a fix to the business logic lands in one engine and not the other. With Kappa, the same streaming job handles both cases: for the monthly report, a second instance of the job is launched that replays the log from the first day of the month and writes to a new table, which replaces the current one once it has caught up to real time. One engine, one logic, but recomputing the whole month is realistic only if the log covers that horizon.

Why it matters for decision makers

By 2026 the question is rarely framed the way it was in 2011 or 2014, because a data lakehouse built on open table formats serves both views from a single stack: transactions and queryable raw history turn historical recomputation into a backfill on the same stack, not a second pipeline. The operational choice today is more often "how much latency do I actually need" than "which of the two architectures do I adopt". That does not make Lambda dead: where historical recomputation is heavy to redo in streaming and historical data consistency is a contractual or regulatory constraint, a separate batch layer remains a defensible choice. But for anyone designing a new pipeline today, the right question to ask first is how many codebases you are willing to keep aligned over time, not which of the two architectures sounds more current.