Stream processing
Stream processing is computing results continuously from an unbounded flow of events as they arrive, rather than collecting data and processing it later in batches. Typical inputs are Kafka topics or change data capture feeds. Typical outputs are alerts, aggregates and updated tables within seconds. Apache Flink, Kafka Streams and Spark Structured Streaming are common engines.
How it works
A streaming job reads events, keeps state such as running counts, and groups events into windows. A tumbling window cuts time into fixed, non-overlapping slices. For example, a job counts payments per merchant in 1-minute windows from 12,000 events per minute. Events carry their own event time, and a watermark tells the job when a window is complete. With 10 seconds of allowed lateness, an event that arrives 30 seconds late misses its window and is dropped or sent to a side output.
Why it matters in production
Streaming cuts the delay from event to insight from hours to seconds, but it adds always-on state, ordering and replay problems that batch jobs avoid. Teams need consumer lag alerts, a plan for late and duplicate events, and checkpointing so a restarted job resumes without loss.
See also
On the One2N blog
How temporal versioning prevents configuration drift in stream processing
Shooting ourselves in the foot by enabling Spring Kafka batching
How we reduced Kafka message latency by 90% and increased throughput 4x
Further reading
