Apache Kafka® Meetup Pune

Kalpesh S. Deo - One2N

Talk: Benchmarking Kafka performance with kafka-producer-perf-test

We explore how to benchmark Kafka performance using kafka-producer-perf-test.sh with different parameter combinations. By automating tests across settings such as compression.type, linger.ms, and batch.size, we improved producer throughput by 245%, cut latency by 90%, and reduced storage by 89% in one workload. This approach makes it easier to find a good configuration for your environment.

Takeaways

• How these producer settings affect throughput, latency, and storage.

• How to automate benchmark runs across many configurations.

• How to read results and apply them to your own Kafka deployment.

Bio: Kalpesh is a tech lead with 16+ years of experience. At One2N he leads backend work in Go and Java across cloud-native systems.

Suyog Chadawar - Confluent

Talk: The architectural evolution of a data platform: from batch processing to real-time AI

We walk through how a data platform for modern recommendation systems evolved from batch (Spark, Hive) to real-time ingestion with Kafka, Flink for in-session personalization, Apache Iceberg for a lakehouse, and declarative tooling - plus operationalizing insights with reverse ETL.

Bio: Suyog is a Senior Software Engineer at Confluent, focused on backend and distributed systems.