← All Articles
Adaptive Kafka Consumer Concurrency with Lag Feedback
Dynamic worker scaling for consumers driven by lag slope and handler latency, with safeguards to avoid rebalance thrash.
12 min read
Static consumer concurrency is almost always wrong by lunchtime.
Feedback Inputs
- partition lag delta
- handler p95
- broker throttle signals
if (lagSlope > HIGH && p95 < 100) workers += 2;
if (p95 > 300) workers = Math.max(1, workers - 1);Scale conservatively to avoid rebalance storms.



