Hi! I'm trying to implement backpressure mechanism that asks producers to stop doing any work when consumers are not able to process all messages in time (producers require statistics calculated by consumers in order to answer client requests, when consumers are lagging behind we have to stop producers from making any responses).
I see several ways to implement it: - compute lag on consumer side and store it somewhere (zk, some db, etc); - use separate service like Burrow; - compute lag on every producer by getting commited and end offsets for every partition via consumer API. Are there any downsides of the latter approach? Would is negatively impact brokers performance? Thanks, Filipp.
