The logic you're requesting is basically what the new producer implements.
The first condition is the batch size limit and the second is linger.ms.
The actual logic is a bit more complicated and has some caveats dealing
with, for example, backing off after failures, but you can see in this code
*bump*
On Tue, Jun 23, 2015 at 1:03 PM, Achanta Vamsi Subhash
achanta.va...@flipkart.com wrote:
Hi,
We are using the batch producer of 0.8.2.1 and we are getting very bad
latencies for the topics. We have ~40K partitions now in a 20-node cluster.
- We have many topics and each with
Hi,
We are using the batch producer of 0.8.2.1 and we are getting very bad
latencies for the topics. We have ~40K partitions now in a 20-node cluster.
- We have many topics and each with messages published to them varying. Ex:
some topics take 10k/sec and other 2000/minute.
- We are seeing
