Re: Producer batch size

2020-12-23 Thread Steve Howard
Hi Dhirenda, As long as buffer.memory (default 32MB) has space, the producer will continue to write here. If that is exhausted, eventually the producer will throw... org.apache.kafka.common.errors.TimeoutException: Failed to allocate memory within the configured max blocking time 6 ms The 6

Re: RE: RE: RE: Maintaining same offset while migrating from Confluent Replicator to Apache Mirror Maker 2.0

2020-12-23 Thread Ning Zhang
Great - that sounds a smart way of bridging the Replicator and MM2. Seems like even though the consumer group can be same between Replicator and MM2, the storage format of offsets are still little different, so we need an "adapter" anyway. Just need to monitor the "adapter" is constantly runni

Re: Producer batch size

2020-12-23 Thread Dhirendra Singh
Thanks steve ! So if I understand correctly, the number of messages buffered can be greater than batch.size upto buffer.memory if the app is sending data faster than the producer i/o thread can send to broker. In this situation buffer.size becomes irrelevant. no ? Thanks, Dhirendra. On Wed, Dec 2