Hi Kafka Dev, With new Producer, we are having to change the # partitions for a topic, and we face this issue BufferExhaustedException.
Here is example, we have set 64MiB and 32 partitions and 1MiB of batch size. But when we increase the partition to 128, it throws BufferExhaustedException right way (non key based message). Buffer is allocated based on batch.size. This is very common need to set auto calculate batch size when partitions increase because we have about ~5000 boxes and it is not practical to deploy code in all machines than expand partition for scalability purpose. What are options available while new producer is running and partition needs to increase and not enough buffer to allocate batch size for additional partition ? buffer.memory=64MiB batch.size=1MiB block.on.buffer.full=false Thanks, Bhavesh
