I have question regarding GZIP compression. My env: I have enabled Compression.codec=1 essentially GZIP . Iam using ProducerData api on the producer side, We use following call in the code:
producer.send(new ProducerData<Integer, String>( topic, messageList)); messageList is list of messages , each message is a String. No of messages in messageList is 2000 always. Each message size is 1 KB . So total load in each call of producer.send is (2000 * 1 KB) max.message.size is set to 3000 KB. I use same value on the consumer side using fetch.size = 3000 KB . Question : 1. Will compression compress each message in the list or , will it compress whole of the list together as a single blob ? 2. max.message.size .In kafka will max.message.size checks for each message or does it checks with whole messageList size`? 3. Having a bigger messagelist size will give me better compression?? -rahul
