merlimat commented on a change in pull request #5045: [client] Introduce
`batchingMaxBytes` setting in pulsar producer
URL: https://github.com/apache/pulsar/pull/5045#discussion_r317884638
##########
File path:
pulsar-client/src/main/java/org/apache/pulsar/client/impl/AbstractBatchMessageContainer.java
##########
@@ -37,11 +37,11 @@
protected ProducerImpl producer;
protected int maxNumMessagesInBatch;
+ protected int maxBytesInBatch;
protected int numMessagesInBatch = 0;
protected long currentBatchSizeBytes = 0;
- protected static final int INITIAL_BATCH_BUFFER_SIZE = 1024;
- protected static final int MAX_MESSAGE_BATCH_SIZE_BYTES = 128 * 1024;
+ protected static final int INITIAL_BATCH_BUFFER_SIZE = 128 * 1024;
Review comment:
Yes, initially we were using the max batch size for this buffer, but it was
using a huge amount of memory, especially when publishing on a partitioned
topic.
The buffer size is self-adapting to the needed usage (eg. it will keep using
the same buffer size), so I don't think we need to make it configurable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
