sijie commented on a change in pull request #4400: PIP 37: [pulsar-client]
support large message size
URL: https://github.com/apache/pulsar/pull/4400#discussion_r373326745
##########
File path:
pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java
##########
@@ -295,6 +295,32 @@
* @return the producer builder instance
*/
ProducerBuilder<T> enableBatching(boolean enableBatching);
+
+ /**
+ * If message size is higher than allowed max publish-payload size by
broker then enableChunking helps producer to
+ * split message into multiple chunks and publish them to broker
separately and in order. So, it allows client to
+ * successfully publish large size of messages in pulsar.
+ *
+ *
+ * This feature allows publisher to publish large size of message by
splitting it to multiple chunks and let
+ * consumer stitch them together to form a original large published
message. Therefore, it's necessary to configure
+ * recommended configuration at pulsar producer and consumer.
Recommendation to use this feature:
+ *
+ * <pre>
+ * 1. This feature is right now only supported by non-shared subscription
and persistent-topic.
Review comment:
This makes me feel that we are not on the right track for implementing this
feature. If a user is required to do these many things in order to enable this
feature, it seems that we are not doing something correctly. Also, this seems
to make the producer and consumer logic very complicated.
Can this chunking feature be implemented in a high level? E.g. over a
producer wrapper or a consumer wrapper? E.g. you can chunk a large message into
smaller messages and add the chunk id in the message properties.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
