cocktail828 commented on code in PR #806:
URL: https://github.com/apache/pulsar-client-go/pull/806#discussion_r921743447
##########
pulsar/producer_partition.go:
##########
@@ -520,7 +520,8 @@ func (p *partitionProducer) internalSend(request
*sendRequest) {
}
// if msg is too large
- if len(payload) > int(p._getConn().GetMaxMessageSize()) {
+ if len(payload) > int(p._getConn().GetMaxMessageSize()) &&
Review Comment:
Yep, this modification just an translation the implement of
`https://github.com/apache/pulsar/blob/782132561ac9fc8430ae3ef12913999e5871d3d2/pulsar-client-cpp/lib/ProducerImpl.cc#L431`.
It indeed do compress twice.
// Wire format
// [TOTAL_SIZE] [CMD_SIZE][CMD] [MAGIC_NUMBER][CHECKSUM] [PAYLOAD]
//
|
// compressed or uncompressed {[METADATA_SIZE_0]
[METADATA_0] [PAYLOAD_0]} ...
Acording to the wire format above, SDK cannot compress messages one by one
but should compress the whole batchs together.
Another way to avoid 'twice-compress', I think is flushing batchs
immediately before and after add big message to batchBuilder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
