liangyepianzhou commented on code in PR #20948:
URL: https://github.com/apache/pulsar/pull/20948#discussion_r1306762121
##########
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java:
##########
@@ -1449,6 +1450,24 @@ private ByteBuf processMessageChunk(ByteBuf
compressedPayload, MessageMetadata m
// discard message if chunk is out-of-order
if (chunkedMsgCtx == null || chunkedMsgCtx.chunkedMsgBuffer == null
|| msgMetadata.getChunkId() !=
(chunkedMsgCtx.lastChunkedMessageId + 1)) {
+ // Filter duplicated chunks instead of discard it. (Only do this
when exist duplication in a chunk message)
+ // For example:
+ // Chunk-1 sequence ID: 0, chunk ID: 0
+ // Chunk-2 sequence ID: 0, chunk ID: 0
+ // Chunk-3 sequence ID: 0, chunk ID: 1
+ if (chunkedMsgCtx != null && msgMetadata.getChunkId() <=
chunkedMsgCtx.lastChunkedMessageId) {
+ log.warn("[{}] Receive a repeated chunk messageId {},
last-chunk-id{}, chunkId = {}",
+ msgMetadata.getProducerName(),
chunkedMsgCtx.lastChunkedMessageId, msgId, msgMetadata.getChunkId());
+ compressedPayload.release();
+ increaseAvailablePermits(cnx);
+ boolean repeatedlyReceived =
Arrays.stream(chunkedMsgCtx.chunkedMessageIds)
+ .anyMatch(messageId1 -> messageId1 != null &&
messageId1.ledgerId == messageId.getLedgerId()
+ && messageId1.entryId ==
messageId.getEntryId());
+ if (!repeatedlyReceived) {
+ doAcknowledge(msgId, AckType.Individual,
Collections.emptyMap(), null);
Review Comment:
> This means we probably need to update the chunking uuid definition logic
and add a suffix there(session id, maybe the producer start-time, or some other
unique id to identify the producer session). Currently,
> ```
> String uuid = totalChunks > 1 ? String.format("%s-%d", producerName,
sequenceId) : null;
> ```
Yeah, this is a good suggestion. I changed it as follows.
```
long timestamp = System.currentTimeMillis();
for (int chunkId = 0; chunkId < totalChunks; chunkId++) {
....
String uuid = totalChunks > 1 ? String.format("%s-%d-%d", producerName,
sequenceId,
timestamp) : null;
....
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
