codelipenghui commented on code in PR #21101:
URL: https://github.com/apache/pulsar/pull/21101#discussion_r1311460506
##########
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java:
##########
@@ -1437,6 +1437,26 @@ private ByteBuf processMessageChunk(ByteBuf
compressedPayload, MessageMetadata m
if (msgMetadata.getChunkId() == 0) {
if (chunkedMsgCtx != null) {
+ // Handle ack hole case:
+ // For example:
+ // Chunk-1 sequence ID: 0, chunk ID: 0, msgID: 1:1
+ // Chunk-2 sequence ID: 0, chunk ID: 1, msgID: 1:2
+ // Chunk-3 sequence ID: 0, chunk ID: 0, msgID: 1:3
+ // Chunk-4 sequence ID: 0, chunk ID: 1, msgID: 1:4
+ // Chunk-5 sequence ID: 0, chunk ID: 2, msgID: 1:5
+ // Consumer ack chunk message via ChunkMessageIdImpl that
consists of all the chunks in this chunk
+ // message(Chunk-3, Chunk-4, Chunk-5). The Chunk-1 and Chunk-2
are not included in the
+ // ChunkMessageIdImpl, so we should process it here.
+ boolean repeatedlyReceived =
Arrays.stream(chunkedMsgCtx.chunkedMessageIds)
+ .anyMatch(messageId1 -> messageId1 != null &&
messageId1.ledgerId == messageId.getLedgerId()
+ && messageId1.entryId ==
messageId.getEntryId());
+ if (!repeatedlyReceived) {
+
Arrays.stream(chunkedMsgCtx.chunkedMessageIds).forEach(messageId1 -> {
+ if (messageId1 != null) {
+ doAcknowledge(messageId1, AckType.Individual,
Collections.emptyMap(), null);
+ }
+ });
+ }
Review Comment:
The permits should also be increased?
increaseAvailablePermits(cnx);
##########
pulsar-broker/src/test/java/org/apache/pulsar/client/impl/MessageChunkingTest.java:
##########
@@ -356,6 +356,38 @@ public void testMaxPendingChunkMessages() throws Exception
{
assertNull(consumer.receive(5, TimeUnit.SECONDS));
}
+ @Test
+ public void testResendChunkMessagesWithoutAckHole() throws Exception {
+ log.info("-- Starting {} test --", methodName);
+ final String topicName =
"persistent://my-property/my-ns/testResendChunkMessagesWithoutAckHole";
+ final String subName = "my-subscriber-name";
+ @Cleanup
+ Consumer<String> consumer = pulsarClient.newConsumer(Schema.STRING)
+ .topic(topicName)
+ .subscriptionName(subName)
+ .maxPendingChunkedMessage(10)
+ .autoAckOldestChunkedMessageOnQueueFull(true)
+ .subscribe();
+ @Cleanup
+ Producer<String> producer = pulsarClient.newProducer(Schema.STRING)
+ .topic(topicName)
+ .chunkMaxMessageSize(100)
+ .enableChunking(true)
+ .enableBatching(false)
+ .create();
+
+ sendSingleChunk(producer, "0", 0, 2);
+
+ sendSingleChunk(producer, "0", 0, 2); // Resending the first chunk
+ sendSingleChunk(producer, "0", 1, 2);
+
+ Message<String> receivedMsg = consumer.receive(5, TimeUnit.SECONDS);
+ assertEquals(receivedMsg.getValue(), "chunk-0-0|chunk-0-1|");
+ consumer.acknowledge(receivedMsg);
+
assertEquals(admin.topics().getStats(topicName).getSubscriptions().get(subName)
Review Comment:
Please also add asserts for the flow permits.
##########
pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java:
##########
@@ -1437,6 +1437,26 @@ private ByteBuf processMessageChunk(ByteBuf
compressedPayload, MessageMetadata m
if (msgMetadata.getChunkId() == 0) {
if (chunkedMsgCtx != null) {
+ // Handle ack hole case:
+ // For example:
+ // Chunk-1 sequence ID: 0, chunk ID: 0, msgID: 1:1
+ // Chunk-2 sequence ID: 0, chunk ID: 1, msgID: 1:2
+ // Chunk-3 sequence ID: 0, chunk ID: 0, msgID: 1:3
+ // Chunk-4 sequence ID: 0, chunk ID: 1, msgID: 1:4
+ // Chunk-5 sequence ID: 0, chunk ID: 2, msgID: 1:5
+ // Consumer ack chunk message via ChunkMessageIdImpl that
consists of all the chunks in this chunk
+ // message(Chunk-3, Chunk-4, Chunk-5). The Chunk-1 and Chunk-2
are not included in the
+ // ChunkMessageIdImpl, so we should process it here.
+ boolean repeatedlyReceived =
Arrays.stream(chunkedMsgCtx.chunkedMessageIds)
+ .anyMatch(messageId1 -> messageId1 != null &&
messageId1.ledgerId == messageId.getLedgerId()
+ && messageId1.entryId ==
messageId.getEntryId());
Review Comment:
Do you think this check is necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
