codelipenghui commented on code in PR #21187:
URL: https://github.com/apache/pulsar/pull/21187#discussion_r1461839137
##########
pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java:
##########
@@ -100,7 +100,11 @@ public void asyncReadEntriesOrWait(ManagedCursor cursor,
boolean isFirstRead,
ReadEntriesCallback callback, Consumer
consumer) {
PositionImpl cursorPosition;
- if (isFirstRead &&
MessageId.earliest.equals(consumer.getStartMessageId())){
+ boolean readFromEarliest = isFirstRead &&
MessageId.earliest.equals(consumer.getStartMessageId())
+ && (!cursor.isDurable() ||
cursor.getName().equals(Compactor.COMPACTION_SUBSCRIPTION)
+ || cursor.getMarkDeletedPosition() == null
+ || cursor.getMarkDeletedPosition().getEntryId() == -1L);
Review Comment:
What does this mean?
##########
pulsar-broker/src/test/java/org/apache/pulsar/client/impl/ReaderTest.java:
##########
@@ -785,4 +785,32 @@ public void testReaderListenerAcknowledgement()
admin.topics().deletePartitionedTopic(partitionedTopic);
}
+ @Test
+ public void testReaderReconnectedFromNextEntry() throws Exception {
Review Comment:
I tried to run this test without this fix many times.
But the test can get passed.
<img width="1649" alt="image"
src="https://github.com/apache/pulsar/assets/12592133/197cdb56-49c8-44b8-af0e-9e273b145c6b";>
Could you please check if the test can actually cover the issue that this PR
try to fix?
##########
pulsar-broker/src/test/java/org/apache/pulsar/compaction/CompactionTest.java:
##########
@@ -2206,4 +2210,158 @@ public void testCompactionWithTTL() throws Exception {
Assert.assertEquals(result, List.of("V3", "V4", "V5"));
}
+
+ @Test
+ public void testAcknowledgeWithReconnection() throws Exception {
+ final String topicName =
"persistent://my-property/use/my-ns/testAcknowledge" + UUID.randomUUID();
+ final String subName = "my-sub";
+ @Cleanup
+ PulsarClient client = newPulsarClient(lookupUrl.toString(), 100);
+ Producer<String> producer = pulsarClient.newProducer(Schema.STRING)
+ .enableBatching(false).topic(topicName).create();
+
+ List<String> expected = new ArrayList<>();
+ for (int i = 0; i < 10; i++) {
+
producer.newMessage().key(String.valueOf(i)).value(String.valueOf(i)).send();
+ expected.add(String.valueOf(i));
+ }
+ producer.flush();
+
+ admin.topics().triggerCompaction(topicName);
+
+ Awaitility.await().untilAsserted(() -> {
+ assertEquals(admin.topics().compactionStatus(topicName).status,
+ LongRunningProcessStatus.Status.SUCCESS);
+ });
+
+ // trim the topic
+ admin.topics().unload(topicName);
+
+ Awaitility.await().untilAsserted(() -> {
+ PersistentTopicInternalStats internalStats =
admin.topics().getInternalStats(topicName, false);
+ assertEquals(internalStats.numberOfEntries, 0);
+ });
+
+ ConsumerImpl<String> consumer = (ConsumerImpl<String>)
client.newConsumer(Schema.STRING)
+
.topic(topicName).readCompacted(true).receiverQueueSize(1).subscriptionName(subName)
+
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
+ .isAckReceiptEnabled(true)
+ .subscribe();
+
+ List<String> results = new ArrayList<>();
+ for (int i = 0; i < 5; i++) {
+ Message<String> message = consumer.receive(3, TimeUnit.SECONDS);
+ if (message == null) {
+ break;
+ }
+ results.add(message.getValue());
+ consumer.acknowledge(message);
+ }
+
+ Awaitility.await().untilAsserted(() ->
+ assertEquals(admin.topics().getStats(topicName,
true).getSubscriptions().get(subName).getMsgBacklog(),
+ 5));
+
+ // Make consumer reconnect to broker
+ admin.topics().unload(topicName);
+
+ // Wait for consumer to reconnect and clear incomingMessages
+ consumer.pause();
+ Awaitility.await().untilAsserted(() -> {
+ Assert.assertEquals(consumer.numMessagesInQueue(), 0);
+ });
+ consumer.resume();
+
+ for (int i = 0; i < 5; i++) {
+ Message<String> message = consumer.receive(3, TimeUnit.SECONDS);
+ if (message == null) {
+ break;
+ }
+ results.add(message.getValue());
+ consumer.acknowledge(message);
+ }
+
+ Awaitility.await().untilAsserted(() ->
+ assertEquals(admin.topics().getStats(topicName,
true).getSubscriptions().get(subName).getMsgBacklog(),
+ 0));
+
+ Assert.assertEquals(results, expected);
+
+ Message<String> message = consumer.receive(3, TimeUnit.SECONDS);
+ Assert.assertNull(message);
+
+ // Make consumer reconnect to broker
+ admin.topics().unload(topicName);
+
+ producer.newMessage().key("K").value("V").send();
+ Message<String> message2 = consumer.receive(3, TimeUnit.SECONDS);
+ Assert.assertEquals(message2.getValue(), "V");
+ consumer.acknowledge(message2);
+
+ Awaitility.await().untilAsserted(() -> {
+ PersistentTopicInternalStats internalStats =
admin.topics().getInternalStats(topicName);
+ Assert.assertEquals(internalStats.lastConfirmedEntry,
+ internalStats.cursors.get(subName).markDeletePosition);
+ });
+
+ consumer.close();
+ producer.close();
+ }
+
+ @Test
+ public void testEarliestSubsAfterRollover() throws Exception {
Review Comment:
This test can also get passed without this fix.
@coderzc Could you please check if the test can cover the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
