massakam opened a new pull request #6211: (WIP) [Issue 5579] Fix bug that backlog message that has not yet expired could be deleted due to TTL URL: https://github.com/apache/pulsar/pull/6211 Fixes #5579 ### Motivation In Pulsar 2.4.1 and later versions, if message TTL is enabled, `PersistentMessageExpiryMonitor` always deletes one non-expired message every 5 minutes. The cause of this bug is https://github.com/apache/pulsar/pull/4744. `PersistentMessageExpiryMonitor` expects `ManagedCursor#asyncFindNewestMatching()` to pass null as its found position to itself as a callback if no expired messages exist. https://github.com/apache/pulsar/blob/c5ba52983fee994de61984aae7d1757e9b738caf/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentMessageExpiryMonitor.java#L119-L130 However, due to the change in https://github.com/apache/pulsar/pull/4744, if no entry is found that matches the search condition, the callback will be passed `startPosition` instead of null now. For this reason, the earliest backlog message is always deleted by `PersistentMessageExpiryMonitor`. This means that unexpected message loss can occur. ### Modifications Revert the https://github.com/apache/pulsar/pull/4744 changes. The motivation of https://github.com/apache/pulsar/pull/4744 is to avoid NPE caused in pulse-sql, but that seems to be fixed in https://github.com/apache/pulsar/pull/4757. https://github.com/apache/pulsar/blob/2069f761753940ed6a1faca8999af70036f20fd6/pulsar-sql/presto-pulsar/src/main/java/org/apache/pulsar/sql/presto/PulsarSplitManager.java#L363-L382
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
