aloyszhang opened a new pull request, #19581: URL: https://github.com/apache/pulsar/pull/19581
### Motivation There're some cases of consumers receiving duplicated messages when there are ongoing transactions within a subscription. #14327 has fixed this problem partially. But there's still a chance for consumers to receive duplicated messages. 1. For non-batch messages, the dispatcher does not filter messages in the pendingAck state(PendingAckHandleImpl#individualAckPositions) 2. For batch messages, if the ackSet in pendingAck state is complimentary with the ackSet in the cursor(the result of the two ackSet AND is empty), the broker still dispatches this entry to the consumer. For example, a batch message has 5 messages internal. - cursor stats has acked 0,1,2 - pendingAck stats has acked 3,4 This message should not dispatch to the consumer. ### Modifications 1. filter messages in pendingAck for non-batch messages 2. filter messages ackSet in pendingAck state is complimentary with the ackSet in the cursor for batch messages ### Verifying this change - [ ] Make sure that the change passes the CI checks. ### Documentation - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs --> ### Matching PR in forked repository PR in forked repository: <!-- ENTER URL HERE --> <!-- After opening this PR, the build in apache/pulsar will fail and instructions will be provided for opening a PR in the PR author's forked repository. apache/pulsar pull requests should be first tested in your own fork since the apache/pulsar CI based on GitHub Actions has constrained resources and quota. GitHub Actions provides separate quota for pull requests that are executed in a forked repository. The tests will be run in the forked repository until all PR review comments have been handled, the tests pass and the PR is approved by a reviewer. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
