hanmz opened a new pull request, #21210: URL: https://github.com/apache/pulsar/pull/21210
### Motivation The configuration autoSkipNonRecoverableData is designed to turn this feature on if we can accept partial data loss. When some entries is lost, the broker will still work. `cursor.readPosition` will move to next position for skip these positions. But now we have this problem: `cursor.readPosition` can move to next positition, but the `cursor.markDeletePosition ` can not forward. for example: 1. topic-1 has some entries: [1:1],[1:2],[1:3],[1:4],[1:5],[1:6],[1:7],[1:8] 2. entry [1,4] and [1,5] lost 3. cursor read entry from ledger, then send message [1:1],[1:2],[1:3],[1:6],[1:7] to consumer. (Current: readPosition=[1:8], markDeletePosition=[1:1]) 4. consumer individual ack [1:1],[1:2],[1:3],[1:6],[1:7] (Current: readPosition=[1:8], markDeletePosition=[1:3]) 5. We can't delete the entry [1:4] and [1:5] because these lost. So markDeletePosition will stay at [1:3] and can not move forward to [1:8] The root cause is: When entries lost, lost entries not be delete from cursor. ### Modifications When skip lost entries, delete these entries from cursor. ### Documentation <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> - [ ] `doc` <!-- Your PR contains doc changes. --> - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later --> - [x] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-complete` <!-- Docs have been already added --> ### Matching PR in forked repository PR in forked repository: https://github.com/hanmz/pulsar/pull/4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
