poorbarcode opened a new pull request, #23951:
URL: https://github.com/apache/pulsar/pull/23951
### Motivation
**Background**: the logic of calculating `backlog`
- There is a variable named `manegedledger.entriesAddedCounter`, which means
how many entries were added after the topic was loaded up at the latest time.
- there is a variable named `cursor.messagesConsumedCounter`, which means
how many entries were acked after the cursor was loaded up at the latest time,
it can be a negative value.
- Backlog calculating mechanism: `manegedledger.entriesAddedCounter -
cursor.messagesConsumedCounter`
**Steps to reproduce the issue**
- `ledgers: [ 1, 2, 3 ]`, `100` entries per ledger.
- `durable-cursor: {mark-deleted-position: 3:0}`
- `manegedledger.entriesAddedCounter`: `0`, no entries were added after the
topic was loaded up.
- concurrently trimming ledger and non-durable cursor creation
| time | `trim ledgers` | non-durable cursor creation |
| --- | --- | --- |
| 1 | | Init the variable `messagesConsumedCounter` by
`managedLegder.ledgers`: `-300` |
| 2 | Calculate ledgers to delete |
| 3 | Move non-durable cursors' mark deleted position |
| 4 | Skip the non-durable cursor because it has not been added to the list
`cursors` |
| 5 | Delete ledgers |
| 6 | | Receive messages `3:0~3:99` and ack them,
`cursor.messagesConsumedCounter += 100 (it will be -200)` |
| 7 | | issue: `cursor.backlog` will be `200`, but there is no message to
consume|
**The issue we encountered**
<img width="1093" alt="Screenshot 2025-02-08 at 17 35 40"
src="https://github.com/user-attachments/assets/50996a27-8810-4c3f-aefb-6c8e5c7b6e62";
/>
<img width="1259" alt="Screenshot 2025-02-08 at 17 35 18"
src="https://github.com/user-attachments/assets/d0ad0e6a-c073-4e50-8496-7e41480e1ec8";
/>
### Modifications
- fix the race condition: since trimming ledger is executed in a lock block
`ml.synchronized`, add a lock when creating a non-durable cursor
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
