horizonzy opened a new pull request, #16590:
URL: https://github.com/apache/pulsar/pull/16590
Original issue: #13238
PIP Issue: #16569
### Motivation
In current ledger deletion, we divided it into two separate steps. It
happens in ManagedLedger and ManagedCursor.
Remove all the waiting to delete ledgers from the ledger list and update the
newest ledger list into a meta store.
In the meta store update callback operation, delete the waiting to delete
ledgers from storage systems, such as BookKeeper or Tiered storage.
Due to the separate step, we can’t ensure the ledger deletion transaction.
If the first step succeeds and the second step fails, it will lead to ledgers
that can't be deleted from the storage system forever. The second step may fail
by broker restart or storage system deletion failed.
In our customer’s environment, we have found many orphan ledgers cause by
the above reason.
### Modifications
Introduce two phase deletion to solve the problem.
Need to update docs?
- [ ] `doc-required`
(Your PR needs to update docs and you will update later)
- [ ] `doc-not-needed`
(Please explain why)
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-complete`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
