gaozhangmin opened a new pull request, #4838: URL: https://github.com/apache/bookkeeper/pull/4838
Descriptions of the changes in this PR: Fixes: N/A Main Issue: N/A BP: N/A ### Motivation Scan-and-compare GC currently walks all ledger metadata ranges from ZooKeeper to decide whether locally active ledgers still exist in metadata. On deployments with many ledgers, fetching every ledger range can make `ScanAndCompareGarbageCollector.gc()` spend minutes in metadata operations even when the bookie only needs to check a small subset of local ledger buckets. ### Changes - Add a ledger metadata bucket watch/cache API to `LedgerManager`, with ZooKeeper-backed support in `AbstractZkLedgerManager`. - Cache child ledgers per metadata bucket and refresh bucket snapshots when ZooKeeper children watches fire, when a local ledger is first observed, when forced GC runs, or when the ZooKeeper session expires. - Make scan-and-compare GC use the bucket cache before falling back to the full metadata range scan, while preserving metadata verification semantics controlled by `verifyMetadataOnGc`. - Notify the ledger manager when ledgers first appear in interleaved and DbLedgerStorage local storage. - Add tests for bucket refresh, stale/unknown cache behavior, deleted buckets, cache pruning, GC cache path behavior, compaction, and ledger deletion timing. ### Verifications - `JAVA_HOME=$(/usr/libexec/java_home -v 17) mvn -pl bookkeeper-server -am -Dmaven.test.skip=true -Dexec.skip=true -Dmaven.antrun.skip=true compile` - `JAVA_HOME=$(/usr/libexec/java_home -v 17) mvn -pl bookkeeper-server -am -DskipTests -Dexec.skip=true -Dmaven.antrun.skip=true checkstyle:check` - Previously ran targeted GC/compaction/ledger-delete coverage locally on this branch. > --- > In order to uphold a high standard for quality for code contributions, Apache BookKeeper runs various precommit > checks for pull requests. A pull request can only be merged when it passes precommit checks. > > --- > Be sure to do all the following to help us incorporate your contribution > quickly and easily: > > If this PR is a BookKeeper Proposal (BP): > > - [ ] Make sure the PR title is formatted like: > `<BP-#>: Description of bookkeeper proposal` > `e.g. BP-1: 64 bits ledger is support` > - [ ] Attach the master issue link in the description of this PR. > - [ ] Attach the google doc link if the BP is written in Google Doc. > > Otherwise: > > - [x] Make sure tests pass via `mvn clean apache-rat:check install spotbugs:check`. > - [ ] Replace `<Issue #>` in the title with the actual Issue number. > > --- -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
