dengziming opened a new pull request, #12265: URL: https://github.com/apache/kafka/pull/12265
*More detailed description of your change* There are 3 bugs when a broker generates a snapshot. 1. Broker should not generate snapshots until it starts publishing. Before a broker starts publishing, `BrokerMetadataListener._publisher=None`, so `_publisher.foreach(publish)` will do nothing, so `featuresDelta.metadataVersionChange().isPresent` is always true, so we will generating a snapshot on every commit, here are the logs, note offset 1 is a LeaderChangeMessage so there is no snapshot: ``` [2022-06-08 13:07:43,010] INFO [BrokerMetadataSnapshotter id=0] Creating a new snapshot at offset 0... (kafka.server.metadata.BrokerMetadataSnapshotter:66) [2022-06-08 13:07:43,222] INFO [BrokerMetadataSnapshotter id=0] Creating a new snapshot at offset 2... (kafka.server.metadata.BrokerMetadataSnapshotter:66) [2022-06-08 13:07:43,727] INFO [BrokerMetadataSnapshotter id=0] Creating a new snapshot at offset 3... (kafka.server.metadata.BrokerMetadataSnapshotter:66) [2022-06-08 13:07:44,228] INFO [BrokerMetadataSnapshotter id=0] Creating a new snapshot at offset 4... (kafka.server.metadata.BrokerMetadataSnapshotter:66) ``` 2. We should compute `metadataVersionChanged` before `_publisher.foreach(publish)` After `_publisher.foreach(publish)` the `BrokerMetadataListener_delta` is always Empty, so `metadataVersionChanged` is always false, this means we will never trigger snapshot generating due to metadata version change. 3. We should try to generate a snapshot when starting publishing When we started publishing, there may be a metadata version change, so we should try to generate a snapshot before first publishing. *Summary of testing strategy (including rationale)* A unit test. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
