jolshan opened a new pull request #11459: URL: https://github.com/apache/kafka/pull/11459
With KAFKA-13102, we added topic IDs to the InitialFetchState and the PartitionFetchState in order to send fetch requests using topic IDs when ibp is 3.1. However, there are some cases where we could initially send topic IDs from the controller and then no longer to do so (controller changes to an IBP < 2.8). If we do not remove from the PartitionFetchState and one broker is still IBP 3.1, it will try to send a version 13 fetch request to brokers that no longer have topic IDs in the metadata cache. This could leave the cluster in a state unable to fetch from these partitions. This PR removes the topic IDs from the PartitionFetchState if the log contains a topic ID but the request does not. This means that we will always handle a leader and isr request if there is no ID in the request but an ID in the log. Such a state should be transient because we are either * upgrading the cluster and somehow switched between a new IBP controller and an old one --> and will eventually have all new IBP controllers/brokers. * downgrading the cluster --> will eventually have all old IBP controllers/brokers and will restart the broker/delete the partition metadata file for them. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
