[
https://issues.apache.org/jira/browse/KAFKA-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17350684#comment-17350684
]
Justine Olshan commented on KAFKA-12835:
----------------------------------------
Hi [~ivanyu]. Previously we could lose topic IDs in the Znode when reassigning
partitions. This could occur if we switched between controllers with IBP 2.8
back to 2.7 (where we reassign partitions) and back to 2.8. If the
partition.metadata file still existed, we would have the old ID in that, but
the new controller would see a topic ID missing in the ZNode and assign a new
one.
I've opened a PR to prevent this loss of topic ID regardless of the IBP of the
controller.
> Topic IDs can mismatch on brokers (after interbroker protocol version update)
> -----------------------------------------------------------------------------
>
> Key: KAFKA-12835
> URL: https://issues.apache.org/jira/browse/KAFKA-12835
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.8.0
> Reporter: Ivan Yurchenko
> Assignee: Justine Olshan
> Priority: Major
>
> We had a Kafka cluster running 2.8 version with interbroker protocol set to
> 2.7. It had a number of topics and everything was fine.
> Then we decided to update the interbroker protocol to 2.8 by the following
> procedure:
> 1. Run new brokers with the interbroker protocol set to 2.8.
> 2. Move the data from the old brokers to the new ones (normal partition
> reassignment API).
> 3. Decommission the old brokers.
> At the stage 2 we had the problem: old brokers started failing on
> {{LeaderAndIsrRequest}} handling with
> {code:java}
> ERROR [Broker id=<...>] Topic Id in memory: <...> does not match the topic Id
> for partition <...> provided in the request: <...>. (state.change.logger)
> {code}
> for multiple topics. Topics were not recreated.
> We checked {{partition.metadata}} files and IDs there were indeed different
> from the values in ZooKeeper. It was fixed by deleting the metadata files
> (and letting them be recreated).
>
> The logs, unfortunately, didn't show anything that might point to the cause
> of the issue (or it happened longer ago than we store the logs).
> We tried to reproduce this also, but no success.
> If the community can point out what to check or beware of in future, it will
> be great. We'll be happy to provide additional information if needed. Thank
> you!
> Sorry for the ticket that might be not very actionable. We hope to at least
> rise awareness of this issue.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
