jsancio commented on PR #13102:
URL: https://github.com/apache/kafka/pull/13102#issuecomment-1457138403
I was thinking about this change today and realized that this change is not
backwards compatible. For example, the user upgrades the cluster and then they
downgrade the cluster, it will stop working. For example, I got the following
error if the `clusterId` is removed:
```
[2023-03-06 14:31:09,673] ERROR [SharedServer id=1] Got exception while
starting SharedServer (kafka.server.SharedServer)
java.lang.RuntimeException: QuorumStateData: unable to locate field
'clusterId', which is mandatory in version 1
at
org.apache.kafka.raft.generated.QuorumStateDataJsonConverter.read(QuorumStateDataJsonConverter.java:40)
at
org.apache.kafka.raft.FileBasedStateStore.readStateFromFile(FileBasedStateStore.java:94)
at
org.apache.kafka.raft.FileBasedStateStore.readElectionState(FileBasedStateStore.java:110)
at org.apache.kafka.raft.QuorumState.initialize(QuorumState.java:116)
at
org.apache.kafka.raft.KafkaRaftClient.initialize(KafkaRaftClient.java:387)
at kafka.raft.KafkaRaftManager.buildRaftClient(RaftManager.scala:256)
at kafka.raft.KafkaRaftManager.<init>(RaftManager.scala:179)
at kafka.server.SharedServer.start(SharedServer.scala:245)
at
kafka.server.SharedServer.startForController(SharedServer.scala:137)
at kafka.server.ControllerServer.startup(ControllerServer.scala:332)
at
kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:134)
at
kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:134)
at scala.Option.foreach(Option.scala:437)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:134)
at kafka.Kafka$.main(Kafka.scala:114)
at kafka.Kafka.main(Kafka.scala)
[2023-03-06 14:31:09,675] INFO Waiting for controller quorum voters future
(kafka.server.ControllerServer)
[2023-03-06 14:31:09,676] INFO Finished waiting for controller quorum voters
future (kafka.server.ControllerServer)
[2023-03-06 14:31:09,677] ERROR [ControllerServer id=1] Fatal error during
controller startup. Prepare to shutdown (kafka.server.ControllerServer)
java.lang.NullPointerException: Cannot invoke
"kafka.raft.KafkaRaftManager.apiVersions()" because the return value of
"kafka.server.SharedServer.raftManager()" is null
at kafka.server.ControllerServer.startup(ControllerServer.scala:349)
at
kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:134)
at
kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:134)
at scala.Option.foreach(Option.scala:437)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:134)
at kafka.Kafka$.main(Kafka.scala:114)
at kafka.Kafka.main(Kafka.scala)
[2023-03-06 14:31:09,678] INFO [ControllerServer id=1] shutting down
(kafka.server.ControllerServer)
[2023-03-06 14:31:09,680] INFO [SocketServer listenerType=CONTROLLER,
nodeId=1] Stopping socket server request processors (kafka.network.SocketServer)
[2023-03-06 14:31:09,683] INFO [SocketServer listenerType=CONTROLLER,
nodeId=1] Stopped socket server request processors (kafka.network.SocketServer)
[2023-03-06 14:31:09,684] INFO [SocketServer listenerType=CONTROLLER,
nodeId=1] Shutting down socket server (kafka.network.SocketServer)
[2023-03-06 14:31:09,698] INFO [SocketServer listenerType=CONTROLLER,
nodeId=1] Shutdown completed (kafka.network.SocketServer)
[2023-03-06 14:31:09,699] ERROR Exiting Kafka due to fatal exception during
startup. (kafka.Kafka$)
java.lang.NullPointerException: Cannot invoke
"kafka.raft.KafkaRaftManager.apiVersions()" because the return value of
"kafka.server.SharedServer.raftManager()" is null
at kafka.server.ControllerServer.startup(ControllerServer.scala:349)
at
kafka.server.KafkaRaftServer.$anonfun$startup$1(KafkaRaftServer.scala:134)
at
kafka.server.KafkaRaftServer.$anonfun$startup$1$adapted(KafkaRaftServer.scala:134)
at scala.Option.foreach(Option.scala:437)
at kafka.server.KafkaRaftServer.startup(KafkaRaftServer.scala:134)
at kafka.Kafka$.main(Kafka.scala:114)
at kafka.Kafka.main(Kafka.scala)
```
I am going to revert this change.
To make a similar change in the future we need a KIP since it requires
changes to the on-disk format of a file.
cc @tinaselenge @showuon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
