Kafka 3.6.0.
I have a KRaft cluster with three quorum servers. A power failure killed
all the controllers at the same time. After rebooting, the controllers
can not connect to each other. So, the cluster is down.
Log:
"""
[...]
[2023-12-01 20:29:24,931] INFO [MetadataLoader id=1000]
initializeNewPublishers: the loader is still catching up because we
still don't know the high water mark yet.
(org.apache.kafka.image.loader.MetadataLoader)
[2023-12-01 20:29:24,957] ERROR [RaftManager id=1000] Unexpected error
UNKNOWN_SERVER_ERROR in VOTE response:
InboundResponse(correlationId=6698, data=VoteResponseData(errorCode=-1,
topics=[]), sourceId=1002) (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:24,957] INFO [RaftManager id=1000] Vote request
VoteRequestData(clusterId='*EDITED*',
topics=[TopicData(topicName='__cluster_metadata',
partitions=[PartitionData(partitionIndex=0, candidateEpoch=13359,
candidateId=1001, lastOffsetEpoch=13079, lastOffset=6304466)])]) with
epoch 13359 is rejected (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:24,983] ERROR [RaftManager id=1000] Unexpected error
UNKNOWN_SERVER_ERROR in VOTE response:
InboundResponse(correlationId=6699, data=VoteResponseData(errorCode=-1,
topics=[]), sourceId=1001) (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:25,020] ERROR [RaftManager id=1000] Unexpected error
UNKNOWN_SERVER_ERROR in VOTE response:
InboundResponse(correlationId=6700, data=VoteResponseData(errorCode=-1,
topics=[]), sourceId=1002) (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:25,032] INFO [MetadataLoader id=1000]
initializeNewPublishers: the loader is still catching up because we
still don't know the high water mark yet.
(org.apache.kafka.image.loader.MetadataLoader)
[2023-12-01 20:29:25,044] ERROR [RaftManager id=1000] Unexpected error
UNKNOWN_SERVER_ERROR in VOTE response:
InboundResponse(correlationId=6701, data=VoteResponseData(errorCode=-1,
topics=[]), sourceId=1001) (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:25,082] ERROR [RaftManager id=1000] Unexpected error
UNKNOWN_SERVER_ERROR in VOTE response:
InboundResponse(correlationId=6702, data=VoteResponseData(errorCode=-1,
topics=[]), sourceId=1002) (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:25,105] ERROR [RaftManager id=1000] Unexpected error
UNKNOWN_SERVER_ERROR in VOTE response:
InboundResponse(correlationId=6703, data=VoteResponseData(errorCode=-1,
topics=[]), sourceId=1001) (org.apache.kafka.raft.KafkaRaftClient)
[2023-12-01 20:29:25,133] INFO [MetadataLoader id=1000]
initializeNewPublishers: the loader is still catching up because we
still don't know the high water mark yet.
(org.apache.kafka.image.loader.MetadataLoader)
[...]
"""
I use SASL_SSL. The controller credentials are "wired" in the
configuration, so no "metadata recovery watermark" knowledge should be
necessary:
"""
listener.name.controller.sasl.enabled.mechanisms=PLAIN,SCRAM-SHA-256
listener.name.controller.plain.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
required \
username="controller" \
password="*EDITED" \
user_controller="*EDITED*";
listener.name.controller.scram-sha-256.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule
required username="*EDITED" password="*EDITED*";
"""
--
Jesús Cea Avión _/_/ _/_/_/ _/_/_/
j...@jcea.es - https://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/
Twitter: @jcea _/_/ _/_/ _/_/_/_/_/
jabber / xmpp:j...@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/
"Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/
"My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
