Jack Vanlightly created KAFKA-16281: ---------------------------------------
Summary: Probable IllegalState possible with KIP-966 Key: KAFKA-16281 URL: https://issues.apache.org/jira/browse/KAFKA-16281 Project: Kafka Issue Type: Task Components: kraft Reporter: Jack Vanlightly I have a TLA+ model of KIP-966 and I have identified an IllegalState exception that would occur with the existing MaybeHandleCommonResponse behavior. The issue stems from the fact that a leader, let's call it r1, can resign (either due to a restart or check quorum) and then later initiate a pre-vote where it ends up in the same epoch as before, but a cleared local leader id. When r1 transitions to Prospective it clears its local leader id. When r1 receives a response from r2 who believes that r1 is still the leader, the logic in MaybeHandleCommonResponse tries to transition r1 to follower of itself, causing an IllegalState exception to be raised. This is an example history: # r1 is the leader in epoch 1. # r1 quorum resigns, or restarts and resigns. # r1 experiences an election timeout and transitions to Prospective clearing its local leader id. # r1 sends a pre vote request to its peers. # r2 thinks r1 is still the leader, sends a vote response, not granting its vote and setting leaderId=r1 and epoch=1. # r1 receives the vote response and executes MaybeHandleCommonResponse which tries to transition r1 to Follower of itself and an illegal state occurs. The relevant else if statement in MaybeHandleCommonResponse is here: https://github.com/apache/kafka/blob/a26a1d847f1884a519561e7a4fb4cd13e051c824/raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java#L1538 In the TLA+ specification, I fixed this issue by adding a fourth condition to this statement, that the leaderId also does not equal this server's id. [https://github.com/Vanlightly/kafka-tlaplus/blob/9b2600d1cd5c65930d666b12792d47362b64c015/kraft/kip_996/kraft_kip_996_functions.tla#L336] We should probably create a test to confirm the issue first and then look at using the fix I made in the TLA+, though there may be other options. -- This message was sent by Atlassian Jira (v8.20.10#820010)