acgtun opened a new pull request, #24091: URL: https://github.com/apache/kafka/pull/24091
When `AlterPartitionReassignments` targets a partition that already has a reassignment in progress, `ReplicationControlManager.changePartitionReassignment` computes the new `adding` / `removing` sets against `part.replicas`, which already contains the replicas the previous reassignment is still adding. `PartitionReassignmentReplicas` is a plain set difference, so a replica that was being added by the in-progress reassignment, has not joined the ISR yet, and appears in the new target is dropped from `addingReplicas` and becomes a regular replica without ever having been in sync. `maybeCompleteReassignment` then only waits for the newly added replicas, and the partition completes with an out-of-sync replica in its final list. Each further override promotes the previous round's adding replicas the same way, so with Cruise Control re-planning stalled moves an RF-3 topic ends up with partitions carrying many more replicas than RF, none of them tracked by any reassignment. Per partition: `[A,B,C]` -> `[A,B,D]` stalls with `adding=[D]`; an override to `[A,B,D,E]` yields `adding=[E] removing=[C]`, and D is now a plain replica. When E joins the ISR the reassignment completes with `replicas=[A,B,D,E] isr=[A,B,E]`. The override path is only correct today when the new adding set happens to be empty (the builder leaves the old one in place) or when the `adding.size() < removing.size()` guard blocks completion. Cancellation (`PartitionReassignmentRevert`) is unaffected. Compute `adding` against the original replicas instead, i.e. the current replicas minus the replicas the in-progress reassignment is still adding: - `removing = currentReplicas - target` (unchanged) - `adding = target - (currentReplicas - currentAdding)` (was `target - currentReplicas`) - `replicas = target U removing` (unchanged) A still-adding replica that the new target keeps stays in `addingReplicas`, so the new reassignment also waits for it to join the ISR. Replicas that the new target drops go through `removingReplicas` exactly as before, which keeps the current leader in the replica list until a new one is elected (the behaviour pinned by `testReassignPartitionsHandlesNewReassignmentThatRemovesPreviouslyAddingReplicas`, #13521). Re-submitting the in-progress target is now a no-op instead of clearing `addingReplicas`. `PartitionReassignmentReplicas` gets a three-argument constructor taking the current adding replicas; the two-argument one delegates with an empty list, so `PartitionChangeBuilder` is unchanged. Testing: - `PartitionReassignmentReplicasTest`: target keeps a still-adding replica, target equals the in-progress target, target drops the still-adding replicas. - `ReplicationControlManagerTest.testReassignPartitionsHandlesNewReassignmentThatKeepsPreviouslyAddingReplicas`: `[0,1,2]` -> `[0,1,3]` with 3 lagging, override to `[0,1,3,4]`, 4 joins the ISR; without the fix `addingReplicas=[4]` instead of `[3,4]` and the reassignment completes with 3 out of sync. - `QuorumControllerTest.testOverridingReassignmentKeepsStillAddingReplica`: same scenario through `alterPartitionReassignments` / `alterPartition` on a `QuorumController`. - `ReassignPartitionsCommandTest.testAdditionalReassignmentKeepsStillAddingReplica`: `kafka-reassign-partitions.sh --execute --additional` against a 5-broker cluster with the first adding replica held out of the ISR by a throttle; checks `--list`, `--verify`, and that every out-of-sync replica is tracked as adding or removing. Passes in both KRaft modes; without the fix it fails with `addingReplicas=[3]` instead of `[3, 4]`. - New ducktape test `ReassignPartitionsOverrideTest.test_override_keeps_still_adding_replica` (1 controller + 5 brokers) runs the same scenario end to end; `KafkaService` gains an `additional` flag on `execute_reassign_partitions` and a `list_partition_reassignments` helper. Passes with the fix; without it fails with `replicas=[1, 2, 4, 5] isr=[1, 2, 5]` and no reassignment in progress. - Also verified manually on a 3-controller / 7-broker cluster across the override variants (keep / drop the adding replica, chained overrides, RF-keeping override, leader removal, same target re-submitted, `--disallow-replication-factor-change`, `--cancel`); details in the first comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
