lucliu1108 opened a new pull request, #24062:
URL: https://github.com/apache/kafka/pull/24062
## Problem
When a reassignment replaces one that is still in progress,
`ReplicationControlManager.changePartitionReassignment` computes the new adding
and removing sets against the partition's current replicas, which already
include the in-flight target.
The current implementation only pass these sets to the
`PartitionChangeBuilder` if they are not empty. The builder starts from the
partition's current adding and removing sets, so an empty set leaves the
previous reassignment's value in place instead of clearing it. This can cause
problems when a new reassignment is submitted before the previous one completes.
To be more specific, some examples:
1. Empty adding: the previous adding replicas stayed, so the same replicas
were both adding and removing. e.g.
- Initial assignment: replicas=[0,1], ISR=[0,1].
- Reassign to [2,3]: replicas=[2,3,0,1], adding=[2,3], removing=[0,1].
2 and 3 have not joined the ISR yet.
- Reassign back to [0,1]: the new delta against [2,3,0,1] is
removing=[2,3] and adding=[]. Since adding was not set, it kept
[2,3]. A reassignment only completes once every adding replica is in
the ISR, so it stays in progress until 2 and 3 catch up, even though they are
being removed.
- Resulting assignment: replicas=[0,1,2,3], adding=[2,3],
removing=[2,3], ISR=[0,1]. The replicas list is expected while the reassignment
is in progress, but adding should be []. The reassignment waits for 2 and 3 to
join the ISR, so replicas never shrinks to [0,1]. .
2. Empty removing: the previous removing replicas stayed, so a replica the
new request kept was removed on completion. e.g.
- Initial assignment: replicas=[1,2,0], ISR=[1,2,0].
- Reassign to [1,2,3]: replicas=[1,2,3,0], adding=[3], removing=[0],
ISR=[1,2,0].
- Reassign to [1,2,3,0,4]: delta is adding=[4], removing=[].
Result: replicas=[1,2,3,0,4], adding=[4], removing=[0], ISR=[1,2,0].
- After 4 joins the ISR: replicas=[1,2,3,4], adding=[], removing=[],
ISR=[1,2,4].
3. Both empty: no record was written, so the request was ignored. e.g.
- Initial assignment: replicas=[0,1], ISR=[0,1].
- Reassign to [2,3]: replicas=[2,3,0,1], adding=[2,3],
removing=[0,1], ISR=[0,1].
- Reassign to [2,3,0,1]: delta is adding=[], removing=[].
- Result: replicas=[2,3,0,1], adding=[2,3], removing=[0,1], ISR=[0,1].
- After 2 and 3 join the ISR: replicas=[2,3], adding=[], removing=[],
ISR=[2,3].
## How to fix it
This change removes the two non-empty checks, so the new adding and removing
sets are always passed to `PartitionChangeBuilder`, even when empty. An empty
set now clears the previous reassignment's value instead of keeping it.
Three tests are added to validate the correct behaviors of the 3 examples
mentioned above after the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
