[GitHub] helix pull request #143: [HELIX-676] Fix the issue that the controller keep ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/helix/pull/143


---

[GitHub] helix pull request #143: [HELIX-676] Fix the issue that the controller keep ...

[jiajunwang]

GitHub user jiajunwang opened a pull request:

https://github.com/apache/helix/pull/143

[HELIX-676] Fix the issue that the controller keep updating idealstates 
when there is no real diff.

The causes of the problem are that:

1.A previous issue introduced into IntermediateStateCalcStage prevents 
ERROR/OFFLINE state replicas from being added to the intermediateState, given 
the controller thinks recovery rebalance is not necessary.
This makes the processed stateMapping in pipeline always different from the 
cached IdeaStates. And then causes endless updating.

2.Another separate change in persistAssignmentStage is also related to this 
issue. When updating the map/list, we used putAll. This call will keep all 
existing items but only modify the intersect. Our assumption previously is 
allow customized items. However, when we investigate this issue, it would be 
error-prone to allow these customized items in the map/list fields. Helix won't 
be able to tell if one item is added by the controller or users. So we decide 
to use clear and set instead. This ensure the map/list fields are always 
uptodate.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jiajunwang/helix helix-676

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/helix/pull/143.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #143


commit 9083950d9a0e5f5c377672fb5b21af13cdde0d9a
Author: Jiajun Wang 
Date:   2018-02-12T19:45:18Z

[HELIX-676] Fix the issue that the controller keep updating idealstates 
when there is no real diff.

The causes of the problem are that:

1.A previous issue introduced into IntermediateStateCalcStage prevents 
ERROR/OFFLINE state replicas from being added to the intermediateState, given 
the controller thinks recovery rebalance is not necessary.
This makes the processed stateMapping in pipeline always different from the 
cached IdeaStates. And then causes endless updating.

2.Another separate change in persistAssignmentStage is also related to this 
issue. When updating the map/list, we used putAll. This call will keep all 
existing items but only modify the intersect. Our assumption previously is 
allow customized items. However, when we investigate this issue, it would be 
error-prone to allow these customized items in the map/list fields. Helix won't 
be able to tell if one item is added by the controller or users. So we decide 
to use clear and set instead. This ensure the map/list fields are always 
uptodate.

commit 08ff9d0b287ece2a17f2430ba82bd4f2373263e7
Author: Jiajun Wang 
Date:   2018-02-17T00:31:27Z

[HELIX-676] Adding test for intermediate state cal stage.

Testing 4 cases (recovery, load balance, recovery with transient states, 
and error partition blocks load balance).
This test covers the recent fix for HELIX-676.




---