GitHub user lei-xia opened a pull request:
https://github.com/apache/helix/pull/67
Merge all new features, bugs and improvements developed from LinkedIn's
internal Helix branch.
A list of commits:
1. Lei Xia: Avoid moving partitions unnecessarily when auto-rebalancing
using default AutoRebalanceStrategy.
2. Lei Xia: Fix bug in AutoRebalanceStrategy to try to assign orphan
replicas to its preferred nodes instead of random nodes.`
3. Lei Xia: Fix a bug in BestPossibleExternalViewVerifier.
4. Lei Xia: Move all options from IdealState to ResourceConfig, add Bulder
for building ResourceConfig, and a new RebalanceConfig to hold all rebalance
options for a resource.
5. Lei Xia: Persist controller leader change history with timestamp for
each leader controller.
6. Lei Xia: Persist the timestamp along with controller leader change
history.
7. Lei Xia: Refactor/renames the instances releated methods in
ClusterDataCache.
8. Lei Xia: Add cluster-level and resource-level config option to allow
disable delayed rebalance of entire cluster or individual resource.
9. Boyan Li: Add more messaging metrics to participant.
10. Lei Xia: Log improvement: add resource name to logs in different places.
11. Lei Xia: New DelayedAutoRebalancer featured with delayed partition
movements during rebalancing.
12. Lei Xia: Persist assignment map using specific format for MasterSlave
resources (This is a short-term solution, we should get rid of this asap).
13. Boyan Li: Add a messaging monitoring mbean to ParticipantStatusMonitor
object.
14. Lei Xia: More refactors on rebalancer releated pipelines. 1) Move get
rebalancer and mappingCalculator logics out to separate methods to make the
main flow clearer. 2) Move definition of ANY_LIVEINSTANCE from HelixConstants
to IdealState.
15. Boyan Li: rename ParticipantMonitor class
16. Boyan Li: Revert "add a mbean for participant and emit received msgs"
17. Lei Xia: Persist participant's offline timestamp in ParticipantHistory.
18. Lei Xia: Refactor: add AbstractRebalancer as an abstract class, which
contains the default implementations as well as utility functions that will be
used by all specific rebalancers.
19. Lei Xia: Add reset() method to MockParticipantManager to allow reuse of
the participant.
20. Lei Xia: Refactor: rename ParticipantManagerHelper to
ParticipantManager and move all logic to handle new participant session into
the ParticipangManager.
21. Lei Xia: Persist session change history with timestamp for each
participant.
22. Boyan Li: Add a mbean for participant and emit received msgs
23. Boyan Li: Add logs for session sync messages
24. Boyan Li: participant syncs session id to controller
25. Junkai Xue: Fix BestPossibleExternalViewVerifier toString NPE
26. Lei Xia: Add support for flexible hirerachy representation of a cluster
topology.
27. Lei Xia: Add StrictMatchExternalViewVerifier that verifies whether the
ExternalViews of given resources (or all resources in the cluster) match
exactly as its ideal mapping (in idealstate).
28. Lei Xia: Restructure ClusterVerifiers. Add HelixClusterVerifier
interface, add abstract class ZkHelixClusterVerifier, and a
BestPossibleExternViewVerifier implementation.
29. Lei Xia: Refactor: put all cluster verifiers into a sub-module of tools.
30. Lei Xia: Add Multi-round CRUSH rebalance strategy.
31. Lei Xia: Add option to allow persisting best possible partition
assignment in IdealState for semi-auto and full-auto modes.
32. Lei Xia: Deprecated AutoModeISBuilder and AutoRebalanceModeIsBuilder
and created SemiAutoISBuilder and FullAutoISBuilder instead.
33. Lei Xia: Support of client's customized threadpool for state-transition
message handling.
34. Lei Xia: Minor fix: Do not set MaxPartitionPerNode in IdealState if it
is not greater than 0.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lei-xia/helix github.helix-0.6.x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/67.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #67
----
commit 68e2fdca5c0fdf98d93ca59bee05a0ff30a3d54d
Author: Lei Xia <[email protected]>
Date: 2017-02-06T00:37:29Z
Minor fix: Do not set MaxPartitionPerNode in IdealState if it is not
greater than 0.
commit eb33370289e2fee521a12f6bae901560bd599f57
Author: Lei Xia <[email protected]>
Date: 2016-04-15T22:38:15Z
Support of client's customized threadpool for state-transition message
handling.
commit f1e9188733ffa72cdc68643061ac367e21bdf460
Author: Lei Xia <[email protected]>
Date: 2016-06-07T23:25:36Z
Deprecated AutoModeISBuilder and AutoRebalanceModeIsBuilder and created
SemiAutoISBuilder and FullAutoISBuilder instead.
commit 7d0885c78dbb94c2224490f53a161a752ee83de5
Author: Lei Xia <[email protected]>
Date: 2016-07-01T23:27:53Z
Add option to allow persisting best possible partition assignment in
IdealState for semi-auto and full-auto modes.
commit 0f7c3e42080ba8e2b17e36ca1c5c51c6209b0f03
Author: Lei Xia <[email protected]>
Date: 2016-06-27T22:46:13Z
Add Multi-round CRUSH rebalance strategy.
commit 695228e0ec6fd5fc48d5408633205f8ab935838d
Author: Lei Xia <[email protected]>
Date: 2016-07-18T23:06:22Z
Refactor: put all cluster verifiers into a sub-module of tools.
commit 04495e707a3a1a9067fd487cac96a169dc45c6dd
Author: Lei Xia <[email protected]>
Date: 2016-07-20T01:17:19Z
Restructure ClusterVerifiers. Add HelixClusterVerifier interface, add
abstract class ZkHelixClusterVerifier, and a BestPossibleExternViewVerifier
implementation.
commit 981d0e295e01cd0839c8d5d0e54350f794ad52f7
Author: Lei Xia <[email protected]>
Date: 2016-07-21T18:29:02Z
Add StrictMatchExternalViewVerifier that verifies whether the ExternalViews
of given resources (or all resources in the cluster) match exactly as its ideal
mapping (in idealstate).
commit 215039b374ba1457d892d5f0c7ad758b393ed23a
Author: Lei Xia <[email protected]>
Date: 2016-07-01T17:20:50Z
Add support for flexible hirerachy representation of a cluster topology.
commit 9bf4437aa3ffe64d4cb983fea0a6b81162edeed9
Author: Junkai Xue <[email protected]>
Date: 2016-07-29T23:25:01Z
Fix BestPossibleExternalViewVerifier toString NPE
commit 2981bbd11dc869f5235e85e71f1b30368ef7596e
Author: Boyan Li <[email protected]>
Date: 2016-08-05T18:07:15Z
participant syncs session id to controller
commit a1278e199b54f29fb37abda20c23262ebb1b9d28
Author: Boyan Li <[email protected]>
Date: 2016-08-23T04:24:43Z
Add logs for session sync messages
commit 821fd04dfb1dc66a85d0dfb76e5837a4c8107448
Author: Boyan Li <[email protected]>
Date: 2016-08-29T16:47:27Z
Add a mbean for participant and emit received msgs
commit 8b320514da58e38652f79f625576bb9895f6312c
Author: Lei Xia <[email protected]>
Date: 2016-08-30T20:29:08Z
Persist session change history with timestamp for each participant.
commit abecf2fe332a37de96ea5679654c38636772a2ed
Author: Lei Xia <[email protected]>
Date: 2016-09-09T18:07:37Z
Refactor: rename ParticipantManagerHelper to ParticipantManager and move
all logic to handle new participant session into the ParticipangManager.
commit 6f471606f18f6de416c3148bead43577c3a24167
Author: Lei Xia <[email protected]>
Date: 2016-09-09T22:58:46Z
Add reset() method to MockParticipantManager to allow reuse of the
participant.
RB=811078
G=nuage-reviewers
R=boli,jxue
A=boli
commit 752fd696ed1b8d098a63196a9fd02342d60c2e6d
Author: Lei Xia <[email protected]>
Date: 2016-09-12T17:59:40Z
Refactor: add AbstractRebalancer as an abstract class, which contains the
default implementations as well as utility functions that will be used by all
specific rebalancers.
commit ebfe2ea3ce174b1431423402be84e064ad47bc6e
Author: Lei Xia <[email protected]>
Date: 2016-09-02T01:21:36Z
Persist participant's offline timestamp in ParticipantHistory.
This is to persist the timestamp when a participant is going offline.
1) If a participant goes offlien gracefully (by calling disconnect()),
participant will write a timestamp to its history record.
2) If a participant goes offline without calling disconnect() (e.g, GC,
machine crashes), controller will try to set the timestamp in its pipeline
triggered by liveInstanceChanges.
commit 543b3207e669710996a643f541ecc29ba2b2a8f8
Author: Boyan Li <[email protected]>
Date: 2016-09-08T23:36:25Z
Revert "add a mbean for participant and emit received msgs"
This reverts commit ccff605eb311072644f355686853351a10ff5b95.
commit 3688630f3cb853658dc1aaefe54d70d29dab5243
Author: Boyan Li <[email protected]>
Date: 2016-09-14T17:48:23Z
rename ParticipantMonitor class
RB=813995
G=nuage-reviewers
R=lxia,jxue
A=lxia
commit d9a47b43c3c16d18168247d1dabde1640932815d
Author: Lei Xia <[email protected]>
Date: 2016-09-15T18:56:10Z
More refactors on rebalancer releated pipelines. 1) Move get rebalancer and
mappingCalculator logics out to separate methods to make the main flow clearer.
2) Move definition of ANY_LIVEINSTANCE from HelixConstants to IdealState.
RB=815358
G=nuage-reviewers
R=boli,jxue
A=jxue
commit db6ab9fe9ea6ebca8c95c14cbc4635ac98935080
Author: Boyan Li <[email protected]>
Date: 2016-09-14T22:29:54Z
Add a messaging monitoring mbean to ParticipantStatusMonitor object.
commit 99e6c4a659ce4616b4338ca14f196fbff3c75f4b
Author: Lei Xia <[email protected]>
Date: 2016-09-09T20:31:14Z
Persist assignment map using specific format for MasterSlave resources
(This is a short-term solution, we should get rid of this asap).
commit d7c035ca41de0c1d01589a3b1b232c5cef21f95c
Author: Lei Xia <[email protected]>
Date: 2016-09-12T23:42:17Z
New DelayedAutoRebalancer featured with delayed partition movements during
rebalancing.
commit e1f0678a0c918f74e57dc6ee2559ca813ecfd7d7
Author: Lei Xia <[email protected]>
Date: 2016-09-19T17:20:01Z
Log improvement: add resource name to logs in different places.
commit 5d9f76e6674458e624fc9142511e7ffcbb7fdc0d
Author: Boyan Li <[email protected]>
Date: 2016-09-19T18:42:11Z
Add more messaging metrics to participant.
commit ad4625e4c431903b29827259672446814e2fd0a0
Author: Lei Xia <[email protected]>
Date: 2016-09-21T17:53:31Z
Add cluster-level and resource-level config option to allow disable delayed
rebalance of entire cluster or individual resource.
commit 7bb3efd15b03ba698c7dc4a8a5277de7870f4014
Author: Lei Xia <[email protected]>
Date: 2016-09-23T15:26:50Z
Refactor/renames the instances releated methods in ClusterDataCache.
commit b562026cb3e98b97b6b5359068de8ca925e89b0c
Author: Lei Xia <[email protected]>
Date: 2016-08-30T20:26:52Z
Persist the timestamp along with controller leader change history.
commit c622d63c4b777e673edf2d26bfd2a3289f92bc9e
Author: Lei Xia <[email protected]>
Date: 2016-10-13T01:15:10Z
Persist controller leader change history with timestamp for each leader
controller.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
