[
https://issues.apache.org/jira/browse/HELIX-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993766#comment-15993766
]
ASF GitHub Bot commented on HELIX-654:
--------------------------------------
GitHub user kongweihan opened a pull request:
https://github.com/apache/helix/pull/88
[HELIX-654] Running task rebalance
Add a job config RebalanceRunningTask.
For generic task, if feature is enabled, Helix will drop running
tasks that are assigned differently from the previous assignment,
which will cause cancellation of that running task on participant.
The task will then be re-assigned to a new instance.
For fix target task, running task always follows the partition, so
tasks are always re-assigned as needed.
Add different test cases for this feature enabled/disabled.
Ticket:
https://issues.apache.org/jira/browse/HELIX-654
Test:
mvn testing
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kongweihan/helix rebalance_running_task
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/88.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #88
----
commit bf75b8988105f4c4d0f91bbdf893e67cabac6f25
Author: Weihan Kong <wk...@linkedin.com>
Date: 2017-04-26T22:34:25Z
[HELIX-654] Running task rebalance
Add a job config RebalanceRunningTask.
For generic task, if feature is enabled, Helix will drop running
tasks that are assigned differently from the previous assignment,
which will cause cancellation of that running task on participant.
The task will then be re-assigned to a new instance.
For fix target task, running task always follows the partition, so
tasks are always re-assigned as needed.
Add different test cases for this feature enabled/disabled.
----
> Rebalance running task
> ----------------------
>
> Key: HELIX-654
> URL: https://issues.apache.org/jira/browse/HELIX-654
> Project: Apache Helix
> Issue Type: New Feature
> Components: helix-core
> Reporter: Weihan Kong
>
> h3. Feature summary
> Helix Task Framework empowers user to run tasks on instances managed by
> Helix. There're 2 type of tasks: generic task and fixed target task. For
> fixed target task, the task always follows the targeted partition and is
> rebalanced if the partition is rebalanced. For generic task, Helix provides
> user the choice to rebalance the running task or not, when the topology of
> the cluster changes.
> For most users, it's better to disabled this feature(as default) since
> there's no need to re-run the task every time new node is added. For users
> with long-running tasks, enabling this feature can be very useful so that
> when new node is added, the load of the tasks are better balanced among the
> cluster.
> h3. Defined system behavior
> h4. When a node fails,
> h6. Feature disabled:
> * Running tasks on that failed node will be rebalanced to a live node, since
> the task no longer exists and failed with the node.
> h6. Feature enabled:
> * Same.
> h4. When a new node is added,
> h6. Feature disabled:
> * Running tasks will continue to run on the current instance.
> * If a running task fails after a while, it might be rebalanced and run on
> other instances, according to the new rebalance assignment under the new
> cluster topology.
> h6. Feature enabled:
> * Running task might be cancelled and rebalanced immediately, according to
> the new rebalance assignment under the new cluster topology.
> h3. Configuration
> A job level config field(RebalanceRunningTask) in JobConfig to enable/disable
> this feature. By default it's false.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
