GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/100
[HELIX-655] Helix per-participant concurrent task throttling
Add per participant concurrent task throttling.
Add a participant configuration item "MAX_CONCURRENT_TASK" for throttling.
New assigned task + existing running/init task <= MAX_CONCURRENT_TASK.
Otherwise, new assignment won't be included in best possible state.
Tasks are assigned in the order of jobs' start time. Older jobs have higher
priority than other jobs.
Add test case (TestTaskThrottling.java) for testing new throttling and
priority.
Ticket:
https://issues.apache.org/jira/browse/HELIX-655
Test:
mvn test in helix-core
Please refer to previous discussions in another pull request:
https://github.com/apache/helix/pull/89
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jiajunwang/helix master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/100.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #100
----
commit 20685cf6e1276aaa1bf6264c1fe6a3173081d22c
Author: Jiajun Wang <[email protected]>
Date: 2017-05-31T21:57:23Z
Helix per-participant concurrent task throttling
Add per participant concurrent task throttling.
1. Add a participant configuration item "MAX_CONCURRENT_TASK" for
throttling setting.
Add cluster configuration item "MAX_CONCURRENT_TASK_PER_INSTANCE" as the
default throttling settings.
New assigned task + existing running/init task <= MAX_CONCURRENT_TASK.
Otherwise, new assignment won't be included in best possible state.
2. Tasks are assigned in the order of jobs' start time. Older jobs have
higher priority than other jobs and regular resources.
3. Add test case (TestTaskThrottling.java) for testing new throttling and
priority.
Ticket:
https://issues.apache.org/jira/browse/HELIX-655
Test:
mvn test in helix-core
commit e35fe4fffc952f7ccae7bfa4cbf89ef75e404a53
Author: Jiajun Wang <[email protected]>
Date: 2017-06-03T06:26:50Z
Add workflow configuration to allow or disallow assigning multiple jobs to
one instance.
By default, it is not allowed that Helix assigns multiple jobs in one
workflow to the same instances.
If it is set to be true, the instance can start executing multiple jobs in
each workflow.
When application sets max tasks throttling for the participants, allowing
overlapping assignment can maximize utilization.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
