GitHub user gyfora opened a pull request:
https://github.com/apache/flink/pull/2129
[FLINK-1003] [WIP] Spread out scheduling of tasks
This is a working progress PR with the core functionality implemented but
no tests yet.
As this is a highly critical part of the system I would like to get some
initial feedback before proceeding to write / change a huge amount of tests :)
About the functionality:
This is an adaptation of https://github.com/apache/flink/pull/60 to the
current flink scheduler. Instead of preferring local instances when scheduling
new task slots the new scheduling strategy allows users to balance the load on
the different task managers.
Every time a new task needs to be scheduled the scheduler considers all
instances that satisfy the scheduling constraints (has available nodes +
locality constraints) and picks the one with the smallest load. Load is
calculated by the percentage of task slots occupied in a given task manager.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gyfora/flink scheduling
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2129.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2129
----
commit f895fd71f392482cf0a50e32dc637f7885995c4e
Author: Gyula Fora <[email protected]>
Date: 2016-06-18T10:19:13Z
[FLINK-1003] Spread out scheduling of tasks
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
