GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/1272
[FLINK-2834] Global round-robin for temporary directories
Multiple TaskManager filesystems can be used by configuring multiple
temporary directories.
This patch changes the process of spilling files from a per-operator
round-robin to a global
round-robin such that each directory is written to in turn across all
operators, reducing
unbalanced I/O due to bunching.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink
2834_global_round_robin_for_temporary_directories
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1272.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1272
----
commit 4af4332bfdf63112e940f706a6b8fedc0b7cff5b
Author: Greg Hogan <[email protected]>
Date: 2015-10-20T14:47:58Z
[FLINK-2834] Global round-robin for temporary directories
Multiple TaskManager filesystems can be used by configuring multiple
temporary directories.
This patch changes the process of spilling files from a per-operator
round-robin to a global
round-robin such that each directory is written to in turn across all
operators, reducing
unbalanced I/O due to bunching.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
