[
https://issues.apache.org/jira/browse/FLINK-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739161#comment-14739161
]
ASF GitHub Bot commented on FLINK-2655:
---------------------------------------
GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/1118
[FLINK-2655] Minimize intermediate merging of spilled buffers
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink
2655_mimimize_intermediate_merging_of_spilled_buffers
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1118
----
----
> Minimize intermediate merging of spilled buffers
> ------------------------------------------------
>
> Key: FLINK-2655
> URL: https://issues.apache.org/jira/browse/FLINK-2655
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Runtime
> Affects Versions: master
> Reporter: Greg Hogan
>
> If the number of spilled buffers exceeds taskmanager.runtime.max-fan then the
> number of files must reduced with an intermediate merge by reading, merging,
> and spilling into a single, larger file.
> The current implementation performs an intermediate merge on all files. An
> optimal implementation minimizes the amount of merged data by performing
> partial merges first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
