[
https://issues.apache.org/jira/browse/HADOOP-3841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12617371#action_12617371
]
Christian Kunz commented on HADOOP-3841:
----------------------------------------
Although compression will help (I will try to turn on compression with 0.18),
would it make sense in general to make the merge algorithm adapt to available
disk space and merge in smaller steps if necessary instead of just failing?
> merge phase runs out of disk space
> ----------------------------------
>
> Key: HADOOP-3841
> URL: https://issues.apache.org/jira/browse/HADOOP-3841
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.1
> Reporter: Christian Kunz
>
> We observe that reduce tasks run out of disk space during merging (after
> fetching all map output) although there would be enough space if the
> framework did not try to generate too large merge files.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
