[
https://issues.apache.org/jira/browse/HADOOP-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated HADOOP-3598:
----------------------------------
Resolution: Fixed
Release Note: Map-Reduce framework now no-longer creates temporary
task-output directories for staging outputs if it isn't necessary.
Status: Resolved (was: Patch Available)
I just committed this.
> Map-Reduce framework needlessly creates temporary _${taskid} directories for
> Maps
> ---------------------------------------------------------------------------------
>
> Key: HADOOP-3598
> URL: https://issues.apache.org/jira/browse/HADOOP-3598
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.18.0
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Priority: Blocker
> Fix For: 0.18.0
>
> Attachments: HADOOP-3598_0_20080619.patch,
> HADOOP-3598_0_20080619.patch, HADOOP-3598_0_20080619.patch,
> HADOOP-3598_1_20080620.patch
>
>
> The staging directory for task-outputs (i.e.
> ${mapred.out.dir}/_temporary/_${taskid}) should only be created when Maps
> produce output on HDFS, which usually isn't the case. This plays very badly
> with HDFS quotas and may lead to thousands of temp names in the FS namespace,
> there-by overhauling the quotas. IAC, it isn't good to needlessly create
> these directories.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
