[
https://issues.apache.org/jira/browse/MAPREDUCE-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271984#comment-13271984
]
Luke Lu commented on MAPREDUCE-2461:
------------------------------------
We ran into this issue as well. The problem is from the staging area dirs from
jobs in local mode (via LocalJobRunner), which creates a staging area dir as
<staging_root>/<user><random>/.staging instead of just
<staging_root>/<user>/.staging in cluster mode. The issue is introduced with
the security releases (since 0.20.20x) when the getStageAreaDir API is
introduced.
The random number for the local mode is presumably used to avoid job
collisions, since there is no jobtracker to issue unique job ids.
Mayb we can introduce a feature (mapreduce.job.staging.keep=<number of latest
jobs to keep>) to prune these directories once in a while.
> Hudson jobs failing because mapred staging directory is full
> ------------------------------------------------------------
>
> Key: MAPREDUCE-2461
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2461
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Fix For: 0.24.0
>
>
> All of the tests that submit MR jobs are failing on the h7 build machine.
> This is because the staging directory is entirely full:
> hudson@h7:/tmp/mr/mr$ ls -l /tmp/hadoop-hudson/mapred/staging/ | wc -l
> 31999
> This makes me think that there's some bug where we're leaking things in the
> staging directory. I will manually clean this for now, but we should
> investigate.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
