[
https://issues.apache.org/jira/browse/MAPREDUCE-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930437#action_12930437
]
Todd Lipcon commented on MAPREDUCE-2181:
----------------------------------------
bq. In any case, I'm still left with /user being not a good place to put system
resources
I fail to see how the job staging directory is considered a "system resource".
It's per-user temporary data during the job submission process. Much like how
web browsers store per-user caches in $HOME/.mozilla, the job submitter should
put its data in $HOME/.staging.
Putting a big quota on /mapred and making /mapred/staging mode 777 (or mode
1777 on trunk) just gives users one more place they can potentially abuse to
store more data than they should be allowed.
> mapreduce.jobtracker.staging.root.dir default is unreasonable
> -------------------------------------------------------------
>
> Key: MAPREDUCE-2181
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2181
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission, jobtracker
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
>
> The default for mapreduce.jobtracker.staging.root.dir is set to
> ${hadoop.tmp.dir}/mapred/staging, which doesn't really work on a normal
> cluster. hadoop.tmp.dir is overloaded in different places where sometimes it
> is a local path and sometimes it is a path on HDFS, which makes things even
> more confusing.
> We should change the default for the staging directory to /user (as is
> suggested by the description of that configuration) and then fix
> LocalJobRunner to use a different configuration -- perhaps
> mapreduce.localjobrunner.staging.root.dir -- to make it clear that it's a
> *local* path. That one could legitimately default to something inside
> hadoop.tmp.dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
