[
https://issues.apache.org/jira/browse/HADOOP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570614#action_12570614
]
Amareshwari Sri Ramadasu commented on HADOOP-2116:
--------------------------------------------------
I feel even with permissions, DistributedCache behavior should be the same. If
the same user wants to share files across jobs, he should be allowed. And if he
wants to share with other user who has permissions to access should be allowed.
TaskTracker need not worry about the user permissions for localizing cache,
those should be taken care in DistributedCache itself. Permissions aspect of
DistributedCache has to be handled in a different JIRA.
I propose the new layout would be the same as Owen suggested without filecache
as part of job cache.
So, it is
{noformat}
mapred/local/taskTracker/jobcache/$jobid/
work/ -- the scratch
space
jars/ -- expanded jar
job.xml -- the generic
job conf
$taskid/
job.xml -- task localized
job conf
output/ -- map outputs
work/ -- cwd for task
mapred/local/taskTracker/archive/ -- distributed cache
{noformat}
Thoughts?
> Job.local.dir to be exposed to tasks
> ------------------------------------
>
> Key: HADOOP-2116
> URL: https://issues.apache.org/jira/browse/HADOOP-2116
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.14.3
> Environment: All
> Reporter: Milind Bhandarkar
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.17.0
>
> Attachments: patch-2116.txt, patch-2116.txt, patch-2116.txt,
> patch-2116.txt
>
>
> Currently, since all task cwds are created under a jobcache directory, users
> that need a job-specific shared directory for use as scratch space, create
> ../work. This is hacky, and will break when HADOOP-2115 is addressed. For
> such jobs, hadoop mapred should expose job.local.dir via localized
> configuration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
