[
https://issues.apache.org/jira/browse/PIG-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13816884#comment-13816884
]
Rohini Palaniswamy commented on PIG-3560:
-----------------------------------------
Instead of us fixing like this, we should have tez provide a API similar to
mapreduce (/basepath/<username>/.staging) and use that instead of checking
TezConfiguration.TEZ_AM_STAGING_DIR ourselves. This is required for security
as well to separate out users and have 700 on the <username> directory making
it readable only by that user.
MRApps.java
public static Path getStagingAreaDir(Configuration conf, String user) {
return new Path(conf.get(MRJobConfig.MR_AM_STAGING_DIR,
MRJobConfig.DEFAULT_MR_AM_STAGING_DIR)
+ Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
}
We can keep this jira open to change to that API once they provide that.
> UniqueTez staging dir should be used for different users
> --------------------------------------------------------
>
> Key: PIG-3560
> URL: https://issues.apache.org/jira/browse/PIG-3560
> Project: Pig
> Issue Type: Sub-task
> Components: tez
> Affects Versions: tez-branch
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Fix For: tez-branch
>
> Attachments: PIG-3560-1.patch, PIG-3560-2.patch
>
>
> I discovered this bug while setting up a multi-user cluster. Since the
> staging dir for Pig jobs is always set to TEZ_AM_STAGING_DIR_DEFAULT
> (/tmp/tez), if multiple users submit Pig jobs, subsequent jobs fail with the
> following error:
> {code}
> java.io.IOException: The ownership on the staging directory
> hdfs://10.170.21.33:9000/tmp/tez/staging is not as expected. It is owned by
> hadoop. The directory must be owned by the submitter cheolsoop or by cheolsoop
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
