[
https://issues.apache.org/jira/browse/MAPREDUCE-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569168#comment-13569168
]
Karthik Kambatla commented on MAPREDUCE-4964:
---------------------------------------------
Thanks Arun - the other patch completely skipped my mind. I agree that the
patch on MAPREDUCE-4843 would fix the immediate issue at hand - jobconf being
written to a different user's directory. However, this patch fixes a
fundamental issue in {{JobLocalizer}} that the test in the patch clearly
exposes.
Let me explain through the code (with comments added after each line,
irrelevant lines removed). The code before the patch:
{code}
public JobLocalizer(JobConf ttConf, String user, String jobid,
String... localDirs) throws IOException {
this.ttConf = ttConf; // shallow copy without any
indication that ttConf can't be changed later, no obvious locks held either
lfs = FileSystem.getLocal(ttConf).getRaw(); // ttConf could have changed
already, and we could potentially read wrong filesystem - agree highly unlikely
ttConf.setStrings(JOB_LOCAL_CTXT, localDirs);// setting the dirs on the
external ttConf, not the instance specific one
}
{code}
I don't think JobLocalizer should be making these assumptions (irrespective of
how it is accessed), and if there is a strong enough reason to make these, we
should communicate it to the users of JobLocalizer.
> JobLocalizer#localizeJobFiles can potentially write job.xml to the wrong
> user's directory
> -----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4964
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4964
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 1.1.1
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4964.patch, MR-4964.patch
>
>
> In the following code, if jobs corresponding to different users (X and Y) are
> localized simultaneously, it is possible that jobconf can be written to the
> wrong user's directory. (X's job.xml can be written to Y's directory)
> {code}
> public void localizeJobFiles(JobID jobid, JobConf jConf,
> Path localJobTokenFile, TaskUmbilicalProtocol taskTracker)
> throws IOException, InterruptedException {
> localizeJobFiles(jobid, jConf,
> lDirAlloc.getLocalPathForWrite(JOBCONF, ttConf), localJobTokenFile,
> taskTracker);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
