[
https://issues.apache.org/jira/browse/HADOOP-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Owen O'Malley updated HADOOP-3135:
----------------------------------
Status: Open (was: Patch Available)
I'm sorry, but this patch is out of sync with respect to trunk. Can you fix it
once more?
> if the 'mapred.system.dir' in the client jobconf is different from the
> JobTracker's value job submission fails
> --------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-3135
> URL: https://issues.apache.org/jira/browse/HADOOP-3135
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.16.1
> Environment: all
> Reporter: Alejandro Abdelnur
> Assignee: Subramaniam Krishnan
> Priority: Critical
> Fix For: 0.18.0
>
> Attachments: patch-3135-v3.txt, patch-3135-v4.txt, patch-3135-v5.txt,
> patch-3135-v6.txt, patch-3135-v6_2.txt, patch-3135-v6_3.txt
>
>
> Until Hadoop 0.13 or so, at submission time the full path of the job.xml and
> all supporting files in DFS was given by the client to the jobtracker.
> Since 0.15 onwards (we did not test 0.14) the jobclient is obtaining the job
> ID from the jobtracker and creating the directory for all the supporting
> files using the a system-dir computed from the local jobconf.
> Line 696-7 in the JobClient:
> String jobId = jobSubmitClient.getNewJobId();
> Path submitJobDir = new Path(job.getSystemDir(), jobId);
> This makes submissions to fail when the value of the 'mapred.system.dir' on
> the client is different from the one in the JobTracker.
> A simple way o fixing this would be to introduce a new method in the
> JobSubmissionProtocol 'getSystemDir()' that would return the jobtracker
> system dir and use that dir for uploading all the files on submission.
> ----
> For the future: A more comprehensive way of this doing would to obtain a base
> jobConf from the jobtracker, carrying final information for each element and
> the construct the job.xml on the client using the final semantics. And, in
> this case the 'mapred.system.dir' property should be set as final in the
> jobtracker. As there may be some configuration properties that are sensitive
> and for security reasons should not be exposed to the clients a new flag
> 'private' could be introduced and only properties that don't have the
> 'private' flag would be send over from the jobtracker to the jobclient for
> job.xml resolution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
