[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14374822#comment-14374822
]
zhihai xu commented on MAPREDUCE-6271:
--------------------------------------
The change(JobConf load job.xml in staging directory) in MAPREDUCE-5875 is
useful.
It can permit multiple clients to read the same job with the same configuration
as the client which submit the job.
We have application which use two Clients to read the same job. Without this
change in MAPREDUCE-5875, the two clients get different configuration.
{code}
RunningJob runningJob = jobClient.getJob(job.getJobID().toString());
{code}
> org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
> -------------------------------------------------------------
>
> Key: MAPREDUCE-6271
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Affects Versions: 2.7.0
> Reporter: Peng Zhang
> Assignee: Peng Zhang
> Attachments: MR-6271.patch
>
>
> When using getJob() with MapReduce 2.7, warn log caused by configuration
> loaded twice is displayed every time. And when job completed, this function
> will display warn log of "java.io.FileNotFoundException"
> And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
> to be not needed, cause it's only for test.
> {noformat}
> 15/03/04 13:41:23 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:23 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:24 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:24 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:25 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:25 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:26 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:26 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:27 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:27 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:28 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:28 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:29 WARN conf.Configuration:
> hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:29 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
> reduce = 0%, Cumulative CPU 2.37 sec
> 15/03/04 13:41:30 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval; Ignoring.
> 15/03/04 13:41:30 WARN conf.Configuration:
> hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
> attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts; Ignoring.
> 15/03/04 13:41:31 WARN mapreduce.Cluster: Job conf missing on
> clusterjava.io.FileNotFoundException: File does not exist:
> /yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
