[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated MAPREDUCE-6271:
Labels: BB2015-05-TBR (was: )
org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
-
Key: MAPREDUCE-6271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Labels: BB2015-05-TBR
Attachments: MAPREDUCE-6271.v2.patch, MR-6271.patch
When using getJob() with MapReduce 2.7, warn log caused by configuration
loaded twice is displayed every time. And when job completed, this function
will display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peng Zhang updated MAPREDUCE-6271:
--
Attachment: MAPREDUCE-6271.v2.patch
org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
-
Key: MAPREDUCE-6271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Attachments: MAPREDUCE-6271.v2.patch, MR-6271.patch
When using getJob() with MapReduce 2.7, warn log caused by configuration
loaded twice is displayed every time. And when job completed, this function
will display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:31
[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peng Zhang updated MAPREDUCE-6271:
--
Status: Patch Available (was: Open)
org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
-
Key: MAPREDUCE-6271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Attachments: MAPREDUCE-6271.v2.patch, MR-6271.patch
When using getJob() with MapReduce 2.7, warn log caused by configuration
loaded twice is displayed every time. And when job completed, this function
will display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:31
[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gera Shegalov updated MAPREDUCE-6271:
-
Status: Open (was: Patch Available)
org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
-
Key: MAPREDUCE-6271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Attachments: MR-6271.patch
When using getJob() with MapReduce 2.7, warn log caused by configuration
loaded twice is displayed every time. And when job completed, this function
will display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:31 WARN
[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peng Zhang updated MAPREDUCE-6271:
--
Attachment: MR-6271.patch
simple patch
org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
-
Key: MAPREDUCE-6271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Peng Zhang
Assignee: J.Andreina
Attachments: MR-6271.patch
When using getJob() with MapReduce 2.7, warn log caused by configuration
loaded twice is displayed every time. And when job completed, this function
will display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:31 WARN mapreduce.Cluster:
[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peng Zhang updated MAPREDUCE-6271:
--
Assignee: Peng Zhang (was: J.Andreina)
Status: Patch Available (was: Open)
org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
-
Key: MAPREDUCE-6271
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6271
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.7.0
Reporter: Peng Zhang
Assignee: Peng Zhang
Attachments: MR-6271.patch
When using getJob() with MapReduce 2.7, warn log caused by configuration
loaded twice is displayed every time. And when job completed, this function
will display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
[jira] [Updated] (MAPREDUCE-6271) org.apache.hadoop.mapreduce.Cluster GetJob() display warn log
[
https://issues.apache.org/jira/browse/MAPREDUCE-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peng Zhang updated MAPREDUCE-6271:
--
Description:
When using getJob() with MapReduce 2.7, warn log caused by configuration loaded
twice is displayed every time. And when job completed, this function will
display warn log of java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:23 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:24 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:25 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:26 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:27 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:28 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfsG://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:29 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:29 INFO exec.Task: 2015-03-04 13:41:29,853 Stage-1 map = 100%,
reduce = 0%, Cumulative CPU 2.37 sec
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.retry.interval; Ignoring.
15/03/04 13:41:30 WARN conf.Configuration:
hdfs://example/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml:an
attempt to override final parameter:
mapreduce.job.end-notification.max.attempts; Ignoring.
15/03/04 13:41:31 WARN mapreduce.Cluster: Job conf missing on
clusterjava.io.FileNotFoundException: File does not exist:
/yarn/example2/staging/test_user/.staging/job_1425388652704_0116/job.xml
{noformat}
Environment: (was: When using getJob() with MapReduce 2.7, warn log
caused by configuration loaded twice is displayed every time. And when job
completed, this function will display warn log of
java.io.FileNotFoundException
And I think this is related with MAPREDUCE-5875, the change in GetJob() seems
to be not needed, cause it's only for test.
{noformat}
15/03/04 13:41:23 WARN
