date:20150204


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305542#comment-14305542
 ] 

Hadoop QA commented on MAPREDUCE-6243:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12696496/MAPREDUCE-6243.002.patch
  against trunk revision ba58da2.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-tools/hadoop-rumen.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5153//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5153//console

This message is automatically generated.

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, MAPREDUCE-6243.002.patch, 
 findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305558#comment-14305558
 ] 

Hudson commented on MAPREDUCE-6243:
---

FAILURE: Integrated in Hadoop-trunk-Commit #7002 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/7002/])
MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. Contributed by Masatake 
Iwasaki. (aajisaka: rev 34fe11c987730932f99dec6eb458a22624eb075b)
* 
hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ReduceAttempt20LineHistoryEventEmitter.java
* 
hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/Hadoop20JHParser.java
* 
hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/HadoopLogsAnalyzer.java
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/MapAttempt20LineHistoryEventEmitter.java
* 
hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/RandomSeedGenerator.java
* 
hadoop-tools/hadoop-rumen/src/main/java/org/apache/hadoop/tools/rumen/ParsedConfigFile.java


 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Fix For: 2.7.0

 Attachments: MAPREDUCE-6243.001.patch, MAPREDUCE-6243.002.patch, 
 findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305516#comment-14305516
 ] 

Akira AJISAKA commented on MAPREDUCE-6243:
--

LGTM, +1. Pending Jenkins.

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, MAPREDUCE-6243.002.patch, 
 findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6243:
-
Issue Type: Bug  (was: Improvement)

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, MAPREDUCE-6243.002.patch, 
 findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen

2015-02-04 Thread Masatake Iwasaki (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Masatake Iwasaki updated MAPREDUCE-6243:

Status: Patch Available  (was: Open)

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen

2015-02-04 Thread Masatake Iwasaki (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Masatake Iwasaki updated MAPREDUCE-6243:

Attachment: MAPREDUCE-6243.001.patch

attached patch.

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6242) Progress report log is incredibly excessive in application master

2015-02-04 Thread Jian Fang (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305738#comment-14305738
 ] 

Jian Fang commented on MAPREDUCE-6242:
--

This seems to be easy to fix. Could you please let me know when the patch will 
be available? We have production issues due to this.


 Progress report log is incredibly excessive in application master
 -

 Key: MAPREDUCE-6242
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6242
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: applicationmaster
Affects Versions: 2.4.0
Reporter: Jian Fang
Assignee: Varun Saxena

 We saw incredibly excessive logs in application master for a long running one 
 with many task attempts. The log write rate is around 1MB/sec in some cases. 
 Most of the log entries were from the progress report such as the following 
 ones.
 2015-02-03 17:46:14,321 INFO [IPC Server handler 56 on 37661] 
 org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
 attempt_1422985365246_0001_m_00_0 is : 0.15605757
 2015-02-03 17:46:17,581 INFO [IPC Server handler 2 on 37661] 
 org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
 attempt_1422985365246_0001_m_00_0 is : 0.4108217
 2015-02-03 17:46:20,426 INFO [IPC Server handler 0 on 37661] 
 org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
 attempt_1422985365246_0001_m_02_0 is : 0.06634143
 2015-02-03 17:46:20,807 INFO [IPC Server handler 4 on 37661] 
 org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
 attempt_1422985365246_0001_m_00_0 is : 0.6506
 2015-02-03 17:46:21,013 INFO [IPC Server handler 6 on 37661] 
 org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
 attempt_1422985365246_0001_m_01_0 is : 0.21723115
 Looks like the report interval is controlled by a hard-coded variable 
 PROGRESS_INTERVAL as 3 seconds in class org.apache.hadoop.mapred.Task. We 
 should allow users to set the appropriate progress interval for their 
 applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6237) DBRecordReader is not thread safe

2015-02-04 Thread Kannan Rajah (JIRA)

[
https://issues.apache.org/jira/browse/MAPREDUCE-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305720#comment-14305720
]

Kannan Rajah commented on MAPREDUCE-6237:
-

I understand that this causes a change in behavior. But I would like to point
out my understanding of how this API is meant to be used. If you still feel
that I should bring back the old behavior, I will do so.

- getConnection is a public method, so anyone can access it. I don't know why
it was done this way because it breaks encapsulation. Except for the
subclasses, no one else should care about accessing the Connection object. In
fact, subclasses can already access it without this method because its a
protected variable.

- But to avoid breaking subclasses that already use this API to access the
connection, we need to keep it. So I updated the patch.

- The connection object is created inside setConf method. So it is guaranteed
to be not null unless the caller explicitly called the close method and then
tried to access the connection again. This is an improper usage of the API. I
doubt if anyone would be using it that way. But if a subclass is doing this,
then it is better to fix that.

DBRecordReader is not thread safe
-

Key: MAPREDUCE-6237
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6237
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.5.0
Reporter: Kannan Rajah
Assignee: Kannan Rajah
Attachments: mapreduce-6237.patch, mapreduce-6237.patch

DBInputFormat.createDBRecorder is reusing JDBC connections across instances
of DBRecordReader. This is not a good idea. We should be creating separate
connection. If performance is a concern, then we should be using connection
pooling instead.
I looked at DBOutputFormat.getRecordReader. It actually creates a new
Connection object for each DBRecordReader. So can we just change
DBInputFormat to create new Connection every time? The connection reuse code
was added as part of connection leak bug in MAPREDUCE-1443. Any reason for
caching the connection?
We observed this issue in a customer setup where they were reading data from
MySQL using Pig. As per customer, the query is returning two records which
causes Pig to create two instances of DBRecordReader. These two instances are
sharing the database connection instance. The first DBRecordReader runs to
extract the first record from MySQL just fine, but then closes the shared
connection instance. When the second DBRecordReader runs, it tries to execute
a query to retrieve the second record on the closed shared connection
instance, which fail. If we set
mapred.map.tasks to 1, the query will be successful.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6240) Hadoop client displays confusing error message


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated MAPREDUCE-6240:
-
Attachment: MAPREDUCE-6240-gera.001.patch

I don't think any of these failures related. Resubmitting the same patch to get 
another Jenkins run.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message

2015-02-04 Thread Mohammad Kamrul Islam (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306015#comment-14306015
 ] 

Mohammad Kamrul Islam commented on MAPREDUCE-6240:
--

[~jira.shegalov] Thanks for your comments.
I also thought the similar way: not to loose the individual stack traces. But 
I'm not sure chaining is the best approach.
The reason: the exceptions created in the loop are not related by Caused by. 
They are all independent. 

Based on this understanding, I want to stringify all stacktraces and then 
append then as one error message with either .. or ..or  pattern. It is not 
fully clean though.

Do you have any suggestion?



 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305964#comment-14305964
 ] 

Gera Shegalov commented on MAPREDUCE-6240:
--

I also ran the tests locally, only TestJobConf MAPREDUCE-6223 failed.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306028#comment-14306028
 ] 

Gera Shegalov commented on MAPREDUCE-6240:
--

The intuition behind chaining is the following. We would not try provider 2 if 
provider 1 worked, in other words: our provider 2 failures are indirectly 
caused by provider 1 failures.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message

2015-02-04 Thread Mohammad Kamrul Islam (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306109#comment-14306109
 ] 

Mohammad Kamrul Islam commented on MAPREDUCE-6240:
--

if provider1 and provider2 both failed, the user will get only the provider 1 
error. He will not get any error for provider 2 that may be resolved easily by 
the user.
My understanding is : if any one provider is fine then  cluster is initialized 
properly. This good case is handled correctly. But when all failed, we would 
not have full stack traces for both. In some instances,  resolving the provider 
2 may be easier than provider 1. 
right?



 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-5718) MR job will fail after commit fail

2015-02-04 Thread Jason Lowe (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305180#comment-14305180
 ] 

Jason Lowe commented on MAPREDUCE-5718:
---

The issue is that the OutputCommitter is user-specified code.  It may not be 
doing filesystem operations at all during the commit (e.g.: committing to a 
database, a REST API, etc.), and that procedure may not be restartable without 
the chance of corrupting or losing data.  Yes, FileOutputCommitter's commit 
procedure is something that can be restarted, but jobs are not required to use 
FileOutputCommitter nor dump their output in a file at all.  That's why 
MAPREDUCE-5485 was filed, because to do this safely the framework needs an 
indication from the output committer whether or not commit is a restartable 
procedure.

 MR job will fail after commit fail
 --

 Key: MAPREDUCE-5718
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5718
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mr-am
Affects Versions: 2.3.0, 2.6.0
Reporter: Karthik Kambatla
Assignee: Yang Hao
 Fix For: 2.6.0

 Attachments: MAPREDUCE-5718.v2.patch, mr-5718-0.patch


 when any of this happens:
 * While testing RM HA, if the RM fails over while an MR AM is in the middle 
 of a commit, 
 * When testing preempting, if the MR AM fails over during the middle of a 
 commit
 the subsequent AM gets spawned but dies with a diagnostic message - We 
 crashed durring a commit. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6237) DBRecordReader is not thread safe

2015-02-04 Thread Tsuyoshi OZAWA (JIRA)

[
https://issues.apache.org/jira/browse/MAPREDUCE-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306716#comment-14306716
]

Tsuyoshi OZAWA commented on MAPREDUCE-6237:
---

Thank you for clarification. I know you're correct and my suggestion is not
clean fix, but I think this issue is one of critical issues to be fixed for
branch-2 and 2.6.1. To commit this to branch-2 and 2.6.1, incompatible changes
are not accepted since it's public class. My suggestion is:

* Let's fix the bug here with full backward compatibility about public methods
to commit this to branch-2.
* How about fixing the behavior cleanly which targets trunk on separate JIRA?

DBRecordReader is not thread safe
-

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306243#comment-14306243
 ] 

Hadoop QA commented on MAPREDUCE-6240:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12696556/MAPREDUCE-6240-gera.001.patch
  against trunk revision e04e8fa.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 13 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-common-project/hadoop-common 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

  org.apache.hadoop.mapreduce.TestLargeSort
  org.apache.hadoop.conf.TestJobConf

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5154//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5154//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5154//console

This message is automatically generated.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch, 
 MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at

[jira] [Updated] (MAPREDUCE-6240) Hadoop client displays confusing error message


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gera Shegalov updated MAPREDUCE-6240:
-
Attachment: MAPREDUCE-6240-gera.002.patch

bq. if provider1 and provider2 both failed, the user will get only the provider 
1 error. He will not get any error for provider 2 that may be resolved easily 
by the user.

Good catch, [~kamrul]! I was talking about my intention to chain all exceptions 
but had a typo.  I was supposed to be:
{code}
reason = new IOException(Failed to use  +
  provider.getClass().getName(), rootCause);
{code}
Uploading 002.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch, 
 MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Moved] (MAPREDUCE-6244) Hadoop examples when run without an argument, gives ERROR instead of just usage info


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA moved HADOOP-8834 to MAPREDUCE-6244:
--

Affects Version/s: (was: trunk-win)
   (was: 0.23.0)
   trunk-win
   0.23.0
   2.6.0
   Issue Type: Improvement  (was: Bug)
  Key: MAPREDUCE-6244  (was: HADOOP-8834)
  Project: Hadoop Map/Reduce  (was: Hadoop Common)

 Hadoop examples when run without an argument, gives ERROR instead of just 
 usage info
 

 Key: MAPREDUCE-6244
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6244
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 2.6.0, 0.23.0, trunk-win
Reporter: Robert Justice
Assignee: Abhishek Kapoor
Priority: Minor
 Attachments: HADOOP-8834.patch, HADOOP-8834.patch


 Hadoop sort example should not give an ERROR and only should display usage 
 when run with no parameters. 
 {code}
 $ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar sort
 ERROR: Wrong number of parameters: 0 instead of 2.
 sort [-m maps] [-r reduces] [-inFormat input format class] [-outFormat 
 output format class] [-outKey output key class] [-outValue output value 
 class] [-totalOrder pcnt num samples max splits] input output
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6244) Hadoop examples when run without an argument, gives ERROR instead of just usage info


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306328#comment-14306328
 ] 

Akira AJISAKA commented on MAPREDUCE-6244:
--

Moved to MapReduce project.

 Hadoop examples when run without an argument, gives ERROR instead of just 
 usage info
 

 Key: MAPREDUCE-6244
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6244
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 0.23.0, trunk-win, 2.6.0
Reporter: Robert Justice
Assignee: Abhishek Kapoor
Priority: Minor
 Attachments: HADOOP-8834.patch, HADOOP-8834.patch


 Hadoop sort example should not give an ERROR and only should display usage 
 when run with no parameters. 
 {code}
 $ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar sort
 ERROR: Wrong number of parameters: 0 instead of 2.
 sort [-m maps] [-r reduces] [-inFormat input format class] [-outFormat 
 output format class] [-outKey output key class] [-outValue output value 
 class] [-totalOrder pcnt num samples max splits] input output
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306322#comment-14306322
 ] 

Gera Shegalov commented on MAPREDUCE-6240:
--

[~chris.douglas], thanks for the pointer, we should use MultipleIOException for 
consistency.  AFS change is visibility for the unit test.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch, 
 MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6240) Hadoop client displays confusing error message

2015-02-04 Thread Chris Douglas (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Douglas updated MAPREDUCE-6240:
-
Status: Open  (was: Patch Available)

bq. The intuition behind chaining is the following. We would not try provider 2 
if provider 1 worked, in other words: our provider 2 failures are indirectly 
caused by provider 1 failures.

Wait, this is synthetically stitching exceptions together from a retry loop? 
That would be very confusing to debug. Have you looked at an approach like 
[MultipleIOException|https://git1-us-west.apache.org/repos/asf?p=hadoop.git;a=blob;f=hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/MultipleIOException.java;h=5e584c9cd0705471a826932d782eec409b5bae37;hb=HEAD]?

There's also a spurious change to {{AbstractFileSystem}} in the latest patch.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch, 
 MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6059) Speed up history server startup time

2015-02-04 Thread Allen Wittenauer (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allen Wittenauer updated MAPREDUCE-6059:

   Resolution: Fixed
Fix Version/s: 3.0.0
   Status: Resolved  (was: Patch Available)

seems reasonable.

+1 committing to trunk.

Thanks!

 Speed up history server startup time
 

 Key: MAPREDUCE-6059
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6059
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Siqi Li
Assignee: Siqi Li
 Fix For: 3.0.0

 Attachments: YARN-2366.v1.patch


 When history server starts up, It scans every history directories and put all 
 history files into a cache, whereas this cache only stores 20K recent history 
 files. Therefore, it is wasting a large portion of time loading old history 
 files into the cache, and the startup time will keep increasing if we don't 
 trim the number of history files. For example, when history server starts up 
 with 2.5M history files in HDFS, it took ~5 minutes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306387#comment-14306387
 ] 

Hadoop QA commented on MAPREDUCE-6240:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12696588/MAPREDUCE-6240-gera.002.patch
  against trunk revision 5f4ef2d.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:red}-1 findbugs{color}.  The patch appears to introduce 13 new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in 
hadoop-common-project/hadoop-common 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

  org.apache.hadoop.ipc.TestIPC
  
org.apache.hadoop.security.token.delegation.web.TestWebDelegationToken
  org.apache.hadoop.conf.TestJobConf
  org.apache.hadoop.mapreduce.TestLargeSort

  The following test timeouts occurred in 
hadoop-common-project/hadoop-common 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:

org.apache.hadoop.mapred.TestLazyOutput

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5155//testReport/
Findbugs warnings: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5155//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5155//console

This message is automatically generated.

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch, 
 MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for

[jira] [Commented] (MAPREDUCE-5988) Fix dead links to the javadocs in mapreduce project

2015-02-04 Thread Allen Wittenauer (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306411#comment-14306411
 ] 

Allen Wittenauer commented on MAPREDUCE-5988:
-

+1 lgtm

 Fix dead links to the javadocs in mapreduce project
 ---

 Key: MAPREDUCE-5988
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5988
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.4.1
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Priority: Minor
 Attachments: MAPREDUCE-5988.2.patch, MAPREDUCE-5988.patch


 In http://hadoop.apache.org/docs/r2.4.1/api/allclasses-frame.html, some 
 classes are listed, but not documented.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6059) Speed up history server startup time


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306424#comment-14306424
 ] 

Hudson commented on MAPREDUCE-6059:
---

SUCCESS: Integrated in Hadoop-trunk-Commit #7013 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/7013/])
MAPREDUCE-6059. Speed up history server startup time (Siqi Li via aw) (aw: rev 
fd57ab2002f97dcc83d455a5e0c770c8efde77a4)
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java


 Speed up history server startup time
 

 Key: MAPREDUCE-6059
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6059
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Siqi Li
Assignee: Siqi Li
 Fix For: 3.0.0

 Attachments: YARN-2366.v1.patch


 When history server starts up, It scans every history directories and put all 
 history files into a cache, whereas this cache only stores 20K recent history 
 files. Therefore, it is wasting a large portion of time loading old history 
 files into the cache, and the startup time will keep increasing if we don't 
 trim the number of history files. For example, when history server starts up 
 with 2.5M history files in HDFS, it took ~5 minutes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6244) Hadoop examples when run without an argument, gives ERROR instead of just usage info


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306386#comment-14306386
 ] 

Hadoop QA commented on MAPREDUCE-6244:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12571589/HADOOP-8834.patch
  against trunk revision 43d5cae.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-mapreduce-project/hadoop-mapreduce-examples.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5156//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5156//console

This message is automatically generated.

 Hadoop examples when run without an argument, gives ERROR instead of just 
 usage info
 

 Key: MAPREDUCE-6244
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6244
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 0.23.0, trunk-win, 2.6.0
Reporter: Robert Justice
Assignee: Abhishek Kapoor
Priority: Minor
 Attachments: HADOOP-8834.patch, HADOOP-8834.patch


 Hadoop sort example should not give an ERROR and only should display usage 
 when run with no parameters. 
 {code}
 $ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar sort
 ERROR: Wrong number of parameters: 0 instead of 2.
 sort [-m maps] [-r reduces] [-inFormat input format class] [-outFormat 
 output format class] [-outKey output key class] [-outValue output value 
 class] [-totalOrder pcnt num samples max splits] input output
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-5988) Fix dead links to the javadocs in mapreduce project


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-5988:
-
   Resolution: Fixed
Fix Version/s: 2.7.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed this to trunk and branch-2. Thanks [~aw] for the review!

 Fix dead links to the javadocs in mapreduce project
 ---

 Key: MAPREDUCE-5988
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5988
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.4.1
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Priority: Minor
 Fix For: 2.7.0

 Attachments: MAPREDUCE-5988.2.patch, MAPREDUCE-5988.patch


 In http://hadoop.apache.org/docs/r2.4.1/api/allclasses-frame.html, some 
 classes are listed, but not documented.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-5988) Fix dead links to the javadocs in mapreduce project


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306443#comment-14306443
 ] 

Hudson commented on MAPREDUCE-5988:
---

FAILURE: Integrated in Hadoop-trunk-Commit #7014 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/7014/])
MAPREDUCE-5988. Fix dead links to the javadocs in mapreduce project. (aajisaka) 
(aajisaka: rev cc6bbfceae1cddfae6a3892cb7e7104531a689be)
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/api/protocolrecords/package-info.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/package-info.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/package-info.java


 Fix dead links to the javadocs in mapreduce project
 ---

 Key: MAPREDUCE-5988
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5988
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.4.1
Reporter: Akira AJISAKA
Assignee: Akira AJISAKA
Priority: Minor
 Fix For: 2.7.0

 Attachments: MAPREDUCE-5988.2.patch, MAPREDUCE-5988.patch


 In http://hadoop.apache.org/docs/r2.4.1/api/allclasses-frame.html, some 
 classes are listed, but not documented.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-5718) MR job will fail after commit fail

2015-02-04 Thread Yang Hao (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yang Hao updated MAPREDUCE-5718:

Affects Version/s: (was: 2.3.0)
   Status: Open  (was: Patch Available)

 MR job will fail after commit fail
 --

 Key: MAPREDUCE-5718
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5718
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mr-am
Affects Versions: 2.6.0
Reporter: Karthik Kambatla
Assignee: Yang Hao
 Fix For: 2.6.0

 Attachments: MAPREDUCE-5718.v2.patch, mr-5718-0.patch


 when any of this happens:
 * While testing RM HA, if the RM fails over while an MR AM is in the middle 
 of a commit, 
 * When testing preempting, if the MR AM fails over during the middle of a 
 commit
 the subsequent AM gets spawned but dies with a diagnostic message - We 
 crashed durring a commit. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6240) Hadoop client displays confusing error message

2015-02-04 Thread Chris Douglas (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306365#comment-14306365
 ] 

Chris Douglas commented on MAPREDUCE-6240:
--

I see. Would it be possible to construct a test that's both more direct for the 
fix and doesn't require a change to common? Possibly initialize two providers 
that throw different exceptions, then verify that both messages are in the 
output. Just looking at the patch so I don't know the context, but wouldn't the 
old code pass, also?

 Hadoop client displays confusing error message
 --

 Key: MAPREDUCE-6240
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6240
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
 Attachments: MAPREDUCE-6240-gera.001.patch, 
 MAPREDUCE-6240-gera.001.patch, MAPREDUCE-6240-gera.002.patch, 
 MAPREDUCE-6240.1.patch


 Hadoop client often throws exception  with java.io.IOException: Cannot 
 initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 This is a misleading and generic message for any cluster initialization 
 problem. It takes a lot of debugging hours to identify the root cause. The 
 correct error message could resolve this problem quickly.
 In one such instance, Oozie log showed the following exception  while the 
 root cause was CNF  that Hadoop client didn't return in the exception.
 {noformat}
  JA009: Cannot initialize Cluster. Please check your configuration for 
 mapreduce.framework.name and the correspond server addresses.
 at 
 org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:412)
 at 
 org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:979)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1134)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
 at 
 org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
 at org.apache.oozie.command.XCommand.call(XCommand.java:281)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
 at 
 org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
 at 
 org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:744)
 Caused by: java.io.IOException: Cannot initialize Cluster. Please check your 
 configuration for mapreduce.framework.name and the correspond server 
 addresses.
 at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:120)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82)
 at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:470)
 at org.apache.hadoop.mapred.JobClient.init(JobClient.java:449)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:372)
 at 
 org.apache.oozie.service.HadoopAccessorService$1.run(HadoopAccessorService.java:370)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
 at 
 org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:379)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:1185)
 at 
 org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:927)
  ... 10 more
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (MAPREDUCE-6059) Speed up history server startup time

2015-02-04 Thread Allen Wittenauer (JIRA)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Allen Wittenauer updated MAPREDUCE-6059:

Issue Type: Improvement  (was: Bug)

 Speed up history server startup time
 

 Key: MAPREDUCE-6059
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6059
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Siqi Li
Assignee: Siqi Li
 Attachments: YARN-2366.v1.patch


 When history server starts up, It scans every history directories and put all 
 history files into a cache, whereas this cache only stores 20K recent history 
 files. Therefore, it is wasting a large portion of time loading old history 
 files into the cache, and the startup time will keep increasing if we don't 
 trim the number of history files. For example, when history server starts up 
 with 2.5M history files in HDFS, it took ~5 minutes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6244) Hadoop examples when run without an argument, gives ERROR instead of just usage info

2015-02-04 Thread Tsuyoshi OZAWA (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14306786#comment-14306786
 ] 

Tsuyoshi OZAWA commented on MAPREDUCE-6244:
---

I checked WordCount job, but it also prints ERROR message. Meanwhile, 
ValueAggregatorJob.createValueAggregatorJob just prints usage. We should 
inspect all job to make their behavior consistent. My thought is that it's 
enough to print usages when the number of given arguments are wrong.

 Hadoop examples when run without an argument, gives ERROR instead of just 
 usage info
 

 Key: MAPREDUCE-6244
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6244
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Affects Versions: 0.23.0, trunk-win, 2.6.0
Reporter: Robert Justice
Assignee: Abhishek Kapoor
Priority: Minor
 Attachments: HADOOP-8834.patch, HADOOP-8834.patch


 Hadoop sort example should not give an ERROR and only should display usage 
 when run with no parameters. 
 {code}
 $ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar sort
 ERROR: Wrong number of parameters: 0 instead of 2.
 sort [-m maps] [-r reduces] [-inFormat input format class] [-outFormat 
 output format class] [-outKey output key class] [-outValue output value 
 class] [-totalOrder pcnt num samples max splits] input output
 Generic options supported are
 -conf configuration file specify an application configuration file
 -D property=valueuse value for given property
 -fs local|namenode:port  specify a namenode
 -jt local|jobtracker:portspecify a job tracker
 -files comma separated list of filesspecify comma separated files to be 
 copied to the map reduce cluster
 -libjars comma separated list of jarsspecify comma separated jar files 
 to include in the classpath.
 -archives comma separated list of archivesspecify comma separated 
 archives to be unarchived on the compute machines.
 The general command line syntax is
 bin/hadoop command [genericOptions] [commandOptions]
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304995#comment-14304995
 ] 

Akira AJISAKA commented on MAPREDUCE-6243:
--

Thanks [~iwasakims] for the patch. Here are two comments:
{code}
+  InputStream is = new 
ByteArrayInputStream(xmlString.getBytes(Charsets.UTF_8));
{code}
# Would you render a line within 80 characters?
# I prefer {{java.nio.Charset.forName(UTF-8)}} to 
{{org.google.common.base.Charsets.UTF-8}} because the latter needs an extra 
dependency. However, the latter is already used in hadoop source code, so this 
comment is trivial. (Eventually, you can use either of them.)

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6237) DBRecordReader is not thread safe

2015-02-04 Thread Tsuyoshi OZAWA (JIRA)

[
https://issues.apache.org/jira/browse/MAPREDUCE-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304911#comment-14304911
]

Tsuyoshi OZAWA commented on MAPREDUCE-6237:
---

[~rkannan82] the latest change looks an incompatible change - new
getConnection() method can return null.

The current semantics of getConnection() is to create new connection to DB and
cannot return null. I think we should preserve not only the method name
getConnection() but also the semantics for backward compatibility though I
know the name of getConnection() is confusing. Would you update a patch?

DBRecordReader is not thread safe
-

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304936#comment-14304936
 ] 

Hudson commented on MAPREDUCE-5800:
---

FAILURE: Integrated in Hadoop-Yarn-trunk #828 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/828/])
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. 
(aajisaka) (aajisaka: rev bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordStandardDeviation.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordCount.java
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/RandomTextWriterJob.java
* 
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/aggregate/ValueAggregatorJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/MultiFileWordCount.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/db/DBInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/Parser.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/jobcontrol/ControlledJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/testjar/UserNamePermission.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/LargeSorter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/SecondarySort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/pi/DistSum.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMedian.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Grep.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/jobcontrol/Job.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/TotalOrderPartitioner.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/QuasiMonteCarlo.java
* 
hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/TestCompressionEmulationUtils.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/dancing/DistributedPentomino.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Sort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/DelegatingInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
*

[jira] [Commented] (MAPREDUCE-5718) MR job will fail after commit fail

2015-02-04 Thread Yang Hao (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305036#comment-14305036
 ] 

Yang Hao commented on MAPREDUCE-5718:
-

Hi, I have read the comment twice. It may not be safe to recover under this 
situation. 

I'm wondering whether you are worring about missing data. But during the 
recovery, task output data will be moved to the new output dir, just as the 
normal AM failover does, then it's safe to recover

Our cluster will add this feature. Can you give more specific information on 
the unsafe situation? Thanks a lot

 MR job will fail after commit fail
 --

 Key: MAPREDUCE-5718
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5718
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mr-am
Affects Versions: 2.3.0, 2.6.0
Reporter: Karthik Kambatla
Assignee: Yang Hao
 Fix For: 2.6.0

 Attachments: MAPREDUCE-5718.v2.patch, mr-5718-0.patch


 when any of this happens:
 * While testing RM HA, if the RM fails over while an MR AM is in the middle 
 of a commit, 
 * When testing preempting, if the MR AM fails over during the middle of a 
 commit
 the subsequent AM gets spawned but dies with a diagnostic message - We 
 crashed durring a commit. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-5718) MR job will fail after commit fail

2015-02-04 Thread Yang Hao (JIRA)


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305035#comment-14305035
 ] 

Yang Hao commented on MAPREDUCE-5718:
-

Hi, I have read the comment twice. It may not be safe to recover under this 
situation. 

I'm wondering whether you are worring about missing data. But during the 
recovery, task output data will be moved to the new output dir, just as the 
normal AM failover does, then it's safe to recover

Our cluster will add this feature. Can you give more specific information on 
the unsafe situation? Thanks a lot

 MR job will fail after commit fail
 --

 Key: MAPREDUCE-5718
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5718
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mr-am
Affects Versions: 2.3.0, 2.6.0
Reporter: Karthik Kambatla
Assignee: Yang Hao
 Fix For: 2.6.0

 Attachments: MAPREDUCE-5718.v2.patch, mr-5718-0.patch


 when any of this happens:
 * While testing RM HA, if the RM fails over while an MR AM is in the middle 
 of a commit, 
 * When testing preempting, if the MR AM fails over during the middle of a 
 commit
 the subsequent AM gets spawned but dies with a diagnostic message - We 
 crashed durring a commit. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-6243) Fix findbugs warnings in hadoop-rumen


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304907#comment-14304907
 ] 

Hadoop QA commented on MAPREDUCE-6243:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12696425/MAPREDUCE-6243.001.patch
  against trunk revision 26dee14.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-tools/hadoop-rumen.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5152//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5152//console

This message is automatically generated.

 Fix findbugs warnings in hadoop-rumen
 -

 Key: MAPREDUCE-6243
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6243
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: tools/rumen
Affects Versions: 2.6.0
Reporter: Akira AJISAKA
Assignee: Masatake Iwasaki
Priority: Minor
  Labels: newbie
 Attachments: MAPREDUCE-6243.001.patch, findbugs.xml


 There are 7 findbugs warnings in hadoop-rumen modules.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14304921#comment-14304921
 ] 

Hudson commented on MAPREDUCE-5800:
---

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #94 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/94/])
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. 
(aajisaka) (aajisaka: rev bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/db/DBInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/testjar/UserNamePermission.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/RandomTextWriterJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/TotalOrderPartitioner.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Grep.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Join.java
* 
hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/MultiFileWordCount.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/jobcontrol/Job.java
* 
hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/TestCompressionEmulationUtils.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/jobcontrol/ControlledJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/LargeSorter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordCount.java
* 
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordStandardDeviation.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/DelegatingInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMean.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Sort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/SecondarySort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/dancing/DistributedPentomino.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMedian.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/aggregate/ValueAggregatorJob.java
*

[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305129#comment-14305129
 ] 

Hudson commented on MAPREDUCE-5800:
---

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #91 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/91/])
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. 
(aajisaka) (aajisaka: rev bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/MultiFileWordCount.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/DelegatingInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/LargeSorter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/RandomTextWriterJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Sort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/dancing/DistributedPentomino.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMean.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/db/DBInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Grep.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Join.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/TotalOrderPartitioner.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordStandardDeviation.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/jobcontrol/ControlledJob.java
* 
hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMedian.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/testjar/UserNamePermission.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/jobcontrol/Job.java
* 
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/Parser.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/QuasiMonteCarlo.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/aggregate/ValueAggregatorJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/InputSampler.java
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/SecondarySort.java
*

[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305102#comment-14305102
 ] 

Hudson commented on MAPREDUCE-5800:
---

FAILURE: Integrated in Hadoop-Hdfs-trunk #2026 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/2026/])
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. 
(aajisaka) (aajisaka: rev bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)
* 
hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Join.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordCount.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Grep.java
* 
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/db/DBInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/RandomTextWriterJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Sort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/MultiFileWordCount.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/TotalOrderPartitioner.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/Parser.java
* 
hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/TestCompressionEmulationUtils.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/dancing/DistributedPentomino.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/QuasiMonteCarlo.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/LargeSorter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/aggregate/ValueAggregatorJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMean.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordStandardDeviation.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/SecondarySort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/pi/DistSum.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/DelegatingInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/testjar/UserNamePermission.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMedian.java
*

[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors


[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14305211#comment-14305211
 ] 

Hudson commented on MAPREDUCE-5800:
---

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #95 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/95/])
MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. 
(aajisaka) (aajisaka: rev bd69fb2d44403e930d1fc0868ed1dd2a49dd9659)
* 
hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/mapreduce/TestStreamXmlRecordReader.java
* 
hadoop-tools/hadoop-gridmix/src/main/java/org/apache/hadoop/mapred/gridmix/GridmixJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/LargeSorter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordMedian.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/RandomTextWriterJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Grep.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordCount.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Sort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/aggregate/ValueAggregatorJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/RandomTextWriter.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/QuasiMonteCarlo.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/db/DBInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.java
* 
hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/TestCompressionEmulationUtils.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/DelegatingInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/dancing/DistributedPentomino.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/testjar/UserNamePermission.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/SecondarySort.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileInputFormat.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/pi/DistSum.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRClientCluster.java
* hadoop-mapreduce-project/CHANGES.txt
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/InputSampler.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/jobcontrol/ControlledJob.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/Join.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/partition/TotalOrderPartitioner.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
* 
hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/WordStandardDeviation.java
*

[jira] [Commented] (MAPREDUCE-5800) Use Job#getInstance instead of deprecated constructors