[jira] [Commented] (MAPREDUCE-6052) Support overriding log4j.properties per job

2014-10-21 Thread Junping Du (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178488#comment-14178488
 ] 

Junping Du commented on MAPREDUCE-6052:
---

bq. Was asked to comment. This makes sense. Let's just make it a 
user-configuration of a local log4j file that we accept and add it 
automatically to the distributed-cache.
Hi [~vinodkv], can you clarify more on this? I think user can explicitly use 
the configuration of -files log4j.properties to add file to distributed 
cached and deliver to each node. Right? So may be we should do here is to add 
the parent directory of dc files to classpath by default. Attaching a patch to 
demonstrate this.


 Support overriding log4j.properties per job
 ---

 Key: MAPREDUCE-6052
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6052.patch


 For current MR application, the log4j.configuration is hard coded to 
 container-log4j.properties within each node. We still need flexibility to 
 override it per job like what we do in MRV1.
 {code}
   public static void addLog4jSystemProperties(
   String logLevel, long logSize, int numBackups, ListString vargs) {
 vargs.add(-Dlog4j.configuration=container-log4j.properties);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6052) Support overriding log4j.properties per job

2014-10-21 Thread Junping Du (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du updated MAPREDUCE-6052:
--
Attachment: MAPREDUCE-6052-v2.patch

 Support overriding log4j.properties per job
 ---

 Key: MAPREDUCE-6052
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052.patch


 For current MR application, the log4j.configuration is hard coded to 
 container-log4j.properties within each node. We still need flexibility to 
 override it per job like what we do in MRV1.
 {code}
   public static void addLog4jSystemProperties(
   String logLevel, long logSize, int numBackups, ListString vargs) {
 vargs.add(-Dlog4j.configuration=container-log4j.properties);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6009) Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled in a reduce slot

2014-10-21 Thread Sandy Ryza (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178503#comment-14178503
 ] 

Sandy Ryza commented on MAPREDUCE-6009:
---

+1

 Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled 
 in a reduce slot
 

 Key: MAPREDUCE-6009
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6009
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client, job submission
Affects Versions: 1.2.1
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Priority: Blocker
 Attachments: MAPREDUCE-6009.v01-branch-1.2.patch, 
 MAPREDUCE-6009.v02-branch-1.2.patch


 In branch 1 job commit is executed in a JOB_CLEANUP task that may run in 
 either map or reduce slot
 in org.apache.hadoop.mapreduce.Job#setUseNewAPI there is a logic setting 
 new-api flag only for reduce-ful jobs.
 {code}
 if (numReduces != 0) {
   conf.setBooleanIfUnset(mapred.reducer.new-api,
  conf.get(oldReduceClass) == null);
   ...
 {code}
 Therefore, when cleanup runs in a reduce slot, ReduceTask inits using the old 
 API and runs incorrect default OutputCommitter, instead of consulting 
 OutputFormat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6009) Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled in a reduce slot

2014-10-21 Thread Sandy Ryza (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandy Ryza updated MAPREDUCE-6009:
--
   Resolution: Fixed
Fix Version/s: 1.2.2
   1.3.0
   Status: Resolved  (was: Patch Available)

 Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled 
 in a reduce slot
 

 Key: MAPREDUCE-6009
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6009
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: client, job submission
Affects Versions: 1.2.1
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Priority: Blocker
 Fix For: 1.3.0, 1.2.2

 Attachments: MAPREDUCE-6009.v01-branch-1.2.patch, 
 MAPREDUCE-6009.v02-branch-1.2.patch


 In branch 1 job commit is executed in a JOB_CLEANUP task that may run in 
 either map or reduce slot
 in org.apache.hadoop.mapreduce.Job#setUseNewAPI there is a logic setting 
 new-api flag only for reduce-ful jobs.
 {code}
 if (numReduces != 0) {
   conf.setBooleanIfUnset(mapred.reducer.new-api,
  conf.get(oldReduceClass) == null);
   ...
 {code}
 Therefore, when cleanup runs in a reduce slot, ReduceTask inits using the old 
 API and runs incorrect default OutputCommitter, instead of consulting 
 OutputFormat.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6052) Support overriding log4j.properties per job

2014-10-21 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14178729#comment-14178729
 ] 

Hadoop QA commented on MAPREDUCE-6052:
--

{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12676100/MAPREDUCE-6052-v2.patch
  against trunk revision 171f237.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4972//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4972//console

This message is automatically generated.

 Support overriding log4j.properties per job
 ---

 Key: MAPREDUCE-6052
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052.patch


 For current MR application, the log4j.configuration is hard coded to 
 container-log4j.properties within each node. We still need flexibility to 
 override it per job like what we do in MRV1.
 {code}
   public static void addLog4jSystemProperties(
   String logLevel, long logSize, int numBackups, ListString vargs) {
 vargs.add(-Dlog4j.configuration=container-log4j.properties);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-5747) Potential null pointer deference in HsTasksBlock#render()

2014-10-21 Thread Rahul Palamuttam (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rahul Palamuttam updated MAPREDUCE-5747:

  Tags: Newbie
Labels: newbie patch  (was: )
Status: Patch Available  (was: Open)

TaskAttemptInfo has a constructor that does not require a TaskType variable.
If the type variable is null then use this constructor instead.

I am fairly new here so any help is appreciated!

 Potential null pointer deference in HsTasksBlock#render()
 -

 Key: MAPREDUCE-5747
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5747
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
  Labels: newbie, patch

 At line 140:
 {code}
 } else {
   ta = new TaskAttemptInfo(successful, type, false);
 {code}
 There is no check for type against null.
 TaskAttemptInfo ctor deferences type:
 {code}
   public TaskAttemptInfo(TaskAttempt ta, TaskType type, Boolean isRunning) {
 final TaskAttemptReport report = ta.getReport();
 this.type = type.toString();
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-5747) Potential null pointer deference in HsTasksBlock#render()

2014-10-21 Thread Rahul Palamuttam (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rahul Palamuttam updated MAPREDUCE-5747:

Attachment: MAPREDUCE-5747-1.patch

 Potential null pointer deference in HsTasksBlock#render()
 -

 Key: MAPREDUCE-5747
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5747
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
  Labels: newbie, patch
 Attachments: MAPREDUCE-5747-1.patch


 At line 140:
 {code}
 } else {
   ta = new TaskAttemptInfo(successful, type, false);
 {code}
 There is no check for type against null.
 TaskAttemptInfo ctor deferences type:
 {code}
   public TaskAttemptInfo(TaskAttempt ta, TaskType type, Boolean isRunning) {
 final TaskAttemptReport report = ta.getReport();
 this.type = type.toString();
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-5747) Potential null pointer deference in HsTasksBlock#render()

2014-10-21 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179291#comment-14179291
 ] 

Hadoop QA commented on MAPREDUCE-5747:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12676205/MAPREDUCE-5747-1.patch
  against trunk revision 4baca31.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4973//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4973//console

This message is automatically generated.

 Potential null pointer deference in HsTasksBlock#render()
 -

 Key: MAPREDUCE-5747
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5747
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
  Labels: newbie, patch
 Attachments: MAPREDUCE-5747-1.patch


 At line 140:
 {code}
 } else {
   ta = new TaskAttemptInfo(successful, type, false);
 {code}
 There is no check for type against null.
 TaskAttemptInfo ctor deferences type:
 {code}
   public TaskAttemptInfo(TaskAttempt ta, TaskType type, Boolean isRunning) {
 final TaskAttemptReport report = ta.getReport();
 this.type = type.toString();
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6052) Support overriding log4j.properties per job

2014-10-21 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179407#comment-14179407
 ] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-6052:


bq. Hi Vinod Kumar Vavilapalli, can you clarify more on this? I think user can 
explicitly use the configuration of -files log4j.properties to add file to 
distributed cached and deliver to each node. Right? So may be we should do here 
is to add the parent directory of dc files to classpath by default. Attaching a 
patch to demonstrate this.
It's weird that we are only letting users configure the _name_ of the log4j 
config file but not letting them specify what the _file itself_ is. I'd rather 
have an option to simply say hey this is my log4j configuration file on HDFS, 
go use it. 

 Support overriding log4j.properties per job
 ---

 Key: MAPREDUCE-6052
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052.patch


 For current MR application, the log4j.configuration is hard coded to 
 container-log4j.properties within each node. We still need flexibility to 
 override it per job like what we do in MRV1.
 {code}
   public static void addLog4jSystemProperties(
   String logLevel, long logSize, int numBackups, ListString vargs) {
 vargs.add(-Dlog4j.configuration=container-log4j.properties);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-5747) Potential null pointer deference in HsTasksBlock#render()

2014-10-21 Thread Rahul Palamuttam (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179451#comment-14179451
 ] 

Rahul Palamuttam commented on MAPREDUCE-5747:
-

Waiting for review.

Also is there need for a test on this patch?
I don't see the TestHsTasksBlock.java file in the test directory.

 Potential null pointer deference in HsTasksBlock#render()
 -

 Key: MAPREDUCE-5747
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5747
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
  Labels: newbie, patch
 Attachments: MAPREDUCE-5747-1.patch


 At line 140:
 {code}
 } else {
   ta = new TaskAttemptInfo(successful, type, false);
 {code}
 There is no check for type against null.
 TaskAttemptInfo ctor deferences type:
 {code}
   public TaskAttemptInfo(TaskAttempt ta, TaskType type, Boolean isRunning) {
 final TaskAttemptReport report = ta.getReport();
 this.type = type.toString();
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6052) Support overriding log4j.properties per job

2014-10-21 Thread Vinod Kumar Vavilapalli (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179455#comment-14179455
 ] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-6052:


bq. hey this is my log4j configuration file on HDFS, go use it
That or hey this is my log4j configuration file on the local-disk, go add it 
to dist-cache and use it for all my tasks.

 Support overriding log4j.properties per job
 ---

 Key: MAPREDUCE-6052
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052.patch


 For current MR application, the log4j.configuration is hard coded to 
 container-log4j.properties within each node. We still need flexibility to 
 override it per job like what we do in MRV1.
 {code}
   public static void addLog4jSystemProperties(
   String logLevel, long logSize, int numBackups, ListString vargs) {
 vargs.add(-Dlog4j.configuration=container-log4j.properties);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6052) Support overriding log4j.properties per job

2014-10-21 Thread Junping Du (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179464#comment-14179464
 ] 

Junping Du commented on MAPREDUCE-6052:
---

bq. That or hey this is my log4j configuration file on the local-disk, go add 
it to dist-cache and use it for all my tasks.
If so, user can add -files log4j.properties as generic option which can 
distribute local file to all target hosts. Right?

 Support overriding log4j.properties per job
 ---

 Key: MAPREDUCE-6052
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052.patch


 For current MR application, the log4j.configuration is hard coded to 
 container-log4j.properties within each node. We still need flexibility to 
 override it per job like what we do in MRV1.
 {code}
   public static void addLog4jSystemProperties(
   String logLevel, long logSize, int numBackups, ListString vargs) {
 vargs.add(-Dlog4j.configuration=container-log4j.properties);
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-5747) Potential null pointer deference in HsTasksBlock#render()

2014-10-21 Thread Ted Yu (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179471#comment-14179471
 ] 

Ted Yu commented on MAPREDUCE-5747:
---

See TestBlocks which has HsTasksBlockForTest

Thanks

 Potential null pointer deference in HsTasksBlock#render()
 -

 Key: MAPREDUCE-5747
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5747
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
  Labels: newbie, patch
 Attachments: MAPREDUCE-5747-1.patch


 At line 140:
 {code}
 } else {
   ta = new TaskAttemptInfo(successful, type, false);
 {code}
 There is no check for type against null.
 TaskAttemptInfo ctor deferences type:
 {code}
   public TaskAttemptInfo(TaskAttempt ta, TaskType type, Boolean isRunning) {
 final TaskAttemptReport report = ta.getReport();
 this.type = type.toString();
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6126) (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type

2014-10-21 Thread Jian He (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179529#comment-14179529
 ] 

Jian He commented on MAPREDUCE-6126:


in JobHistoryEventHandler, seems we already skip writing this event
{code}
HistoryEvent historyEvent = event.getHistoryEvent();
if (! (historyEvent instanceof NormalizedResourceEvent)) {
  mi.writeEvent(historyEvent);
}
{code}

 (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: 
 JobBuilder.process(HistoryEvent): unknown event type
 --

 Key: MAPREDUCE-6126
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6126
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6126.patch


 java.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown 
 event type 
 at org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:172) 
 at 
 org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:305)
 at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:259) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) 
 at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:186) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6126) (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type

2014-10-21 Thread Junping Du (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179537#comment-14179537
 ] 

Junping Du commented on MAPREDUCE-6126:
---

Thanks [~jianhe] for review and comments. 
Yes. But I cannot find other event that could cause this exception. May be 
somewhere else could write NormalizedResourceEvent as well? Anyway, I think the 
safe way to do is: handle NormalizedResourceEvent in JobBuilder but log a warn 
message to say that event should be ignored before. Also, I think we should 
improve the log to log the actual event that cannot be handled. 
Will deliver a quick update on patch soon.

 (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: 
 JobBuilder.process(HistoryEvent): unknown event type
 --

 Key: MAPREDUCE-6126
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6126
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6126.patch


 java.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown 
 event type 
 at org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:172) 
 at 
 org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:305)
 at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:259) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) 
 at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:186) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6126) (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type

2014-10-21 Thread Junping Du (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du updated MAPREDUCE-6126:
--
Attachment: MAPREDUCE-6126-v2.patch

 (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: 
 JobBuilder.process(HistoryEvent): unknown event type
 --

 Key: MAPREDUCE-6126
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6126
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6126-v2.patch, MAPREDUCE-6126.patch


 java.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown 
 event type 
 at org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:172) 
 at 
 org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:305)
 at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:259) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) 
 at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:186) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6126) (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type

2014-10-21 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14179559#comment-14179559
 ] 

Hadoop QA commented on MAPREDUCE-6126:
--

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12676257/MAPREDUCE-6126-v2.patch
  against trunk revision c0e0343.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 eclipse:eclipse{color}.  The patch built with 
eclipse:eclipse.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in 
hadoop-tools/hadoop-rumen.

{color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4974//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4974//console

This message is automatically generated.

 (Rumen) Rumen tool returns error ava.lang.IllegalArgumentException: 
 JobBuilder.process(HistoryEvent): unknown event type
 --

 Key: MAPREDUCE-6126
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6126
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Junping Du
Assignee: Junping Du
 Attachments: MAPREDUCE-6126-v2.patch, MAPREDUCE-6126.patch


 java.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown 
 event type 
 at org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:172) 
 at 
 org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:305)
 at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:259) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) 
 at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:186) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)