[jira] [Updated] (MAPREDUCE-5812) Make task context available to OutputCommitter.isRecoverySupported()
[ https://issues.apache.org/jira/browse/MAPREDUCE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated MAPREDUCE-5812: - Status: Patch Available (was: Open) Make task context available to OutputCommitter.isRecoverySupported() - Key: MAPREDUCE-5812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5812 Project: Hadoop Map/Reduce Issue Type: Bug Components: mr-am Affects Versions: 2.3.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Attachments: MAPREDUCE-5812.1.patch Background == The system like Hive provides its version of OutputCommitter. The custom implementation of isRecoverySupported() requires task context. From taskContext:getConfiguration(), hive checks if hive-defined specific property is set or not. Based on the property value, it returns true or false. However, in the current OutputCommitter:isRecoverySupported(), there is no way of getting task config. As a result, user can't turn on/off the MRAM recovery feature. Proposed resolution: === 1. Pass Task Context into isRecoverySupported() method. Pros: Easy and clean Cons: Possible backward compatibility issue due to aPI changes. (Is it true?) 2. Call outputCommitter.setupTask(taskContext) from MRAM: The new OutputCommitter will store the context in the class level variable and use it from isRecoverySupported() Props: No API changes. No backward compatibility issue. This call can be made from MRAppMaster.getOutputCommitter() method for old API case. Cons: Might not be very clean solution due to class level variable. Please give your comments. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (MAPREDUCE-5812) Make task context available to OutputCommitter.isRecoverySupported()
[ https://issues.apache.org/jira/browse/MAPREDUCE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated MAPREDUCE-5812: - Attachment: MAPREDUCE-5812.1.patch Patch uploaded based on option 2. Testing done: * New test cases added. Also tested in Hive with an outstanding RM restart patch in a single node cluster. Make task context available to OutputCommitter.isRecoverySupported() - Key: MAPREDUCE-5812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5812 Project: Hadoop Map/Reduce Issue Type: Bug Components: mr-am Affects Versions: 2.3.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Attachments: MAPREDUCE-5812.1.patch Background == The system like Hive provides its version of OutputCommitter. The custom implementation of isRecoverySupported() requires task context. From taskContext:getConfiguration(), hive checks if hive-defined specific property is set or not. Based on the property value, it returns true or false. However, in the current OutputCommitter:isRecoverySupported(), there is no way of getting task config. As a result, user can't turn on/off the MRAM recovery feature. Proposed resolution: === 1. Pass Task Context into isRecoverySupported() method. Pros: Easy and clean Cons: Possible backward compatibility issue due to aPI changes. (Is it true?) 2. Call outputCommitter.setupTask(taskContext) from MRAM: The new OutputCommitter will store the context in the class level variable and use it from isRecoverySupported() Props: No API changes. No backward compatibility issue. This call can be made from MRAppMaster.getOutputCommitter() method for old API case. Cons: Might not be very clean solution due to class level variable. Please give your comments. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5812) Make task context available to OutputCommitter.isRecoverySupported()
[
https://issues.apache.org/jira/browse/MAPREDUCE-5812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954633#comment-13954633
]
Hadoop QA commented on MAPREDUCE-5812:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12637700/MAPREDUCE-5812.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4469//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4469//console
This message is automatically generated.
Make task context available to OutputCommitter.isRecoverySupported()
-
Key: MAPREDUCE-5812
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5812
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.3.0
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Attachments: MAPREDUCE-5812.1.patch
Background
==
The system like Hive provides its version of OutputCommitter. The custom
implementation of isRecoverySupported() requires task context. From
taskContext:getConfiguration(), hive checks if hive-defined specific
property is set or not. Based on the property value, it returns true or
false. However, in the current OutputCommitter:isRecoverySupported(), there
is no way of getting task config. As a result, user can't turn on/off the
MRAM recovery feature.
Proposed resolution:
===
1. Pass Task Context into isRecoverySupported() method.
Pros: Easy and clean
Cons: Possible backward compatibility issue due to aPI changes. (Is it true?)
2. Call outputCommitter.setupTask(taskContext) from MRAM: The new
OutputCommitter will store the context in the class level variable and use it
from isRecoverySupported()
Props: No API changes. No backward compatibility issue. This call can be made
from MRAppMaster.getOutputCommitter() method for old API case.
Cons: Might not be very clean solution due to class level variable.
Please give your comments.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-5810) TestStreamingTaskLog#testStreamingTaskLogWithHadoopCmd is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954645#comment-13954645 ] Hudson commented on MAPREDUCE-5810: --- SUCCESS: Integrated in Hadoop-Yarn-trunk #524 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/524/]) MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. Contributed by Akira Ajisaka. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1582896) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestStreamingTaskLog.java TestStreamingTaskLog#testStreamingTaskLogWithHadoopCmd is failing - Key: MAPREDUCE-5810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5810 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/streaming Affects Versions: 3.0.0, 2.4.0 Reporter: Mit Desai Assignee: Akira AJISAKA Fix For: 2.4.0 Attachments: MAPREDUCE-5810.patch, log.txt testStreamingTaskLogWithHadoopCmd(org.apache.hadoop.streaming.TestStreamingTaskLog) Time elapsed: 44.069 sec FAILURE! java.lang.AssertionError: environment set for child is wrong at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.hadoop.streaming.TestStreamingTaskLog.runStreamJobAndValidateEnv(TestStreamingTaskLog.java:157) at org.apache.hadoop.streaming.TestStreamingTaskLog.testStreamingTaskLogWithHadoopCmd(TestStreamingTaskLog.java:107) Results : Failed tests: TestStreamingTaskLog.testStreamingTaskLogWithHadoopCmd:107-runStreamJobAndValidateEnv:157 environment set for child is wrong -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5813) YarnChild does not load job.xml with mapreduce.job.classloader=true
[
https://issues.apache.org/jira/browse/MAPREDUCE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954654#comment-13954654
]
Hudson commented on MAPREDUCE-5813:
---
SUCCESS: Integrated in Hadoop-Yarn-trunk #524 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/524/])
MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the
local-filesystem, rather than rely on the classpath. Contributed by Gera
Shegalov. (acmurthy:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1583050)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
YarnChild does not load job.xml with mapreduce.job.classloader=true
Key: MAPREDUCE-5813
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5813
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, task
Affects Versions: 2.3.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Priority: Blocker
Fix For: 2.4.0
Attachments: MAPREDUCE-5813.v01.patch, MAPREDUCE-5813.v02.patch
{{YarnChild.main}} uses {{JobConf.addResource(String)}} to load {{job.xml}}
that relies on class loading. When {{mapreduce.job.classloader=true}} the
job-speicific part of the class path is separated from {{CLASSPATH}} into
{{APP_CLASSPATH}}. Therefore {{job.xml}} is inaccessible for the default
class loader. Later {{writeLocalJobFile}} overwrites the correct localized
{{job.xml}} on disk as well.
This problem is easily avoided by using {{JobConf.addResource(Path)}} to
read the localized {{job.xml}} without relying on class loading.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-5813) YarnChild does not load job.xml with mapreduce.job.classloader=true
[
https://issues.apache.org/jira/browse/MAPREDUCE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954671#comment-13954671
]
Hudson commented on MAPREDUCE-5813:
---
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1742 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1742/])
MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the
local-filesystem, rather than rely on the classpath. Contributed by Gera
Shegalov. (acmurthy:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1583050)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
YarnChild does not load job.xml with mapreduce.job.classloader=true
Key: MAPREDUCE-5813
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5813
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, task
Affects Versions: 2.3.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Priority: Blocker
Fix For: 2.4.0
Attachments: MAPREDUCE-5813.v01.patch, MAPREDUCE-5813.v02.patch
{{YarnChild.main}} uses {{JobConf.addResource(String)}} to load {{job.xml}}
that relies on class loading. When {{mapreduce.job.classloader=true}} the
job-speicific part of the class path is separated from {{CLASSPATH}} into
{{APP_CLASSPATH}}. Therefore {{job.xml}} is inaccessible for the default
class loader. Later {{writeLocalJobFile}} overwrites the correct localized
{{job.xml}} on disk as well.
This problem is easily avoided by using {{JobConf.addResource(Path)}} to
read the localized {{job.xml}} without relying on class loading.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (MAPREDUCE-5816) TestMRAppMaster fails in trunk
Ted Yu created MAPREDUCE-5816:
-
Summary: TestMRAppMaster fails in trunk
Key: MAPREDUCE-5816
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5816
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Ted Yu
As can be seen from
https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1741/console:
{code}
Tests in error:
TestMRAppMaster.testMRAppMasterMidLock:163 » NullPointer
TestMRAppMaster.testMRAppMasterSuccessLock:202 » NullPointer
TestMRAppMaster.testMRAppMasterFailLock:241 » NullPointer
{code}
I got the following locally:
{code}
Tests run: 7, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 2.964 sec
FAILURE! - in org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster
testMRAppMasterMidLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.963 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterMidLock(TestMRAppMaster.java:163)
testMRAppMasterSuccessLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.25 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterSuccessLock(TestMRAppMaster.java:202)
testMRAppMasterFailLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.232 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterFailLock(TestMRAppMaster.java:241)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-5810) TestStreamingTaskLog#testStreamingTaskLogWithHadoopCmd is failing
[ https://issues.apache.org/jira/browse/MAPREDUCE-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954678#comment-13954678 ] Hudson commented on MAPREDUCE-5810: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #1716 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1716/]) MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. Contributed by Akira Ajisaka. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1582896) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-streaming/src/test/java/org/apache/hadoop/streaming/TestStreamingTaskLog.java TestStreamingTaskLog#testStreamingTaskLogWithHadoopCmd is failing - Key: MAPREDUCE-5810 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5810 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/streaming Affects Versions: 3.0.0, 2.4.0 Reporter: Mit Desai Assignee: Akira AJISAKA Fix For: 2.4.0 Attachments: MAPREDUCE-5810.patch, log.txt testStreamingTaskLogWithHadoopCmd(org.apache.hadoop.streaming.TestStreamingTaskLog) Time elapsed: 44.069 sec FAILURE! java.lang.AssertionError: environment set for child is wrong at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.hadoop.streaming.TestStreamingTaskLog.runStreamJobAndValidateEnv(TestStreamingTaskLog.java:157) at org.apache.hadoop.streaming.TestStreamingTaskLog.testStreamingTaskLogWithHadoopCmd(TestStreamingTaskLog.java:107) Results : Failed tests: TestStreamingTaskLog.testStreamingTaskLogWithHadoopCmd:107-runStreamJobAndValidateEnv:157 environment set for child is wrong -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MAPREDUCE-5813) YarnChild does not load job.xml with mapreduce.job.classloader=true
[
https://issues.apache.org/jira/browse/MAPREDUCE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954687#comment-13954687
]
Hudson commented on MAPREDUCE-5813:
---
FAILURE: Integrated in Hadoop-Hdfs-trunk #1716 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1716/])
MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the
local-filesystem, rather than rely on the classpath. Contributed by Gera
Shegalov. (acmurthy:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1583050)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/YarnChild.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
YarnChild does not load job.xml with mapreduce.job.classloader=true
Key: MAPREDUCE-5813
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5813
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, task
Affects Versions: 2.3.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Priority: Blocker
Fix For: 2.4.0
Attachments: MAPREDUCE-5813.v01.patch, MAPREDUCE-5813.v02.patch
{{YarnChild.main}} uses {{JobConf.addResource(String)}} to load {{job.xml}}
that relies on class loading. When {{mapreduce.job.classloader=true}} the
job-speicific part of the class path is separated from {{CLASSPATH}} into
{{APP_CLASSPATH}}. Therefore {{job.xml}} is inaccessible for the default
class loader. Later {{writeLocalJobFile}} overwrites the correct localized
{{job.xml}} on disk as well.
This problem is easily avoided by using {{JobConf.addResource(Path)}} to
read the localized {{job.xml}} without relying on class loading.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-5816) TestMRAppMaster fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13954861#comment-13954861
]
Jian He commented on MAPREDUCE-5816:
dup of MAPREDUCE-5815 ?
TestMRAppMaster fails in trunk
--
Key: MAPREDUCE-5816
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5816
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Ted Yu
As can be seen from
https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1741/console:
{code}
Tests in error:
TestMRAppMaster.testMRAppMasterMidLock:163 » NullPointer
TestMRAppMaster.testMRAppMasterSuccessLock:202 » NullPointer
TestMRAppMaster.testMRAppMasterFailLock:241 » NullPointer
{code}
I got the following locally:
{code}
Tests run: 7, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 2.964 sec
FAILURE! - in org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster
testMRAppMasterMidLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.963 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterMidLock(TestMRAppMaster.java:163)
testMRAppMasterSuccessLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.25 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterSuccessLock(TestMRAppMaster.java:202)
testMRAppMasterFailLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.232 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterFailLock(TestMRAppMaster.java:241)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (MAPREDUCE-5816) TestMRAppMaster fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved MAPREDUCE-5816.
---
Resolution: Duplicate
TestMRAppMaster fails in trunk
--
Key: MAPREDUCE-5816
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5816
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Ted Yu
As can be seen from
https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1741/console:
{code}
Tests in error:
TestMRAppMaster.testMRAppMasterMidLock:163 » NullPointer
TestMRAppMaster.testMRAppMasterSuccessLock:202 » NullPointer
TestMRAppMaster.testMRAppMasterFailLock:241 » NullPointer
{code}
I got the following locally:
{code}
Tests run: 7, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 2.964 sec
FAILURE! - in org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster
testMRAppMasterMidLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.963 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterMidLock(TestMRAppMaster.java:163)
testMRAppMasterSuccessLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.25 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterSuccessLock(TestMRAppMaster.java:202)
testMRAppMasterFailLock(org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster)
Time elapsed: 0.232 sec ERROR!
java.lang.NullPointerException: null
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.escapeDelimiters(FileNameIndexUtils.java:275)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:97)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:743)
at
org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
at
org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
at
org.apache.hadoop.service.CompositeService.stop(CompositeService.java:158)
at
org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1491)
at
org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1099)
at
org.apache.hadoop.mapreduce.v2.app.TestMRAppMaster.testMRAppMasterFailLock(TestMRAppMaster.java:241)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
