[jira] [Commented] (MAPREDUCE-5397) AM crashes because Webapp failed to start on multi node cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801816#comment-13801816
]
Josh Clum commented on MAPREDUCE-5397:
--
[~jianhe] - I'm getting the same error. Have you made any progress debugging?
AM crashes because Webapp failed to start on multi node cluster
---
Key: MAPREDUCE-5397
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5397
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: log.txt
I set up a 12 nodes cluster and tried submitting jobs but get this exception.
But job is able to succeed after AM crashes and retry a few times(2 or 3)
{code}
2013-07-12 18:56:28,438 INFO [main] org.mortbay.log: Extract
jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce
to /tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp
2013-07-12 18:56:28,528 WARN [main] org.mortbay.log: Failed startup of
context
org.mortbay.jetty.webapp.WebAppContext@2726b2{/,jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce}
java.io.FileNotFoundException:
/tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp/webapps/mapreduce/.keep
(No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.init(FileOutputStream.java:194)
at java.io.FileOutputStream.init(FileOutputStream.java:145)
at org.mortbay.resource.JarResource.extract(JarResource.java:215)
at
org.mortbay.jetty.webapp.WebAppContext.resolveWebApp(WebAppContext.java:974)
at
org.mortbay.jetty.webapp.WebAppContext.getWebInf(WebAppContext.java:832)
at
org.mortbay.jetty.webapp.WebInfConfiguration.configureClassLoader(WebInfConfiguration.java:62)
at
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:489)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at
org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at org.mortbay.jetty.Server.doStart(Server.java:224)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:684)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:211)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:134)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1394)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1390)
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801874#comment-13801874 ] Jason Lowe commented on MAPREDUCE-5561: --- Yes, the test is definitely racy. There's no guarantee the job will be in the FAIL_ABORT state while when we look at it asynchronously. A couple of approaches to fixing this: # As [~kkambatl] points out, we can skip the FAIL_ABORT check. The real purpose of this test is to verify we eventually get to the FAILED state without hanging when tasks fail. # A more deterministic, explicit test for FAIL_ABORT would be to use an output committer with a barrier, similar to TestingOutputCommitter but with the barrier in the abortJob method, so we can guarantee the job will pause in the FAIL_ABORT state. Then we can release the committer from the barrier and verify the job proceeds to failed. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Attachments: org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801907#comment-13801907 ] Jonathan Eagles commented on MAPREDUCE-5221: I haven't seen any update on this ticket in some time. Can a comment on the current status be provided we can get this improvement in? Reduce side Combiner is not used when using the new API --- Key: MAPREDUCE-5221 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.4-alpha Reporter: Siddharth Seth Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch, MAPREDUCE-5221.6.patch If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this will silently ignored on the reduce side since the reduce side usage is only aware of the old api combiner. This doesn't fail the job - since the new combiner key does not deprecate the old key. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801931#comment-13801931 ] Ravi Prakash commented on MAPREDUCE-3860: - Hi Andrey! Thanks for your answers. 1. It will wait for 10s (100ms * 100), but you're right, that is tolerable too. 2. Aah! I did not know that. Thanks. 3. I agree that username should not be null after anonymization. So maybe we should remove the line anonymizedJob.getUser() != null 4. I can see it either way and I'm not a stickler about either way. I'm fine with how it is. I would probaly classify all my comments as nits above. So I'm fine with this patch. I'll let a committer make the final judgement call. Thanks Andrey! [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: MAPREDUCE-3860--n2.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802016#comment-13802016 ] Tsuyoshi OZAWA commented on MAPREDUCE-5221: --- I think almost work is done. I'll rebase the latest patch soon and wait for review. Reduce side Combiner is not used when using the new API --- Key: MAPREDUCE-5221 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.4-alpha Reporter: Siddharth Seth Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch, MAPREDUCE-5221.6.patch If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this will silently ignored on the reduce side since the reduce side usage is only aware of the old api combiner. This doesn't fail the job - since the new combiner key does not deprecate the old key. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: (was: MAPREDUCE-4980--n7.patch) Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: MAPREDUCE-4980--n7.patch Rebased the patch. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802190#comment-13802190
]
Hadoop QA commented on MAPREDUCE-4980:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609712/MAPREDUCE-4980--n7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 120
new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-tools/hadoop-gridmix hadoop-tools/hadoop-extras
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4129//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4129//console
This message is automatically generated.
Parallel test execution of hadoop-mapreduce-client-core
---
Key: MAPREDUCE-4980
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
Project: Hadoop Map/Reduce
Issue Type: Test
Components: test
Affects Versions: 3.0.0
Reporter: Tsuyoshi OZAWA
Assignee: Andrey Klochkov
Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch,
MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch,
MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch
The maven surefire plugin supports parallel testing feature. By using it, the
tests can be run more faster.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5397) AM crashes because Webapp failed to start on multi node cluster
[
https://issues.apache.org/jira/browse/MAPREDUCE-5397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802191#comment-13802191
]
Jian He commented on MAPREDUCE-5397:
[~jclum], I made a clean build and didn't reproduce this any more
AM crashes because Webapp failed to start on multi node cluster
---
Key: MAPREDUCE-5397
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5397
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: log.txt
I set up a 12 nodes cluster and tried submitting jobs but get this exception.
But job is able to succeed after AM crashes and retry a few times(2 or 3)
{code}
2013-07-12 18:56:28,438 INFO [main] org.mortbay.log: Extract
jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce
to /tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp
2013-07-12 18:56:28,528 WARN [main] org.mortbay.log: Failed startup of
context
org.mortbay.jetty.webapp.WebAppContext@2726b2{/,jar:file:/grid/0/dev/jhe/hadoop-2.1.0-beta/share/hadoop/yarn/hadoop-yarn-common-2.1.0-beta.jar!/webapps/mapreduce}
java.io.FileNotFoundException:
/tmp/Jetty_0_0_0_0_43554_mapreduceljbmlg/webapp/webapps/mapreduce/.keep
(No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.init(FileOutputStream.java:194)
at java.io.FileOutputStream.init(FileOutputStream.java:145)
at org.mortbay.resource.JarResource.extract(JarResource.java:215)
at
org.mortbay.jetty.webapp.WebAppContext.resolveWebApp(WebAppContext.java:974)
at
org.mortbay.jetty.webapp.WebAppContext.getWebInf(WebAppContext.java:832)
at
org.mortbay.jetty.webapp.WebInfConfiguration.configureClassLoader(WebInfConfiguration.java:62)
at
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:489)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at
org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at org.mortbay.jetty.Server.doStart(Server.java:224)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:684)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:211)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:134)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1019)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1394)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1390)
{code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5561: Attachment: mr-5561-1.patch Trivial fix that drops the assert. Ran the test multiple times and haven't seen it fail - I know that doesn't say much. [~jlowe], in this particular case, given the purpose of the test is to verify we eventually get to the FAILED state, I went ahead with the simpler approach. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5561: Target Version/s: 2.3.0 Status: Patch Available (was: Open) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802280#comment-13802280 ] Akira AJISAKA commented on MAPREDUCE-5561: -- +1, LGTM. I ran the test 3 times and haven't seen it fail. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-5561: - Hadoop Flags: Reviewed org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802289#comment-13802289
]
Hadoop QA commented on MAPREDUCE-5561:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609720/mr-5561-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4130//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4130//console
This message is automatically generated.
org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on
trunk
-
Key: MAPREDUCE-5561
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Cindy Li
Assignee: Karthik Kambatla
Priority: Critical
Attachments: mr-5561-1.patch,
org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt
Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec
FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl
testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl)
Time elapsed: 5.507 sec FAILURE!
java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817)
at
org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802357#comment-13802357 ] Jason Lowe commented on MAPREDUCE-5561: --- +1, lgtm. It's a reasonable approach. The test failure is unrelated and most likely caused by something like YARN-1183. I tested the patch locally and it does not fail. I'll commit this shortly. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-5561: -- Resolution: Fixed Fix Version/s: 2.2.1 0.23.10 2.3.0 3.0.0 Status: Resolved (was: Patch Available) Thanks for the contribution, Karthik, and thanks for the review Akira! I committed this to trunk, branch-2, branch-2.2, and branch-0.23. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Fix For: 3.0.0, 2.3.0, 0.23.10, 2.2.1 Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5561) org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk
[ https://issues.apache.org/jira/browse/MAPREDUCE-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802390#comment-13802390 ] Hudson commented on MAPREDUCE-5561: --- SUCCESS: Integrated in Hadoop-trunk-Commit #4645 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4645/]) MAPREDUCE-5561. org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk. Contributed by Karthik Kambatla (jlowe: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1534832) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestJobImpl.java org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk - Key: MAPREDUCE-5561 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5561 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.2.0 Reporter: Cindy Li Assignee: Karthik Kambatla Priority: Critical Fix For: 3.0.0, 2.3.0, 0.23.10, 2.2.1 Attachments: mr-5561-1.patch, org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl-output.txt Running org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl Tests run: 15, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 9.029 sec FAILURE! - in org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testFailAbortDoesntHang(org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl) Time elapsed: 5.507 sec FAILURE! java.lang.AssertionError: expected:FAIL_ABORT but was:FAILED at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.assertJobState(TestJobImpl.java:817) at org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl.testFailAbortDoesntHang(TestJobImpl.java:418) -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5186: Affects Version/s: 2.2.0 mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail -- Key: MAPREDUCE-5186 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5186 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 2.0.4-alpha, 2.2.0 Reporter: Sangjin Lee Priority: Critical CombineFileInputFormat can easily create splits that can come from many different locations (during the last pass of creating global splits). However, we observe that this often runs afoul of the mapreduce.job.max.split.locations check that's done by JobSplitWriter. The default value for mapreduce.job.max.split.locations is 10, and with any decent size cluster, CombineFileInputFormat creates splits that are well above this limit. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5186: Component/s: (was: mrv1) (was: mrv2) job submission mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail -- Key: MAPREDUCE-5186 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5186 Project: Hadoop Map/Reduce Issue Type: Bug Components: job submission Affects Versions: 2.0.4-alpha, 2.2.0 Reporter: Sangjin Lee Priority: Critical CombineFileInputFormat can easily create splits that can come from many different locations (during the last pass of creating global splits). However, we observe that this often runs afoul of the mapreduce.job.max.split.locations check that's done by JobSplitWriter. The default value for mapreduce.job.max.split.locations is 10, and with any decent size cluster, CombineFileInputFormat creates splits that are well above this limit. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5186) mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-5186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802414#comment-13802414 ] Karthik Kambatla commented on MAPREDUCE-5186: - We recently ran into this as well. While the check is superficial (throws a warning) in Hadoop-1, it throws an Exception essentially failing the job submission in Hadoop-2. MAPREDUCE-4146 seems to have introduced this. [~tomwhite], do you remember the reason for throwing an Exception. If there is no particular reason, it would be nice to revert to old behavior. mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail -- Key: MAPREDUCE-5186 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5186 Project: Hadoop Map/Reduce Issue Type: Bug Components: job submission Affects Versions: 2.0.4-alpha, 2.2.0 Reporter: Sangjin Lee Priority: Critical CombineFileInputFormat can easily create splits that can come from many different locations (during the last pass of creating global splits). However, we observe that this often runs afoul of the mapreduce.job.max.split.locations check that's done by JobSplitWriter. The default value for mapreduce.job.max.split.locations is 10, and with any decent size cluster, CombineFileInputFormat creates splits that are well above this limit. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Assigned] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA reassigned MAPREDUCE-4282: Assignee: Akira AJISAKA (was: Eli Reisman) Convert Forrest docs to APT --- Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Akira AJISAKA Labels: newbie Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, MAPREDUCE-4282-2.patch MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802435#comment-13802435 ] Akira AJISAKA commented on MAPREDUCE-4282: -- I converted MapReduce Tutorial and attaching a patch. The patch is not only to convert forrest to APT but also to * Update MRv1-specific word (ex. jobtracker, tasktracker) to MRv2(YARN). * Use new API and properties. I also attach a html. Convert Forrest docs to APT --- Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Akira AJISAKA Labels: newbie Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, MAPREDUCE-4282-2.patch, MapReduceTutorial.html MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Attachment: MapReduceTutorial.html Convert Forrest docs to APT --- Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Akira AJISAKA Labels: newbie Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, MAPREDUCE-4282-2.patch, MapReduceTutorial.html MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Target Version/s: (was: ) Status: Open (was: Patch Available) Convert Forrest docs to APT --- Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Akira AJISAKA Labels: newbie Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, MAPREDUCE-4282-2.patch, MapReduceTutorial.html MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Attachment: MAPREDUCE-4282-tutorial.patch Convert Forrest docs to APT --- Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Akira AJISAKA Labels: newbie Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, MAPREDUCE-4282-2.patch, MAPREDUCE-4282-tutorial.patch, MapReduceTutorial.html MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4282) Convert Forrest docs to APT
[ https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-4282: - Target Version/s: 2.3.0 Status: Patch Available (was: Open) Convert Forrest docs to APT --- Key: MAPREDUCE-4282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282 Project: Hadoop Map/Reduce Issue Type: Task Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Akira AJISAKA Labels: newbie Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch, MAPREDUCE-4282-2.patch, MAPREDUCE-4282-tutorial.patch, MapReduceTutorial.html MR side of HADOOP-8427. Not all of the old forrest docs in src/documentation/content/xdocs have been converted over to APT yet, let's do that and remove the forrest docs. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-4282) Convert Forrest docs to APT
[
https://issues.apache.org/jira/browse/MAPREDUCE-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802457#comment-13802457
]
Hadoop QA commented on MAPREDUCE-4282:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609756/MAPREDUCE-4282-tutorial.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4131//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4131//console
This message is automatically generated.
Convert Forrest docs to APT
---
Key: MAPREDUCE-4282
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4282
Project: Hadoop Map/Reduce
Issue Type: Task
Affects Versions: 2.0.0-alpha
Reporter: Eli Collins
Assignee: Akira AJISAKA
Labels: newbie
Attachments: bad-forrest2apt-attempt1.tar, MAPREDUCE-4282-1.patch,
MAPREDUCE-4282-2.patch, MAPREDUCE-4282-tutorial.patch, MapReduceTutorial.html
MR side of HADOOP-8427. Not all of the old forrest docs in
src/documentation/content/xdocs have been converted over to APT yet, let's do
that and remove the forrest docs.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802553#comment-13802553
]
Jonathan Eagles commented on MAPREDUCE-4980:
Looks like many failures like this in the unit tests runs as show in the
console log
{quote}
---
T E S T S
---
Error occurred during initialization of VM
java.lang.OutOfMemoryError: unable to create new native thread
{quote}
Also, this
{quote}
[WARNING] The requested profile native could not be activated because it does
not exist.
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on
project hadoop-mapreduce-client-jobclient: ExecutionException; nested exception
is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The
forked VM terminated without saying properly goodbye. VM crash or System.exit
called ?
{quote}
Parallel test execution of hadoop-mapreduce-client-core
---
Key: MAPREDUCE-4980
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
Project: Hadoop Map/Reduce
Issue Type: Test
Components: test
Affects Versions: 3.0.0
Reporter: Tsuyoshi OZAWA
Assignee: Andrey Klochkov
Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch,
MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch,
MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch
The maven surefire plugin supports parallel testing feature. By using it, the
tests can be run more faster.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[ https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-5221: -- Attachment: MAPREDUCE-5221.7.patch Reduce side Combiner is not used when using the new API --- Key: MAPREDUCE-5221 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.0.4-alpha Reporter: Siddharth Seth Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch, MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch, MAPREDUCE-5221.6.patch, MAPREDUCE-5221.7.patch If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this will silently ignored on the reduce side since the reduce side usage is only aware of the old api combiner. This doesn't fail the job - since the new combiner key does not deprecate the old key. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5221) Reduce side Combiner is not used when using the new API
[
https://issues.apache.org/jira/browse/MAPREDUCE-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802564#comment-13802564
]
Hadoop QA commented on MAPREDUCE-5221:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609792/MAPREDUCE-5221.7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4132//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4132//console
This message is automatically generated.
Reduce side Combiner is not used when using the new API
---
Key: MAPREDUCE-5221
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5221
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.4-alpha
Reporter: Siddharth Seth
Assignee: Tsuyoshi OZAWA
Attachments: MAPREDUCE-5221.1.patch, MAPREDUCE-5221.2.patch,
MAPREDUCE-5221.3.patch, MAPREDUCE-5221.4.patch, MAPREDUCE-5221.5.patch,
MAPREDUCE-5221.6.patch, MAPREDUCE-5221.7.patch
If a combiner is specified using o.a.h.mapreduce.Job.setCombinerClass - this
will silently ignored on the reduce side since the reduce side usage is only
aware of the old api combiner.
This doesn't fail the job - since the new combiner key does not deprecate the
old key.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13802573#comment-13802573 ] Jonathan Eagles commented on MAPREDUCE-4980: Sorry. Those seems to with the build itself and not due to this patch. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
