[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: (was: HADOOP-9287--N2.patch) Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Improvement Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-4980.1.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: HADOOP-9287--N2.patch Patch is updated with a few additional fixes. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Improvement Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-4980.1.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: MAPREDUCE-4980.patch Attaching a patch which does fixes similar to the ones introduced in HADOOP-9287 and HDFS-4491 and it depends on these jiras. The patch introduces multi-fork execution for the hadoop-mapreduce-client-jobclient module only. This patch, together with HADOOP-9287 and HDFS-4491 introduces multi-fork execution for 3 Hadoop Core modules which take majority of time to test: hadoop-common, hadoop-hdfs and hadoop-mapreduce-client-jobclient. Overview of changes: 1. A new profile parallel-tests is introduced in hadoop-mapreduce-client-jobclient/pom.xml 2. Tests are refactored to use PathUtils.getTestDir/getTestPath methods to get a directory/path to be used for test data. Earlier, the refactored tests implemented it in a similar but slightly different way. 3. MiniMRClientClusterFactory is replaced with MiniMRClientClusterBuilder which makes it more flexible when configuring clusters, in a way similar to MiniDFSCluster.Builder 4. All usages of deprecated class MiniMRCluster are replaced using MiniMRClientClusterBuilder. This is required to avoid FS contention when running mini mr clusters in parallel. 5. All usages of MiniDFSCluster constructors are replaced with MiniDFSCluster.Builder usage, same purpose as prev item. The changes are tested comparing stableness of tests to trunk running both in serial mode and parallel mode. No additional flakiness is noticed. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Improvement Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
Ivan Mitic created MAPREDUCE-5056: - Summary: TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated MAPREDUCE-5056: -- Attachment: MAPREDUCE-5056.trunk.patch Attaching the patch. An easy one, the test fails to match the process dump header because of a line ending (test expects: \n, and it is: \r\n). TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated MAPREDUCE-5056: -- Status: Patch Available (was: Open) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599045#comment-13599045 ] Ivan Mitic commented on MAPREDUCE-5056: --- Just to add, not sure how valuable it is to run TestProcfsBasedProcessTree on Windows, given that we actually use WindowsBasedProcessTree. Alternative is to skip the test as a whole if ProcfsBasedProcessTree is not available. TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5057) Datajoin Package for reduce side join (in contrib folder)
Vikas Jadhav created MAPREDUCE-5057: --- Summary: Datajoin Package for reduce side join (in contrib folder) Key: MAPREDUCE-5057 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5057 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/data-join Affects Versions: 1.0.3 Reporter: Vikas Jadhav Priority: Trivial DataJoin Package contributed to Hadoop has bug 1) MRJobConfig config is not present and will not return input file name (MRJobConfig.MAP_INPUT_FILE) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated MAPREDUCE-5056: -- Attachment: MAPREDUCE-5056.trunk.2.patch Attaching a slightly better patch. TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.2.patch, MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599054#comment-13599054 ] Bikas Saha commented on MAPREDUCE-5056: --- I am in favor of disabling the test with if(Shell.Linux). TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.2.patch, MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5057) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3
[ https://issues.apache.org/jira/browse/MAPREDUCE-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Jadhav updated MAPREDUCE-5057: Assignee: Vikas Jadhav Summary: Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3 (was: Datajoin Package for reduce side join (in contrib folder)) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3 Key: MAPREDUCE-5057 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5057 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/data-join Affects Versions: 1.0.3 Reporter: Vikas Jadhav Assignee: Vikas Jadhav Priority: Trivial DataJoin Package contributed to Hadoop has bug 1) MRJobConfig config is not present and will not return input file name (MRJobConfig.MAP_INPUT_FILE) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (MAPREDUCE-5057) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3
[ https://issues.apache.org/jira/browse/MAPREDUCE-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-5057 started by Vikas Jadhav. Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3 Key: MAPREDUCE-5057 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5057 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/data-join Affects Versions: 1.0.3 Reporter: Vikas Jadhav Assignee: Vikas Jadhav Priority: Trivial DataJoin Package contributed to Hadoop has bug 1) MRJobConfig config is not present and will not return input file name (MRJobConfig.MAP_INPUT_FILE) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5057) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3
[ https://issues.apache.org/jira/browse/MAPREDUCE-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Jadhav updated MAPREDUCE-5057: Status: Patch Available (was: In Progress) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3 Key: MAPREDUCE-5057 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5057 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/data-join Affects Versions: 1.0.3 Reporter: Vikas Jadhav Assignee: Vikas Jadhav Priority: Trivial DataJoin Package contributed to Hadoop has bug 1) MRJobConfig config is not present and will not return input file name (MRJobConfig.MAP_INPUT_FILE) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5057) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3
[ https://issues.apache.org/jira/browse/MAPREDUCE-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Jadhav updated MAPREDUCE-5057: Attachment: DataJoinMapperBase.java Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3 Key: MAPREDUCE-5057 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5057 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/data-join Affects Versions: 1.0.3 Reporter: Vikas Jadhav Assignee: Vikas Jadhav Priority: Trivial Attachments: DataJoinMapperBase.java DataJoin Package contributed to Hadoop has bug 1) MRJobConfig config is not present and will not return input file name (MRJobConfig.MAP_INPUT_FILE) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5057) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3
[ https://issues.apache.org/jira/browse/MAPREDUCE-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikas Jadhav updated MAPREDUCE-5057: Resolution: Fixed Status: Resolved (was: Patch Available) Datajoin Package for reduce side join (in contrib folder) MRJobCOnfig class not present hadoop 1.0.3 Key: MAPREDUCE-5057 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5057 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/data-join Affects Versions: 1.0.3 Reporter: Vikas Jadhav Assignee: Vikas Jadhav Priority: Trivial Attachments: DataJoinMapperBase.java DataJoin Package contributed to Hadoop has bug 1) MRJobConfig config is not present and will not return input file name (MRJobConfig.MAP_INPUT_FILE) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599090#comment-13599090 ] Ivan Mitic commented on MAPREDUCE-5056: --- bq. I am in favor of disabling the test with if(Shell.Linux). Thanks Bikas for the quick response, I agree, will attach the new patch shortly. TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.2.patch, MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[ https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated MAPREDUCE-5056: -- Attachment: MAPREDUCE-5056.trunk.3.patch Attaching the updated patch. A few notes: - I kept the previous fix for newline to achieve symmetry with the implementation from ProcfsBasedProcessTree#getProcessTreeDump - I changed ProcfsBasedProcessTree#isAvailable to use the existing API to check if running on Linux (not to duplicate the code) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header - Key: MAPREDUCE-5056 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0 Reporter: Ivan Mitic Assignee: Ivan Mitic Attachments: MAPREDUCE-5056.trunk.2.patch, MAPREDUCE-5056.trunk.3.patch, MAPREDUCE-5056.trunk.patch Test fails on the below assertion: Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec FAILURE! testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree) Time elapsed: 0 sec FAILURE! junit.framework.AssertionFailedError: Process-tree dump doesn't start with a proper header at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[
https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599112#comment-13599112
]
Hadoop QA commented on MAPREDUCE-5056:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573129/MAPREDUCE-5056.trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3396//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3396//console
This message is automatically generated.
TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't
start with a proper header
-
Key: MAPREDUCE-5056
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
Attachments: MAPREDUCE-5056.trunk.2.patch,
MAPREDUCE-5056.trunk.3.patch, MAPREDUCE-5056.trunk.patch
Test fails on the below assertion:
Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec
FAILURE!
testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree)
Time elapsed: 0 sec FAILURE!
junit.framework.AssertionFailedError: Process-tree dump doesn't start with a
proper header
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at
org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
--
This
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599133#comment-13599133 ] Sandy Ryza commented on MAPREDUCE-5047: --- Attempted to make sure that IsolationRunner still works, and it appears to be broken for other reasons, on both secure and non-secure clusters. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599134#comment-13599134 ] Alejandro Abdelnur commented on MAPREDUCE-5047: --- +1. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[
https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599137#comment-13599137
]
Hadoop QA commented on MAPREDUCE-5056:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573131/MAPREDUCE-5056.trunk.2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3397//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3397//console
This message is automatically generated.
TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't
start with a proper header
-
Key: MAPREDUCE-5056
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
Attachments: MAPREDUCE-5056.trunk.2.patch,
MAPREDUCE-5056.trunk.3.patch, MAPREDUCE-5056.trunk.patch
Test fails on the below assertion:
Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec
FAILURE!
testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree)
Time elapsed: 0 sec FAILURE!
junit.framework.AssertionFailedError: Process-tree dump doesn't start with a
proper header
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at
org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
--
This
[jira] [Updated] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated MAPREDUCE-5047: -- Resolution: Fixed Fix Version/s: 1.3.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 1.3.0 Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5056) TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't start with a proper header
[
https://issues.apache.org/jira/browse/MAPREDUCE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599166#comment-13599166
]
Hadoop QA commented on MAPREDUCE-5056:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573139/MAPREDUCE-5056.trunk.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3398//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3398//console
This message is automatically generated.
TestProcfsBasedProcessTree fails on Windows with Process-tree dump doesn't
start with a proper header
-
Key: MAPREDUCE-5056
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5056
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
Attachments: MAPREDUCE-5056.trunk.2.patch,
MAPREDUCE-5056.trunk.3.patch, MAPREDUCE-5056.trunk.patch
Test fails on the below assertion:
Running org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.266 sec
FAILURE!
testProcessTreeDump(org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree)
Time elapsed: 0 sec FAILURE!
junit.framework.AssertionFailedError: Process-tree dump doesn't start with a
proper header
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.assertTrue(Assert.java:20)
at
org.apache.hadoop.mapreduce.util.TestProcfsBasedProcessTree.testProcessTreeDump(TestProcfsBasedProcessTree.java:564)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Target Version/s: 3.0.0 Status: Patch Available (was: Open) Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Improvement Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599265#comment-13599265
]
Hadoop QA commented on MAPREDUCE-4980:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573117/MAPREDUCE-4980.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 119
new or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3399//console
This message is automatically generated.
Parallel test execution of hadoop-mapreduce-client-core
---
Key: MAPREDUCE-4980
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: test
Affects Versions: 3.0.0
Reporter: Tsuyoshi OZAWA
Assignee: Tsuyoshi OZAWA
Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980.patch
The maven surefire plugin supports parallel testing feature. By using it, the
tests can be run more faster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599286#comment-13599286 ] Andrey Klochkov commented on MAPREDUCE-4980: Build fails due to dependency on HDFS-4491 which is not in trunk yet. The patch affects large number of tests, setting timeout for all of them shouldn't be done as part of this patch. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Improvement Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599295#comment-13599295 ] Sandy Ryza commented on MAPREDUCE-5047: --- To elaborate on this a little further, IsolationRunner throws a NPE whenever it starts due to not reading in mapred.local.dir from the job conf before trying to use it. IsolationRunner has been broken for a long time and was removed in MR2 (MAPREDUCE-2606). While this patch doesn't fix IsolationRunner, keep.failed.task.files still has utility without it, and this fixes the bug that was causing jobs to fail every time it's turned on. keep.failed.task.files=true causes job failure on secure clusters - Key: MAPREDUCE-5047 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047 Project: Hadoop Map/Reduce Issue Type: Bug Components: task, tasktracker Affects Versions: 1.1.1 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 1.3.0 Attachments: MAPREDUCE-5047.patch To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-434) local map-reduce job limited to single reducer
[ https://issues.apache.org/jira/browse/MAPREDUCE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-434: - Attachment: MAPREDUCE-434.patch local map-reduce job limited to single reducer -- Key: MAPREDUCE-434 URL: https://issues.apache.org/jira/browse/MAPREDUCE-434 Project: Hadoop Map/Reduce Issue Type: Bug Environment: local job tracker Reporter: Yoram Arnon Assignee: Aaron Kimball Priority: Minor Attachments: MAPREDUCE-434.2.patch, MAPREDUCE-434.3.patch, MAPREDUCE-434.4.patch, MAPREDUCE-434.5.patch, MAPREDUCE-434.6.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch when mapred.job.tracker is set to 'local', my setNumReduceTasks call is ignored, and the number of reduce tasks is set at 1. This prevents me from locally debugging my partition function, which tries to partition based on the number of reduce tasks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-434) local map-reduce job limited to single reducer
[
https://issues.apache.org/jira/browse/MAPREDUCE-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599498#comment-13599498
]
Hadoop QA commented on MAPREDUCE-434:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573214/MAPREDUCE-434.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3400//console
This message is automatically generated.
local map-reduce job limited to single reducer
--
Key: MAPREDUCE-434
URL: https://issues.apache.org/jira/browse/MAPREDUCE-434
Project: Hadoop Map/Reduce
Issue Type: Bug
Environment: local job tracker
Reporter: Yoram Arnon
Assignee: Aaron Kimball
Priority: Minor
Attachments: MAPREDUCE-434.2.patch, MAPREDUCE-434.3.patch,
MAPREDUCE-434.4.patch, MAPREDUCE-434.5.patch, MAPREDUCE-434.6.patch,
MAPREDUCE-434.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch,
MAPREDUCE-434.patch, MAPREDUCE-434.patch, MAPREDUCE-434.patch
when mapred.job.tracker is set to 'local', my setNumReduceTasks call is
ignored, and the number of reduce tasks is set at 1.
This prevents me from locally debugging my partition function, which tries to
partition based on the number of reduce tasks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5005) TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the process to die
[
https://issues.apache.org/jira/browse/MAPREDUCE-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-5005:
Attachment: mr-5005.patch
Here is a trivial patch that seems to be fixing the test:
# Added a simple loop to check if process is alive.
# Ran the test in a loop and noticed no failures.
TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the
process to die
---
Key: MAPREDUCE-5005
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5005
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: mr-5005.patch
TestProcfsProcessTree#testProcessTree fails occasionally with the following
stack trace
{noformat}
Stack Trace:
junit.framework.AssertionFailedError: expected:false but was:true
at
org.apache.hadoop.util.TestProcfsBasedProcessTree.testProcessTree(TestProcfsBasedProcessTree.java)
{noformat}
kill -9 is executed asynchronously, the signal is delivered when the process
comes out of the kernel (sys call). Checking if the process died immediately
after can fail at times.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5005) TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the process to die
[
https://issues.apache.org/jira/browse/MAPREDUCE-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-5005:
Status: Patch Available (was: Open)
TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the
process to die
---
Key: MAPREDUCE-5005
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5005
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: mr-5005.patch
TestProcfsProcessTree#testProcessTree fails occasionally with the following
stack trace
{noformat}
Stack Trace:
junit.framework.AssertionFailedError: expected:false but was:true
at
org.apache.hadoop.util.TestProcfsBasedProcessTree.testProcessTree(TestProcfsBasedProcessTree.java)
{noformat}
kill -9 is executed asynchronously, the signal is delivered when the process
comes out of the kernel (sys call). Checking if the process died immediately
after can fail at times.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4990) Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput()
[
https://issues.apache.org/jira/browse/MAPREDUCE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-4990:
Attachment: mr-4990.patch
Trivial patch that checks if the log is debug enabled before logging.
Construct debug strings conditionally in
ShuffleHandler.Shuffle#sendMapOutput()
---
Key: MAPREDUCE-4990
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4990
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.0.3-alpha, 0.23.5
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Trivial
Attachments: mr-4990.patch
In {{#sendMapOutput()}}, the debug statements are not wrapped in the
customary {{#isDebugEnabled()}} condition. Given this piece of code is
critical for performance, it would be nice to fix it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4990) Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput()
[
https://issues.apache.org/jira/browse/MAPREDUCE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-4990:
Status: Patch Available (was: Open)
Construct debug strings conditionally in
ShuffleHandler.Shuffle#sendMapOutput()
---
Key: MAPREDUCE-4990
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4990
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 0.23.5, 2.0.3-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Trivial
Attachments: mr-4990.patch
In {{#sendMapOutput()}}, the debug statements are not wrapped in the
customary {{#isDebugEnabled()}} condition. Given this piece of code is
critical for performance, it would be nice to fix it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4990) Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput()
[
https://issues.apache.org/jira/browse/MAPREDUCE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599588#comment-13599588
]
Hadoop QA commented on MAPREDUCE-4990:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573231/mr-4990.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3402//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3402//console
This message is automatically generated.
Construct debug strings conditionally in
ShuffleHandler.Shuffle#sendMapOutput()
---
Key: MAPREDUCE-4990
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4990
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.0.3-alpha, 0.23.5
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Trivial
Attachments: mr-4990.patch
In {{#sendMapOutput()}}, the debug statements are not wrapped in the
customary {{#isDebugEnabled()}} condition. Given this piece of code is
critical for performance, it would be nice to fix it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5005) TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the process to die
[
https://issues.apache.org/jira/browse/MAPREDUCE-5005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599625#comment-13599625
]
Hadoop QA commented on MAPREDUCE-5005:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573227/mr-5005.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3401//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3401//console
This message is automatically generated.
TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the
process to die
---
Key: MAPREDUCE-5005
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5005
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.2-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Attachments: mr-5005.patch
TestProcfsProcessTree#testProcessTree fails occasionally with the following
stack trace
{noformat}
Stack Trace:
junit.framework.AssertionFailedError: expected:false but was:true
at
org.apache.hadoop.util.TestProcfsBasedProcessTree.testProcessTree(TestProcfsBasedProcessTree.java)
{noformat}
kill -9 is executed asynchronously, the signal is delivered when the process
comes out of the kernel (sys call). Checking if the process died immediately
after can fail at times.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4990) Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput()
[
https://issues.apache.org/jira/browse/MAPREDUCE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599632#comment-13599632
]
Karthik Kambatla commented on MAPREDUCE-4990:
-
Didn't include a test because the changes are pretty trivial.
Construct debug strings conditionally in
ShuffleHandler.Shuffle#sendMapOutput()
---
Key: MAPREDUCE-4990
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4990
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.0.3-alpha, 0.23.5
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Trivial
Attachments: mr-4990.patch
In {{#sendMapOutput()}}, the debug statements are not wrapped in the
customary {{#isDebugEnabled()}} condition. Given this piece of code is
critical for performance, it would be nice to fix it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4990) Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput()
[
https://issues.apache.org/jira/browse/MAPREDUCE-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599634#comment-13599634
]
Karthik Kambatla commented on MAPREDUCE-4990:
-
I mean trivial and concern only log messages.
Construct debug strings conditionally in
ShuffleHandler.Shuffle#sendMapOutput()
---
Key: MAPREDUCE-4990
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4990
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.0.3-alpha, 0.23.5
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Trivial
Attachments: mr-4990.patch
In {{#sendMapOutput()}}, the debug statements are not wrapped in the
customary {{#isDebugEnabled()}} condition. Given this piece of code is
critical for performance, it would be nice to fix it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
