[jira] [Updated] (MAPREDUCE-6234) TestHighRamJob fails due to the change in MAPREDUCE-5785
[
https://issues.apache.org/jira/browse/MAPREDUCE-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-6234:
Resolution: Fixed
Fix Version/s: 3.0.0
Hadoop Flags: Incompatible change,Reviewed (was: Incompatible change)
Status: Resolved (was: Patch Available)
Just committed this to trunk. Thanks Masatake for fixing this, Gera and
Tsuyoshi for your reviews.
TestHighRamJob fails due to the change in MAPREDUCE-5785
Key: MAPREDUCE-6234
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6234
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/gridmix, mrv2
Affects Versions: 3.0.0
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Fix For: 3.0.0
Attachments: MAPREDUCE-6234.001.patch, MAPREDUCE-6234.002.patch,
MAPREDUCE-6234.003.patch
TestHighRamJob fails by this.
{code}
---
T E S T S
---
Running org.apache.hadoop.mapred.gridmix.TestHighRamJob
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.162 sec
FAILURE! - in org.apache.hadoop.mapred.gridmix.TestHighRamJob
testHighRamFeatureEmulation(org.apache.hadoop.mapred.gridmix.TestHighRamJob)
Time elapsed: 1.102 sec FAILURE!
java.lang.AssertionError: expected:1024 but was:-1
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.mapred.gridmix.TestHighRamJob.testHighRamConfig(TestHighRamJob.java:98)
at
org.apache.hadoop.mapred.gridmix.TestHighRamJob.testHighRamFeatureEmulation(TestHighRamJob.java:117)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6261) NullPointerException if MapOutputBuffer.flush invoked twice
[ https://issues.apache.org/jira/browse/MAPREDUCE-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325075#comment-14325075 ] Jason Lowe commented on MAPREDUCE-6261: --- Thanks for the patch, Tsuyoshi! Minor nit: Skipping to flush. should be Skipping flush. Other than that patch looks good. NullPointerException if MapOutputBuffer.flush invoked twice --- Key: MAPREDUCE-6261 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6261 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.5.0 Reporter: Jason Lowe Attachments: MAPREDUCE-6261.1.patch MapOutputBuffer.flush will throw an NPE if it is invoked twice, since it blindly assumes kvbuffer is not null yet sets kvbuffer to null towards the end of the method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6234) TestHighRamJob fails due to the change in MAPREDUCE-5785
[
https://issues.apache.org/jira/browse/MAPREDUCE-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325040#comment-14325040
]
Hudson commented on MAPREDUCE-6234:
---
FAILURE: Integrated in Hadoop-trunk-Commit #7136 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/7136/])
MAPREDUCE-6234. TestHighRamJob fails due to the change in MAPREDUCE-5785.
(Masatake Iwasaki via kasha) (kasha: rev
409113d8f97fcfdb96cb028dbb6a20c9a1df81b0)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/TestHighRamJob.java
TestHighRamJob fails due to the change in MAPREDUCE-5785
Key: MAPREDUCE-6234
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6234
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/gridmix, mrv2
Affects Versions: 3.0.0
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Fix For: 3.0.0
Attachments: MAPREDUCE-6234.001.patch, MAPREDUCE-6234.002.patch,
MAPREDUCE-6234.003.patch
TestHighRamJob fails by this.
{code}
---
T E S T S
---
Running org.apache.hadoop.mapred.gridmix.TestHighRamJob
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.162 sec
FAILURE! - in org.apache.hadoop.mapred.gridmix.TestHighRamJob
testHighRamFeatureEmulation(org.apache.hadoop.mapred.gridmix.TestHighRamJob)
Time elapsed: 1.102 sec FAILURE!
java.lang.AssertionError: expected:1024 but was:-1
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.mapred.gridmix.TestHighRamJob.testHighRamConfig(TestHighRamJob.java:98)
at
org.apache.hadoop.mapred.gridmix.TestHighRamJob.testHighRamFeatureEmulation(TestHighRamJob.java:117)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5785) Derive heap size or mapreduce.*.memory.mb automatically
[ https://issues.apache.org/jira/browse/MAPREDUCE-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325039#comment-14325039 ] Hudson commented on MAPREDUCE-5785: --- FAILURE: Integrated in Hadoop-trunk-Commit #7136 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7136/]) MAPREDUCE-6234. TestHighRamJob fails due to the change in MAPREDUCE-5785. (Masatake Iwasaki via kasha) (kasha: rev 409113d8f97fcfdb96cb028dbb6a20c9a1df81b0) * hadoop-mapreduce-project/CHANGES.txt * hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/TestHighRamJob.java Derive heap size or mapreduce.*.memory.mb automatically --- Key: MAPREDUCE-5785 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5785 Project: Hadoop Map/Reduce Issue Type: New Feature Components: mr-am, task Reporter: Gera Shegalov Assignee: Gera Shegalov Fix For: 3.0.0 Attachments: MAPREDUCE-5785.v01.patch, MAPREDUCE-5785.v02.patch, MAPREDUCE-5785.v03.patch, mr-5785-4.patch, mr-5785-5.patch, mr-5785-6.patch, mr-5785-7.patch, mr-5785-8.patch, mr-5785-9.patch Currently users have to set 2 memory-related configs per Job / per task type. One first chooses some container size map reduce.\*.memory.mb and then a corresponding maximum Java heap size Xmx map reduce.\*.memory.mb. This makes sure that the JVM's C-heap (native memory + Java heap) does not exceed this mapreduce.*.memory.mb. If one forgets to tune Xmx, MR-AM might be - allocating big containers whereas the JVM will only use the default -Xmx200m. - allocating small containers that will OOM because Xmx is too high. With this JIRA, we propose to set Xmx automatically based on an empirical ratio that can be adjusted. Xmx is not changed automatically if provided by the user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6261) NullPointerException if MapOutputBuffer.flush invoked twice
[ https://issues.apache.org/jira/browse/MAPREDUCE-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-6261: -- Attachment: MAPREDUCE-6261.2.patch [~jlowe], thank you for the review. Updating a patch to address your comments. NullPointerException if MapOutputBuffer.flush invoked twice --- Key: MAPREDUCE-6261 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6261 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.5.0 Reporter: Jason Lowe Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-6261.1.patch, MAPREDUCE-6261.2.patch MapOutputBuffer.flush will throw an NPE if it is invoked twice, since it blindly assumes kvbuffer is not null yet sets kvbuffer to null towards the end of the method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[
https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325480#comment-14325480
]
Tsuyoshi OZAWA commented on MAPREDUCE-4286:
---
+1. Checking this in.
{code}
I have removed the case because there is no JTClientProvider to verify the +ve
test scenario.
{code}
About classic mode, I think we can remove it from document. Let's discuss the
topic on another JIRA.
TestClientProtocolProviderImpls passes on failure conditions also
-
Key: MAPREDUCE-4286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch,
MAPREDUCE-4286.patch, MAPREDUCE-4286.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4286: -- Resolution: Fixed Fix Version/s: 2.7.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks [~devaraj.k] for your contribution. TestClientProtocolProviderImpls passes on failure conditions Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.6.0 Reporter: Devaraj K Assignee: Devaraj K Fix For: 2.7.0 Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MAPREDUCE-6261) NullPointerException if MapOutputBuffer.flush invoked twice
[ https://issues.apache.org/jira/browse/MAPREDUCE-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA reassigned MAPREDUCE-6261: - Assignee: Tsuyoshi OZAWA NullPointerException if MapOutputBuffer.flush invoked twice --- Key: MAPREDUCE-6261 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6261 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.5.0 Reporter: Jason Lowe Assignee: Tsuyoshi OZAWA Attachments: MAPREDUCE-6261.1.patch MapOutputBuffer.flush will throw an NPE if it is invoked twice, since it blindly assumes kvbuffer is not null yet sets kvbuffer to null towards the end of the method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6261) NullPointerException if MapOutputBuffer.flush invoked twice
[
https://issues.apache.org/jira/browse/MAPREDUCE-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325445#comment-14325445
]
Hadoop QA commented on MAPREDUCE-6261:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699418/MAPREDUCE-6261.2.patch
against trunk revision 57db50c.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5205//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5205//console
This message is automatically generated.
NullPointerException if MapOutputBuffer.flush invoked twice
---
Key: MAPREDUCE-6261
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6261
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.5.0
Reporter: Jason Lowe
Assignee: Tsuyoshi OZAWA
Attachments: MAPREDUCE-6261.1.patch, MAPREDUCE-6261.2.patch
MapOutputBuffer.flush will throw an NPE if it is invoked twice, since it
blindly assumes kvbuffer is not null yet sets kvbuffer to null towards the
end of the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325508#comment-14325508 ] Hudson commented on MAPREDUCE-4286: --- SUCCESS: Integrated in Hadoop-trunk-Commit #7140 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7140/]) MAPREDUCE-4286. TestClientProtocolProviderImpls passes on failure conditions. Contributed by Devaraj K. (ozawa: rev 7c782047c609b29178945bd566a0d162e64dbfdb) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/TestClientProtocolProviderImpls.java * hadoop-mapreduce-project/CHANGES.txt TestClientProtocolProviderImpls passes on failure conditions Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.6.0 Reporter: Devaraj K Assignee: Devaraj K Fix For: 2.7.0 Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4286: -- Summary: TestClientProtocolProviderImpls passes on failure conditions (was: TestClientProtocolProviderImpls passes on failure conditions also) TestClientProtocolProviderImpls passes on failure conditions Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.6.0 Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi OZAWA updated MAPREDUCE-4286: -- Affects Version/s: 2.6.0 TestClientProtocolProviderImpls passes on failure conditions also - Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.6.0 Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6260) Convert site documentation to markdown
[ https://issues.apache.org/jira/browse/MAPREDUCE-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324470#comment-14324470 ] Hudson commented on MAPREDUCE-6260: --- FAILURE: Integrated in Hadoop-trunk-Commit #7130 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7130/]) MAPREDUCE-6260. Convert site documentation to markdown (Masatake Iwasaki via aw) (aw: rev 8b787e2fdbd0050c0345cf14b26af9d61049068f) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/DistributedCacheDeploy.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredAppMasterRest.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/DistributedCacheDeploy.md.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredAppMasterRest.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapReduce_Compatibility_Hadoop1_Hadoop2.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/EncryptedShuffle.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/markdown/HistoryServerRest.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/EncryptedShuffle.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/PluggableShuffleAndPluggableSort.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapredCommands.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduce_Compatibility_Hadoop1_Hadoop2.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/markdown/MapReduceTutorial.md * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/site/apt/HistoryServerRest.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapReduceTutorial.apt.vm * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/PluggableShuffleAndPluggableSort.apt.vm * hadoop-mapreduce-project/CHANGES.txt Convert site documentation to markdown -- Key: MAPREDUCE-6260 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6260 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation Affects Versions: 3.0.0 Reporter: Allen Wittenauer Assignee: Masatake Iwasaki Fix For: 3.0.0 Attachments: MAPREDUCE-6260-00.patch, MAPREDUCE-6260-01.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6096) SummarizedJob class NPEs with some jhist files
[
https://issues.apache.org/jira/browse/MAPREDUCE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324271#comment-14324271
]
Hadoop QA commented on MAPREDUCE-6096:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699263/MAPREDUCE-6096-v4.patch
against trunk revision 2f0f756.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5204//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5204//artifact/patchprocess/patchReleaseAuditProblems.txt
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5204//console
This message is automatically generated.
SummarizedJob class NPEs with some jhist files
--
Key: MAPREDUCE-6096
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6096
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver
Affects Versions: trunk
Reporter: zhangyubiao
Labels: easyfix, patch
Attachments: MAPREDUCE-6096-v2.patch, MAPREDUCE-6096-v3.patch,
MAPREDUCE-6096-v4.patch, MAPREDUCE-6096.patch,
job_1410427642147_0124-1411726671220-hadp-word+count-1411726696863-1-1-SUCCEEDED-default.jhist
When I Parse the JobHistory in the HistoryFile,I use the Hadoop System's
map-reduce-client-core project
org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser class and
HistoryViewer$SummarizedJob to Parse the JobHistoryFile(Just Like
job_1408862281971_489761-1410883171851_XXX.jhist)
and it throw an Exception Just Like
Exception in thread pool-1-thread-1 java.lang.NullPointerException
at
org.apache.hadoop.mapreduce.jobhistory.HistoryViewer$SummarizedJob.init(HistoryViewer.java:626)
at
com.jd.hadoop.log.parse.ParseLogService.getJobDetail(ParseLogService.java:70)
After I'm see the SummarizedJob class I find that attempt.getTaskStatus() is
NULL ， So I change the order of
attempt.getTaskStatus().equals (TaskStatus.State.FAILED.toString()) to
TaskStatus.State.FAILED.toString().equals(attempt.getTaskStatus())
and it works well .
So I wonder If we can change all attempt.getTaskStatus() after
TaskStatus.State.XXX.toString() ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6225) Fix new findbug warnings in hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324314#comment-14324314 ] Hudson commented on MAPREDUCE-6225: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #108 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/108/]) MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena (junping_du: rev 814afa46efef201cb782072432fc744e1cb9c463) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IndexCache.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLogAppender.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/fieldsel/FieldSelectionHelper.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java Fix new findbug warnings in hadoop-mapreduce-client-core Key: MAPREDUCE-6225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6225 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.7.0 Reporter: Jason Lowe Assignee: Varun Saxena Fix For: 2.7.0 Attachments: MAPREDUCE-6225.001.patch, MAPREDUCE-6225.002.patch, MAPREDUCE-6225.003.patch Recent precommit builds in hadoop-mapreduce-client-core are flagging findbug warnings that appear to be new with the recent findbugs upgrade. These need to be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6260) Convert site documentation to markdown
[ https://issues.apache.org/jira/browse/MAPREDUCE-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated MAPREDUCE-6260: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) +1 committed to trunk. Thanks! Convert site documentation to markdown -- Key: MAPREDUCE-6260 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6260 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation Affects Versions: 3.0.0 Reporter: Allen Wittenauer Assignee: Masatake Iwasaki Fix For: 3.0.0 Attachments: MAPREDUCE-6260-00.patch, MAPREDUCE-6260-01.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6096) SummarizedJob class NPEs with some jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyubiao updated MAPREDUCE-6096: --- Attachment: MAPREDUCE-6096-v4.patch SummarizedJob class NPEs with some jhist files -- Key: MAPREDUCE-6096 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6096 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: trunk Reporter: zhangyubiao Labels: easyfix, patch Attachments: MAPREDUCE-6096-v2.patch, MAPREDUCE-6096-v3.patch, MAPREDUCE-6096-v4.patch, MAPREDUCE-6096.patch, job_1410427642147_0124-1411726671220-hadp-word+count-1411726696863-1-1-SUCCEEDED-default.jhist When I Parse the JobHistory in the HistoryFile,I use the Hadoop System's map-reduce-client-core project org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser class and HistoryViewer$SummarizedJob to Parse the JobHistoryFile(Just Like job_1408862281971_489761-1410883171851_XXX.jhist) and it throw an Exception Just Like Exception in thread pool-1-thread-1 java.lang.NullPointerException at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer$SummarizedJob.init(HistoryViewer.java:626) at com.jd.hadoop.log.parse.ParseLogService.getJobDetail(ParseLogService.java:70) After I'm see the SummarizedJob class I find that attempt.getTaskStatus() is NULL ， So I change the order of attempt.getTaskStatus().equals (TaskStatus.State.FAILED.toString()) to TaskStatus.State.FAILED.toString().equals(attempt.getTaskStatus()) and it works well . So I wonder If we can change all attempt.getTaskStatus() after TaskStatus.State.XXX.toString() ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6096) SummarizedJob class NPEs with some jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyubiao updated MAPREDUCE-6096: --- Attachment: MAPREDUCE-6096.v4.patch MAPREDUCE-6096.v4.patch is commit SummarizedJob class NPEs with some jhist files -- Key: MAPREDUCE-6096 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6096 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: trunk Reporter: zhangyubiao Labels: easyfix, patch Attachments: MAPREDUCE-6096-v2.patch, MAPREDUCE-6096-v3.patch, MAPREDUCE-6096.patch, MAPREDUCE-6096.v4.patch, job_1410427642147_0124-1411726671220-hadp-word+count-1411726696863-1-1-SUCCEEDED-default.jhist When I Parse the JobHistory in the HistoryFile,I use the Hadoop System's map-reduce-client-core project org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser class and HistoryViewer$SummarizedJob to Parse the JobHistoryFile(Just Like job_1408862281971_489761-1410883171851_XXX.jhist) and it throw an Exception Just Like Exception in thread pool-1-thread-1 java.lang.NullPointerException at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer$SummarizedJob.init(HistoryViewer.java:626) at com.jd.hadoop.log.parse.ParseLogService.getJobDetail(ParseLogService.java:70) After I'm see the SummarizedJob class I find that attempt.getTaskStatus() is NULL ， So I change the order of attempt.getTaskStatus().equals (TaskStatus.State.FAILED.toString()) to TaskStatus.State.FAILED.toString().equals(attempt.getTaskStatus()) and it works well . So I wonder If we can change all attempt.getTaskStatus() after TaskStatus.State.XXX.toString() ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6225) Fix new findbug warnings in hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324236#comment-14324236 ] Hudson commented on MAPREDUCE-6225: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #2039 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2039/]) MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena (junping_du: rev 814afa46efef201cb782072432fc744e1cb9c463) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/fieldsel/FieldSelectionHelper.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IndexCache.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLogAppender.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java * hadoop-mapreduce-project/CHANGES.txt Fix new findbug warnings in hadoop-mapreduce-client-core Key: MAPREDUCE-6225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6225 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.7.0 Reporter: Jason Lowe Assignee: Varun Saxena Fix For: 2.7.0 Attachments: MAPREDUCE-6225.001.patch, MAPREDUCE-6225.002.patch, MAPREDUCE-6225.003.patch Recent precommit builds in hadoop-mapreduce-client-core are flagging findbug warnings that appear to be new with the recent findbugs upgrade. These need to be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6096) SummarizedJob class NPEs with some jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-6096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangyubiao updated MAPREDUCE-6096: --- Attachment: (was: MAPREDUCE-6096.v4.patch) SummarizedJob class NPEs with some jhist files -- Key: MAPREDUCE-6096 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6096 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: trunk Reporter: zhangyubiao Labels: easyfix, patch Attachments: MAPREDUCE-6096-v2.patch, MAPREDUCE-6096-v3.patch, MAPREDUCE-6096.patch, job_1410427642147_0124-1411726671220-hadp-word+count-1411726696863-1-1-SUCCEEDED-default.jhist When I Parse the JobHistory in the HistoryFile,I use the Hadoop System's map-reduce-client-core project org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser class and HistoryViewer$SummarizedJob to Parse the JobHistoryFile(Just Like job_1408862281971_489761-1410883171851_XXX.jhist) and it throw an Exception Just Like Exception in thread pool-1-thread-1 java.lang.NullPointerException at org.apache.hadoop.mapreduce.jobhistory.HistoryViewer$SummarizedJob.init(HistoryViewer.java:626) at com.jd.hadoop.log.parse.ParseLogService.getJobDetail(ParseLogService.java:70) After I'm see the SummarizedJob class I find that attempt.getTaskStatus() is NULL ， So I change the order of attempt.getTaskStatus().equals (TaskStatus.State.FAILED.toString()) to TaskStatus.State.FAILED.toString().equals(attempt.getTaskStatus()) and it works well . So I wonder If we can change all attempt.getTaskStatus() after TaskStatus.State.XXX.toString() ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6225) Fix new findbug warnings in hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324302#comment-14324302 ] Hudson commented on MAPREDUCE-6225: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2058 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2058/]) MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena (junping_du: rev 814afa46efef201cb782072432fc744e1cb9c463) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLogAppender.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/fieldsel/FieldSelectionHelper.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IndexCache.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java Fix new findbug warnings in hadoop-mapreduce-client-core Key: MAPREDUCE-6225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6225 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.7.0 Reporter: Jason Lowe Assignee: Varun Saxena Fix For: 2.7.0 Attachments: MAPREDUCE-6225.001.patch, MAPREDUCE-6225.002.patch, MAPREDUCE-6225.003.patch Recent precommit builds in hadoop-mapreduce-client-core are flagging findbug warnings that appear to be new with the recent findbugs upgrade. These need to be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6260) Convert site documentation to markdown
[ https://issues.apache.org/jira/browse/MAPREDUCE-6260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324773#comment-14324773 ] Masatake Iwasaki commented on MAPREDUCE-6260: - Thanks [~aw]! Convert site documentation to markdown -- Key: MAPREDUCE-6260 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6260 Project: Hadoop Map/Reduce Issue Type: Improvement Components: documentation Affects Versions: 3.0.0 Reporter: Allen Wittenauer Assignee: Masatake Iwasaki Fix For: 3.0.0 Attachments: MAPREDUCE-6260-00.patch, MAPREDUCE-6260-01.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4815) FileOutputCommitter.commitJob can be very slow for jobs with many output files
[
https://issues.apache.org/jira/browse/MAPREDUCE-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324701#comment-14324701
]
Siqi Li commented on MAPREDUCE-4815:
I think in {code}testRecovery/CommitInternal{code} the output directory
contents are also validated for version 2. For {code}version == 1{code} there
was an extra validation for task output directory
FileOutputCommitter.commitJob can be very slow for jobs with many output files
--
Key: MAPREDUCE-4815
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4815
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.3, 2.0.1-alpha, 2.4.1
Reporter: Jason Lowe
Assignee: Siqi Li
Attachments: MAPREDUCE-4815.v10.patch, MAPREDUCE-4815.v11.patch,
MAPREDUCE-4815.v3.patch, MAPREDUCE-4815.v4.patch, MAPREDUCE-4815.v5.patch,
MAPREDUCE-4815.v6.patch, MAPREDUCE-4815.v7.patch, MAPREDUCE-4815.v8.patch,
MAPREDUCE-4815.v9.patch
If a job generates many files to commit then the commitJob method call at the
end of the job can take minutes. This is a performance regression from 1.x,
as 1.x had the tasks commit directly to the final output directory as they
were completing and commitJob had very little to do. The commit work was
processed in parallel and overlapped the processing of outstanding tasks. In
0.23/2.x, the commit is single-threaded and waits until all tasks have
completed before commencing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4815) FileOutputCommitter.commitJob can be very slow for jobs with many output files
[
https://issues.apache.org/jira/browse/MAPREDUCE-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324699#comment-14324699
]
Siqi Li commented on MAPREDUCE-4815:
I think in {code}testRecovery/CommitInternal{code} the output directory
contents are also validated for version 2. For {code}version == 1{code} there
was an extra validation for task output directory
FileOutputCommitter.commitJob can be very slow for jobs with many output files
--
Key: MAPREDUCE-4815
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4815
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.3, 2.0.1-alpha, 2.4.1
Reporter: Jason Lowe
Assignee: Siqi Li
Attachments: MAPREDUCE-4815.v10.patch, MAPREDUCE-4815.v11.patch,
MAPREDUCE-4815.v3.patch, MAPREDUCE-4815.v4.patch, MAPREDUCE-4815.v5.patch,
MAPREDUCE-4815.v6.patch, MAPREDUCE-4815.v7.patch, MAPREDUCE-4815.v8.patch,
MAPREDUCE-4815.v9.patch
If a job generates many files to commit then the commitJob method call at the
end of the job can take minutes. This is a performance regression from 1.x,
as 1.x had the tasks commit directly to the final output directory as they
were completing and commitJob had very little to do. The commit work was
processed in parallel and overlapped the processing of outstanding tasks. In
0.23/2.x, the commit is single-threaded and waits until all tasks have
completed before commencing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6225) Fix new findbug warnings in hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324100#comment-14324100 ] Hudson commented on MAPREDUCE-6225: --- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #107 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/107/]) MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena (junping_du: rev 814afa46efef201cb782072432fc744e1cb9c463) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLogAppender.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IndexCache.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/fieldsel/FieldSelectionHelper.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java Fix new findbug warnings in hadoop-mapreduce-client-core Key: MAPREDUCE-6225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6225 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.7.0 Reporter: Jason Lowe Assignee: Varun Saxena Fix For: 2.7.0 Attachments: MAPREDUCE-6225.001.patch, MAPREDUCE-6225.002.patch, MAPREDUCE-6225.003.patch Recent precommit builds in hadoop-mapreduce-client-core are flagging findbug warnings that appear to be new with the recent findbugs upgrade. These need to be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324101#comment-14324101 ] Devaraj K commented on MAPREDUCE-4286: -- These test failures are not related to the patch. TestClientProtocolProviderImpls passes on failure conditions also - Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6225) Fix new findbug warnings in hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323821#comment-14323821 ] Hudson commented on MAPREDUCE-6225: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena (junping_du: rev 814afa46efef201cb782072432fc744e1cb9c463) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLogAppender.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IndexCache.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/fieldsel/FieldSelectionHelper.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java Fix new findbug warnings in hadoop-mapreduce-client-core Key: MAPREDUCE-6225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6225 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.7.0 Reporter: Jason Lowe Assignee: Varun Saxena Fix For: 2.7.0 Attachments: MAPREDUCE-6225.001.patch, MAPREDUCE-6225.002.patch, MAPREDUCE-6225.003.patch Recent precommit builds in hadoop-mapreduce-client-core are flagging findbug warnings that appear to be new with the recent findbugs upgrade. These need to be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6221) Stringifier is left unclosed in Chain#getChainElementConf()
[
https://issues.apache.org/jira/browse/MAPREDUCE-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323812#comment-14323812
]
Hudson commented on MAPREDUCE-6221:
---
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/])
MAPREDUCE-6221. Stringifier is left unclosed in Chain#getChainElementConf().
Contributed by Ted Yu. (ozawa: rev 9b0ba59b8284fae132535fbca5ce372d7a6c38c0)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/chain/Chain.java
Stringifier is left unclosed in Chain#getChainElementConf()
---
Key: MAPREDUCE-6221
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6221
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.6.0
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.7.0
Attachments: mapreduce-6221-001.patch
{code}
StringifierConfiguration stringifier =
new DefaultStringifierConfiguration(jobConf, Configuration.class);
String confString = jobConf.get(confKey, null);
if (confString != null) {
conf = stringifier.fromString(jobConf.get(confKey, null));
{code}
stringifier is not closed upon return from the method.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6253) Update use of Iterator to Iterable
[ https://issues.apache.org/jira/browse/MAPREDUCE-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323888#comment-14323888 ] Hudson commented on MAPREDUCE-6253: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-6253. Update use of Iterator to Iterable. Contributed by Ray (devaraj: rev 76e309ead01f02b32335330cd920536f907fb71f) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JobHistoryEventHandler.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/QueueManager.java * hadoop-mapreduce-project/CHANGES.txt Update use of Iterator to Iterable -- Key: MAPREDUCE-6253 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6253 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Ray Chiang Assignee: Ray Chiang Priority: Minor Fix For: 2.7.0 Attachments: MAPREDUCE-6253.001.patch Found these using the IntelliJ Findbugs-IDEA plugin, which uses findbugs3. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6256) Removed unused private methods in o.a.h.mapreduce.Job.java
[
https://issues.apache.org/jira/browse/MAPREDUCE-6256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323884#comment-14323884
]
Hudson commented on MAPREDUCE-6256:
---
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/])
MAPREDUCE-6256. Removed unused private methods in o.a.h.mapreduce.Job.java.
Contributed by Naganarasimha G R. (ozawa: rev
3338f6d90795dafd445891cbd2f4d46b53012382)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Job.java
* hadoop-mapreduce-project/CHANGES.txt
Removed unused private methods in o.a.h.mapreduce.Job.java
--
Key: MAPREDUCE-6256
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6256
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.6.0
Reporter: Devaraj K
Assignee: Naganarasimha G R
Priority: Minor
Labels: newbie
Fix For: 2.7.0
Attachments: MAPREDUCE-6256.20150214-1.patch
These below methods are not used any where in the code and these can be
removed.
{code:xml}
private void setStatus(JobStatus status)
private boolean shouldDownloadProfile()
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4413) MR lib dir contains jdiff (which is gpl)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323860#comment-14323860 ] Hudson commented on MAPREDUCE-4413: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-4413. MR lib dir contains jdiff (which is gpl) (Nemon Lou via aw) (aw: rev aab459c904bf2007c5b230af8c058793935faf89) * hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml * hadoop-mapreduce-project/CHANGES.txt MR lib dir contains jdiff (which is gpl) Key: MAPREDUCE-4413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4413 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 2.0.0-alpha Reporter: Eli Collins Assignee: Nemon Lou Priority: Critical Fix For: 3.0.0 Attachments: MAPREDUCE-4413.patch, MAPREDUCE-4413.patch A tarball built from trunk contains the following: ./share/hadoop/mapreduce/lib/jdiff-1.0.9.jar jdiff is gplv2, we need to exclude it from the build artifact. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6255) Fix JobCounter's format to use grouping separator
[ https://issues.apache.org/jira/browse/MAPREDUCE-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323889#comment-14323889 ] Hudson commented on MAPREDUCE-6255: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-6255. Fix JobCounter's format to use grouping separator. Contributed by Ryu Kobayashi. (ozawa: rev ba3c80a5ca8fefae9d0e67232f9973e1a0458f58) * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/CountersBlock.java Fix JobCounter's format to use grouping separator - Key: MAPREDUCE-6255 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6255 Project: Hadoop Map/Reduce Issue Type: Improvement Components: client Affects Versions: 2.6.0, 2.7.0 Reporter: Ryu Kobayashi Assignee: Ryu Kobayashi Priority: Minor Fix For: 2.7.0 Attachments: YARN-3186.patch MRv2 Job Counter has not been comma format, this is hard to see. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6250) deprecate sbin/mr-jobhistory-daemon.sh
[ https://issues.apache.org/jira/browse/MAPREDUCE-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323887#comment-14323887 ] Hudson commented on MAPREDUCE-6250: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-6250. deprecate sbin/mr-jobhistory-daemon.sh (aw) (aw: rev 6f5290b0309c0d06a7e05f64354ca0c1fb5a4676) * hadoop-mapreduce-project/bin/mr-jobhistory-daemon.sh * hadoop-mapreduce-project/CHANGES.txt deprecate sbin/mr-jobhistory-daemon.sh -- Key: MAPREDUCE-6250 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6250 Project: Hadoop Map/Reduce Issue Type: Improvement Components: scripts Reporter: Allen Wittenauer Assignee: Allen Wittenauer Fix For: 3.0.0 Attachments: MAPREDUCE-6250-00.patch Functionality has been moved to bin/mapred. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4431) mapred command should print the reason on killing already completed jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323863#comment-14323863 ] Hudson commented on MAPREDUCE-4431: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-4431. mapred command should print the reason on killing already completed jobs. Contributed by Devaraj K. (ozawa: rev ac8d52bf50bba1a29489ee75fd90717d8a2b0cc9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java * hadoop-mapreduce-project/CHANGES.txt mapred command should print the reason on killing already completed jobs Key: MAPREDUCE-4431 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4431 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Reporter: Nishan Shetty Assignee: Devaraj K Priority: Minor Fix For: 2.7.0 Attachments: MAPREDUCE-4431-1.patch, MAPREDUCE-4431-2.patch, MAPREDUCE-4431.patch If we try to kill the already completed job by the following command it gives ambiguous message as Killed job job id ./mapred job -kill already completed job id -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-5335) Rename Job Tracker terminology in ShuffleSchedulerImpl
[
https://issues.apache.org/jira/browse/MAPREDUCE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323866#comment-14323866
]
Hudson commented on MAPREDUCE-5335:
---
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/])
MAPREDUCE-5335. Rename Job Tracker terminology in ShuffleSchedulerImpl.
Contributed by Devaraj K. (ozawa: rev b42d09eb62bd1725d70da59f1a6fdac83cea82d1)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
* hadoop-mapreduce-project/CHANGES.txt
Rename Job Tracker terminology in ShuffleSchedulerImpl
--
Key: MAPREDUCE-5335
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5335
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: applicationmaster
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 2.7.0
Attachments: MAPREDUCE-5335-1.patch, MAPREDUCE-5335.patch
{code:xml}
2013-06-17 17:27:30,134 INFO [fetcher#2]
org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Reporting fetch
failure for attempt_1371467533091_0005_m_10_0 to jobtracker.
{code}
{code:title=ShuffleSchedulerImpl.java|borderStyle=solid}
// Notify the JobTracker
// after every read error, if 'reportReadErrorImmediately' is true or
// after every 'maxFetchFailuresBeforeReporting' failures
private void checkAndInformJobTracker(
int failures, TaskAttemptID mapId, boolean readError,
boolean connectExcpt) {
if (connectExcpt || (reportReadErrorImmediately readError)
|| ((failures % maxFetchFailuresBeforeReporting) == 0)) {
LOG.info(Reporting fetch failure for + mapId + to jobtracker.);
status.addFetchFailedMap((org.apache.hadoop.mapred.TaskAttemptID)
mapId);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6237) Multiple mappers with DBInputFormat don't work because of reusing conections
[ https://issues.apache.org/jira/browse/MAPREDUCE-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323807#comment-14323807 ] Hudson commented on MAPREDUCE-6237: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of reusing conections. Contributed by Kannan Rajah. (ozawa: rev 241336ca2b7cf97d7e0bd84dbe0542b72f304dc9) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/db/DataDrivenDBInputFormat.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/db/OracleDataDrivenDBInputFormat.java Multiple mappers with DBInputFormat don't work because of reusing conections Key: MAPREDUCE-6237 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6237 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.5.0, 2.6.0 Reporter: Kannan Rajah Assignee: Kannan Rajah Fix For: 2.6.1 Attachments: mapreduce-6237.patch, mapreduce-6237.patch, mapreduce-6237.patch DBInputFormat.createDBRecorder is reusing JDBC connections across instances of DBRecordReader. This is not a good idea. We should be creating separate connection. If performance is a concern, then we should be using connection pooling instead. I looked at DBOutputFormat.getRecordReader. It actually creates a new Connection object for each DBRecordReader. So can we just change DBInputFormat to create new Connection every time? The connection reuse code was added as part of connection leak bug in MAPREDUCE-1443. Any reason for caching the connection? We observed this issue in a customer setup where they were reading data from MySQL using Pig. As per customer, the query is returning two records which causes Pig to create two instances of DBRecordReader. These two instances are sharing the database connection instance. The first DBRecordReader runs to extract the first record from MySQL just fine, but then closes the shared connection instance. When the second DBRecordReader runs, it tries to execute a query to retrieve the second record on the closed shared connection instance, which fail. If we set mapred.map.tasks to 1, the query will be successful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[
https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323939#comment-14323939
]
Tsuyoshi OZAWA commented on MAPREDUCE-4286:
---
[~devaraj.k], thank you for taking this JIRA. Looks good to me overall.
{code}
-try {
- conf.set(MRConfig.FRAMEWORK_NAME, classic);
- conf.set(JTConfig.JT_IPC_ADDRESS, local);
- new Cluster(conf);
- fail(Cluster with classic Framework name shouldnot use local JT
address);
{code}
Should we keep testClusterWithJTClientProvider like this in addition to
testClusterWithYarnClientProvider for backward compatibility?
{code}
@Test
public void testClusterWithJTClientProvider() throws Exception {
Configuration conf = new Configuration();
try {
conf.set(MRConfig.FRAMEWORK_NAME, classic);
conf.set(JTConfig.JT_IPC_ADDRESS, local);
new Cluster(conf);
fail(Cluster with classic Framework name should not use +
local JT address);
} catch (IOException e){
}
}
{code}
TestClientProtocolProviderImpls passes on failure conditions also
-
Key: MAPREDUCE-4286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286.patch,
MAPREDUCE-4286.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6234) TestHighRamJob fails due to the change in MAPREDUCE-5785
[
https://issues.apache.org/jira/browse/MAPREDUCE-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325029#comment-14325029
]
Karthik Kambatla commented on MAPREDUCE-6234:
-
+1. Checking this in.
TestHighRamJob fails due to the change in MAPREDUCE-5785
Key: MAPREDUCE-6234
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6234
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/gridmix, mrv2
Affects Versions: 3.0.0
Reporter: Masatake Iwasaki
Assignee: Masatake Iwasaki
Attachments: MAPREDUCE-6234.001.patch, MAPREDUCE-6234.002.patch,
MAPREDUCE-6234.003.patch
TestHighRamJob fails by this.
{code}
---
T E S T S
---
Running org.apache.hadoop.mapred.gridmix.TestHighRamJob
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.162 sec
FAILURE! - in org.apache.hadoop.mapred.gridmix.TestHighRamJob
testHighRamFeatureEmulation(org.apache.hadoop.mapred.gridmix.TestHighRamJob)
Time elapsed: 1.102 sec FAILURE!
java.lang.AssertionError: expected:1024 but was:-1
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.mapred.gridmix.TestHighRamJob.testHighRamConfig(TestHighRamJob.java:98)
at
org.apache.hadoop.mapred.gridmix.TestHighRamJob.testHighRamFeatureEmulation(TestHighRamJob.java:117)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4286: - Status: Open (was: Patch Available) TestClientProtocolProviderImpls passes on failure conditions also - Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4286: - Attachment: MAPREDUCE-4286-2.patch TestClientProtocolProviderImpls passes on failure conditions also - Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[ https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4286: - Status: Patch Available (was: Open) Thanks [~] ozawa for your prompt review. I have removed the case because there is no JTClientProvider to verify the +ve test scenario. Anyway I have added back the exception scenario case for JTClientProvider as you suggested in the updated patch. TestClientProtocolProviderImpls passes on failure conditions also - Key: MAPREDUCE-4286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Devaraj K Assignee: Devaraj K Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch, MAPREDUCE-4286.patch, MAPREDUCE-4286.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4286) TestClientProtocolProviderImpls passes on failure conditions also
[
https://issues.apache.org/jira/browse/MAPREDUCE-4286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324093#comment-14324093
]
Hadoop QA commented on MAPREDUCE-4286:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699229/MAPREDUCE-4286-2.patch
against trunk revision cf4b7f5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.security.TestJHSSecurity
org.apache.hadoop.conf.TestJobConf
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5203//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5203//console
This message is automatically generated.
TestClientProtocolProviderImpls passes on failure conditions also
-
Key: MAPREDUCE-4286
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4286
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4286-1.patch, MAPREDUCE-4286-2.patch,
MAPREDUCE-4286.patch, MAPREDUCE-4286.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6225) Fix new findbug warnings in hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-6225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324082#comment-14324082 ] Hudson commented on MAPREDUCE-6225: --- FAILURE: Integrated in Hadoop-Yarn-trunk #841 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/841/]) MAPREDUCE-6225. Fix new findbug warnings in hadoop-mapreduce-client-core. Contributed by Varun Saxena (junping_du: rev 814afa46efef201cb782072432fc744e1cb9c463) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/util/ResourceBundles.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/InMemoryReader.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/TaskLogAppender.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IndexCache.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/SecureShuffleUtils.java * hadoop-mapreduce-project/CHANGES.txt * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/fieldsel/FieldSelectionHelper.java Fix new findbug warnings in hadoop-mapreduce-client-core Key: MAPREDUCE-6225 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6225 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 2.7.0 Reporter: Jason Lowe Assignee: Varun Saxena Fix For: 2.7.0 Attachments: MAPREDUCE-6225.001.patch, MAPREDUCE-6225.002.patch, MAPREDUCE-6225.003.patch Recent precommit builds in hadoop-mapreduce-client-core are flagging findbug warnings that appear to be new with the recent findbugs upgrade. These need to be cleaned up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
