[jira] [Created] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
Jonathan Eagles created MAPREDUCE-5587: -- Summary: TestTextOutputFormat fails on JDK7 Key: MAPREDUCE-5587 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5586) TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797779#comment-13797779 ] Hudson commented on MAPREDUCE-5586: --- SUCCESS: Integrated in Hadoop-Yarn-trunk #365 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/365/]) MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532887) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory - Key: MAPREDUCE-5586 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5586 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5586.patch, MAPREDUCE-5586.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5585) TestCopyCommitter#testNoCommitAction Fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797781#comment-13797781 ] Hudson commented on MAPREDUCE-5585: --- SUCCESS: Integrated in Hadoop-Yarn-trunk #365 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/365/]) MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532875) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java TestCopyCommitter#testNoCommitAction Fails on JDK7 -- Key: MAPREDUCE-5585 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5585 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 0.23.9, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5585.patch, MAPREDUCE-5585.patch TestCopyCommitter#testNoCommitAction fails on JDK7 when run after testAtomicCommitMissingFinal or testAtomicCommitExistingFinal. Config settings are from atomic tests are being accidentally used for testNoCommitAction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5586) TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797806#comment-13797806 ] Hudson commented on MAPREDUCE-5586: --- FAILURE: Integrated in Hadoop-Hdfs-0.23-Build #763 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/763/]) MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532896) * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory - Key: MAPREDUCE-5586 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5586 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5586.patch, MAPREDUCE-5586.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5585) TestCopyCommitter#testNoCommitAction Fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797807#comment-13797807 ] Hudson commented on MAPREDUCE-5585: --- FAILURE: Integrated in Hadoop-Hdfs-0.23-Build #763 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/763/]) MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532881) * /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java TestCopyCommitter#testNoCommitAction Fails on JDK7 -- Key: MAPREDUCE-5585 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5585 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 0.23.9, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5585.patch, MAPREDUCE-5585.patch TestCopyCommitter#testNoCommitAction fails on JDK7 when run after testAtomicCommitMissingFinal or testAtomicCommitExistingFinal. Config settings are from atomic tests are being accidentally used for testNoCommitAction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Created] (MAPREDUCE-5588) TaskTrackers get killed by JettyBugMonitor because of incredibly high cpu usage
Anthony MOI created MAPREDUCE-5588:
--
Summary: TaskTrackers get killed by JettyBugMonitor because of
incredibly high cpu usage
Key: MAPREDUCE-5588
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5588
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 1.1.2
Reporter: Anthony MOI
We are running a little cluster with 10 servers running task trackers. All of
them are getting killed randomly with the following message
{quote}
2013-10-17 11:32:31,037 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
Jetty CPU usage: 120093277.1%. This is greater than the fatal threshold
mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
2013-10-17 11:32:31,039 INFO org.apache.hadoop.mapred.TaskTracker:
SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down TaskTracker
/
{quote}
Everytime, the message notices a cpu usage above 120M%. Everything has been
running for a while now (since 1.1.2 release) without any problems, and it
started just like that.
Any idea of what could cause this ?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5586) TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797879#comment-13797879 ] Hudson commented on MAPREDUCE-5586: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #1555 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1555/]) MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532887) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory - Key: MAPREDUCE-5586 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5586 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5586.patch, MAPREDUCE-5586.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5585) TestCopyCommitter#testNoCommitAction Fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797881#comment-13797881 ] Hudson commented on MAPREDUCE-5585: --- FAILURE: Integrated in Hadoop-Hdfs-trunk #1555 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1555/]) MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532875) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java TestCopyCommitter#testNoCommitAction Fails on JDK7 -- Key: MAPREDUCE-5585 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5585 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 0.23.9, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5585.patch, MAPREDUCE-5585.patch TestCopyCommitter#testNoCommitAction fails on JDK7 when run after testAtomicCommitMissingFinal or testAtomicCommitExistingFinal. Config settings are from atomic tests are being accidentally used for testNoCommitAction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5586) TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-5586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797940#comment-13797940 ] Hudson commented on MAPREDUCE-5586: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1581 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1581/]) MAPREDUCE-5586. TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532887) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory - Key: MAPREDUCE-5586 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5586 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5586.patch, MAPREDUCE-5586.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5585) TestCopyCommitter#testNoCommitAction Fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797942#comment-13797942 ] Hudson commented on MAPREDUCE-5585: --- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1581 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1581/]) MAPREDUCE-5585. TestCopyCommitter#testNoCommitAction Fails on JDK7 (jeagles) (jeagles: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1532875) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyCommitter.java TestCopyCommitter#testNoCommitAction Fails on JDK7 -- Key: MAPREDUCE-5585 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5585 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 0.23.9, 2.3.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Fix For: 3.0.0, 2.3.0, 0.23.10 Attachments: MAPREDUCE-5585.patch, MAPREDUCE-5585.patch TestCopyCommitter#testNoCommitAction fails on JDK7 when run after testAtomicCommitMissingFinal or testAtomicCommitExistingFinal. Config settings are from atomic tests are being accidentally used for testNoCommitAction. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-5587: --- Description: the test method run order on JDK7 is not fixed causing test method inter-dependencies to show themselves. TestTextOutputFormat fails on JDK7 -- Key: MAPREDUCE-5587 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles the test method run order on JDK7 is not fixed causing test method inter-dependencies to show themselves. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13797986#comment-13797986 ] Jonathan Eagles commented on MAPREDUCE-5587: Through experimentation, only two different test method orders were shown to succeed. |testFormatWithCustomSeparator| |testCompress| |testFormat| |testFormat| |testFormatWithCustomSeparator| |testCompress| All other orders produced failures. Thus the inter-dependency is that _testCompress_ must be run *immediately after* _testFormatWithCustomSeparator_. *Dependencies Identified* * All tests verify success by reading input from the same test file test.txt * _testCompress_ relies on having the same custom separator as _testFormatWithCustomSeparator_ * _testCompress_ actually writes a file name test.txt.deflate TestTextOutputFormat fails on JDK7 -- Key: MAPREDUCE-5587 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles the test method run order on JDK7 is not fixed causing test method inter-dependencies to show themselves. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5577) Allow querying the JobHistoryServer by job arrival time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798091#comment-13798091 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-5577: Okay, that gives me a little more detail, but I still don't get it fully. Let's say the client is looking for finished jobs and the window it uses it an hour. It first asks JHS to give jobs that finished from 3PM to 4PM. JHS returns three jobs that finished at 3:01, 3:03, 3:15. Now all client needs to do is to move the window already and ask for jobs that finished between 3:15PM and 4:15PM, no? One significant point is that finish-time is set by AMs. But every time the getJobs() API is called, JHS scans the intermediate done directory and populates its cache. So, when a client asks for finished-jobs beween 3:PM-4:PM, it is guaranteed to get any finished jobs in that duration. It seems like you are hinting that this contract is broken. Is it? Also in the patch, I don't see where aBegin and aEnd are getting used. In CachedHistoryStorage, aBegin and aEnd are validated but never used. It does look like I'm missing something, please bear with me. I'm just making sure we are doing the right thing. The same API if needed here can be added to Application History Server which I am watching over. Tx. Allow querying the JobHistoryServer by job arrival time --- Key: MAPREDUCE-5577 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5577 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5577.patch The JobHistoryServer REST APIs currently allow querying by job submit time and finish time. However, jobs don't necessarily arrive in order of their finish time, meaning that a client who wants to stay on top of all completed jobs needs to query large time intervals to make sure they're not missing anything. Exposing functionality to allow querying by the time a job lands at the JobHistoryServer would allow clients to set the start of their query interval to the time of their last query. The arrival time of a job would be defined as the time that it lands in the done directory and can be picked up using the last modified date on history files. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5577) Allow querying the JobHistoryServer by job arrival time
[ https://issues.apache.org/jira/browse/MAPREDUCE-5577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798111#comment-13798111 ] Sandy Ryza commented on MAPREDUCE-5577: --- bq. Now all client needs to do is to move the window already and ask for jobs that finished between 3:15PM and 4:15PM, no? My understanding is that an AM could set its finish time at 3:10, then GC pause for a few minutes, and then write its info to the intermediate-done dir after the client's request that you mentioned is serviced. The client would then miss the job because 3:10 wouldn't be in its 3:15 to 4:15 window. bq. Also in the patch, I don't see where aBegin and aEnd are getting used. In CachedHistoryStorage, aBegin and aEnd are validated but never used. This is my mistake, they should be checked in CachedHistoryStorage in the same way sBegin/sEnd and fBegin/sEnd are. I didn't think that the patch would get attention already and should have commented that this was a preliminary version that I hadn't proofread. It still needs tests and some manual verification as well. Allow querying the JobHistoryServer by job arrival time --- Key: MAPREDUCE-5577 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5577 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: MAPREDUCE-5577.patch The JobHistoryServer REST APIs currently allow querying by job submit time and finish time. However, jobs don't necessarily arrive in order of their finish time, meaning that a client who wants to stay on top of all completed jobs needs to query large time intervals to make sure they're not missing anything. Exposing functionality to allow querying by the time a job lands at the JobHistoryServer would allow clients to set the start of their query interval to the time of their last query. The arrival time of a job would be defined as the time that it lands in the done directory and can be picked up using the last modified date on history files. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Resolved] (MAPREDUCE-5588) TaskTrackers get killed by JettyBugMonitor because of incredibly high cpu usage
[
https://issues.apache.org/jira/browse/MAPREDUCE-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth resolved MAPREDUCE-5588.
--
Resolution: Duplicate
Assignee: Chris Nauroth
Hello, [~moian].
There is a known bug in Jetty that can cause it to consume a ton of CPU and not
make any real progress. As a workaround, the Hadoop code monitors for this
condition and kills the tasktracker process if it finds Jetty consuming too
much CPU. (It's better to kill the tasktracker outright then leave it running
in an unresponsive state.) This monitoring thread was added in MAPREDUCE-3184.
If you review that issue, you can find more background information on
configuration and tuning.
The bug is set to be fixed in Jetty version 6.1.27, but we've been waiting a
long time on that release. We also can't easily jump to a new major version of
Jetty due to API backwards-compatibility issues. MAPREDUCE-2980 contains
ongoing discussion about our upgrade plan. As a workaround, there are also
patches available that you can apply on top of Jetty 6.1.26.
I'm going to resolve this as a duplicate of MAPREDUCE-2980.
TaskTrackers get killed by JettyBugMonitor because of incredibly high cpu
usage
---
Key: MAPREDUCE-5588
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5588
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Affects Versions: 1.1.2
Reporter: Anthony MOI
Assignee: Chris Nauroth
Labels: cpu-usage, jetty, tasktracker
We are running a little cluster with 10 servers running task trackers. All of
them are getting killed randomly with the following message
{quote}
2013-10-17 11:32:31,037 FATAL org.apache.hadoop.mapred.JettyBugMonitor:
Jetty CPU usage: 120093277.1%. This is greater than the fatal threshold
mapred.tasktracker.jetty.cpu.threshold.fatal. Aborting JVM.
2013-10-17 11:32:31,039 INFO org.apache.hadoop.mapred.TaskTracker:
SHUTDOWN_MSG:
/
SHUTDOWN_MSG: Shutting down TaskTracker
/
{quote}
Everytime, the message notices a cpu usage above 120M%. Everything has been
running for a while now (since 1.1.2 release) without any problems, and it
started just like that.
Any idea of what could cause this ?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Created] (MAPREDUCE-5589) MapReduce Job setup error leaves no useful info to users (when LinuxTaskController is used)
Benoy Antony created MAPREDUCE-5589: --- Summary: MapReduce Job setup error leaves no useful info to users (when LinuxTaskController is used) Key: MAPREDUCE-5589 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5589 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1 Affects Versions: 1.2.1 Environment: I found that a significant portion of the main() in JobLocalizer will not have its output displayed in the tasklogs. Before fix : Error initializing attempt_201310122204_0002_m_02_0: java.io.IOException: Job initialization failed (1) with output: Reading task controller config from /etc/hadoop/taskcontroller.cfg main : command provided 0 main : user is bantony Good mapred-local-dirs are /hadoop12/scratch,/hadoop04/scratch,/hadoop09/scratch,/hadoop03/scratch,/hadoop05/scratch,/hadoop01/scratch,/hadoop11/scratch,/hadoop10/scratch,/hadoop08/scratch,/hadoop06/scratch,/hadoop07/scratch,/hadoop02/scratch at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193) at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1340) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1315) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1230) at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2641) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:255) at org.apache.hadoop.util.Shell.run(Shell.java:182) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375) at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186) ... 8 more Reporter: Benoy Antony Assignee: Benoy Antony Fix For: 1.3.0 -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5589) MapReduce Job setup error leaves no useful info to users (when LinuxTaskController is used)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benoy Antony updated MAPREDUCE-5589: Attachment: MAPREDUCE-5368.patch Patch for branch-1 MapReduce Job setup error leaves no useful info to users (when LinuxTaskController is used) Key: MAPREDUCE-5589 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5589 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1 Affects Versions: 1.2.1 Environment: I found that a significant portion of the main() in JobLocalizer will not have its output displayed in the tasklogs. Before fix : Error initializing attempt_201310122204_0002_m_02_0: java.io.IOException: Job initialization failed (1) with output: Reading task controller config from /etc/hadoop/taskcontroller.cfg main : command provided 0 main : user is bantony Good mapred-local-dirs are /hadoop12/scratch,/hadoop04/scratch,/hadoop09/scratch,/hadoop03/scratch,/hadoop05/scratch,/hadoop01/scratch,/hadoop11/scratch,/hadoop10/scratch,/hadoop08/scratch,/hadoop06/scratch,/hadoop07/scratch,/hadoop02/scratch at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193) at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1340) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1315) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1230) at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2641) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:255) at org.apache.hadoop.util.Shell.run(Shell.java:182) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375) at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:186) ... 8 more Reporter: Benoy Antony Assignee: Benoy Antony Fix For: 1.3.0 Attachments: MAPREDUCE-5368.patch -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5589) MapReduce Job setup error leaves no useful info to users (when LinuxTaskController is used)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798169#comment-13798169 ] Benoy Antony commented on MAPREDUCE-5589: - Proper error output by JobLocalizer is displayed when the patch is applied. Error initializing attempt_201310140952_0001_m_02_0: java.io.IOException: Job initialization failed (255) with output: Reading task controller config from /etc/hadoop/taskcontroller.cfg main : command provided 0 main : user is bantony Good mapred-local-dirs are /hadoop12/scratch,/hadoop04/scratch,/hadoop09/scratch,/hadoop05/scratch,/hadoop01/scratch,/hadoop11/scratch,/hadoop10/scratch,/hadoop08/scratch,/hadoop06/scratch,/hadoop07/scratch,/hadoop02/scratch java.io.IOException: Call to localhost/127.0.0.1:60487 failed on local exception: java.io.IOException: javax.security.sasl.SaslException: DIGEST-MD5: No common protection layer between client and server at org.apache.hadoop.ipc.Client.wrapException(Client.java:1155) at org.apache.hadoop.ipc.Client.call(Client.java:1123) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229) at $Proxy5.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.checkVersion(RPC.java:422) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:414) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:392) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:374) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:444) at org.apache.hadoop.mapred.JobLocalizer$2.run(JobLocalizer.java:529) at org.apache.hadoop.mapred.JobLocalizer$2.run(JobLocalizer.java:527) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.mapred.JobLocalizer.main(JobLocalizer.java:526) Caused by: java.io.IOException: javax.security.sasl.SaslException: DIGEST-MD5: No common protection layer between client and server at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:568) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:513) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:616) at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:203) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1254) at org.apache.hadoop.ipc.Client.call(Client.java:1098) ... 13 more Caused by: javax.security.sasl.SaslException: DIGEST-MD5: No common protection layer between client and server at com.sun.security.sasl.digest.DigestMD5Client.checkQopSupport(DigestMD5Client.java:396) at com.sun.security.sasl.digest.DigestMD5Client.evaluateChallenge(DigestMD5Client.java:208) at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:168) at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:410) at org.apache.hadoop.ipc.Client$Connection.access$1300(Client.java:203) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:609) at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:606) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:605) ... 16 more at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskController.java:193) at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1340) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1315) at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1230) at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2641) at java.lang.Thread.run(Thread.java:662) Caused by: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:255) at org.apache.hadoop.util.Shell.run(Shell.java:182) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375) at
[jira] [Updated] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-5587: --- Attachment: MAPREDUCE-5587.patch TestTextOutputFormat fails on JDK7 -- Key: MAPREDUCE-5587 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: MAPREDUCE-5587.patch the test method run order on JDK7 is not fixed causing test method inter-dependencies to show themselves. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
[ https://issues.apache.org/jira/browse/MAPREDUCE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated MAPREDUCE-5587: --- Affects Version/s: 0.23.10 2.3.0 3.0.0 Status: Patch Available (was: Open) TestTextOutputFormat fails on JDK7 -- Key: MAPREDUCE-5587 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0, 2.3.0, 0.23.10 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: MAPREDUCE-5587.patch the test method run order on JDK7 is not fixed causing test method inter-dependencies to show themselves. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Moved] (MAPREDUCE-5590) MiniMRClientClusterFactory in Hadoop 1.x forces local mode
[
https://issues.apache.org/jira/browse/MAPREDUCE-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas moved HADOOP-10056 to MAPREDUCE-5590:
-
Affects Version/s: (was: 1.0.4)
1.0.4
Key: MAPREDUCE-5590 (was: HADOOP-10056)
Project: Hadoop Map/Reduce (was: Hadoop Common)
MiniMRClientClusterFactory in Hadoop 1.x forces local mode
--
Key: MAPREDUCE-5590
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5590
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.0.4
Reporter: Abraham Elmahrek
Noticed the following in the 1.x branches:
{code}
public class MiniMRClientClusterFactory {
public static MiniMRClientCluster create(Class? caller, int noOfNMs,
Configuration conf) throws IOException {
FileSystem fs = FileSystem.getLocal(new Configuration());
MiniMRCluster miniMRCluster = new MiniMRCluster(noOfNMs, fs.getUri()
.toString(), 1);
return new MiniMRClusterAdapter(miniMRCluster);
}
}
{code}
The 2.x branches allow you to set what hadoop configuration to use (and which
DFS cluster to connect to). For instance, in the Sqoop project we are using
the MiniDFSCluster in conjunction with the MiniMRCluster.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798439#comment-13798439
]
Ravi Prakash commented on MAPREDUCE-3860:
-
Aleksey!
Tests are failing and erring.
{noformat}
Running org.apache.hadoop.tools.rumen.TestRumenJobTraces
Tests run: 12, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 63.803 sec
FAILURE! - in org.apache.hadoop.tools.rumen.TestRumenJobTraces
Running org.apache.hadoop.tools.rumen.TestRumenAnonymization
Tests run: 29, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 10.742 sec
FAILURE! - in org.apache.hadoop.tools.rumen.TestRumenAnonymization
{noformat}
[Rumen] Bring back the removed Rumen unit tests
---
Key: MAPREDUCE-3860
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tools/rumen
Reporter: Ravi Gummadi
Assignee: Andrey Klochkov
Attachments: MAPREDUCE-3860.patch, rumen-test-data.tar.gz
MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder
and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need
to be brought back:
TestZombieJob.java
TestRumenJobTraces.java
TestRumenFolder.java
TestRumenAnonymization.java
TestParsedLine.java
TestConcurrentRead.java
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798459#comment-13798459 ] Andrey Klochkov commented on MAPREDUCE-3860: Ravi, Can you please provide more info? I can't reproduce it in my environment. Surefire logs would be fine. BTW my name is Andrey :-) [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[ https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-4980: --- Attachment: MAPREDUCE-4980--n7.patch Rebasing the patch. Parallel test execution of hadoop-mapreduce-client-core --- Key: MAPREDUCE-4980 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980 Project: Hadoop Map/Reduce Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch, MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch, MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
[
https://issues.apache.org/jira/browse/MAPREDUCE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798546#comment-13798546
]
Ravi Prakash commented on MAPREDUCE-5587:
-
Thanks Jon. Minor nits:
- in testCompress(), you didn't have to remove separator. I'd rather check
\u0001 than \t.
- {code} output += out + \n; {code}
- Do you think we should clean up the output files in an @After ? Or do you
think its more useful to have them around after the test for debugging? I could
go either way.
Otherwise patch looks good.
TestTextOutputFormat fails on JDK7
--
Key: MAPREDUCE-5587
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 3.0.0, 2.3.0, 0.23.10
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
Attachments: MAPREDUCE-5587.patch
the test method run order on JDK7 is not fixed causing test method
inter-dependencies to show themselves.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798551#comment-13798551
]
Ravi Prakash commented on MAPREDUCE-3860:
-
Oops! Sorry Andrey!
Here are the messages:
{code}
testCurrentJHParser(org.apache.hadoop.tools.rumen.TestRumenJobTraces) Time
elapsed: 35.14 sec FAILURE!
java.lang.AssertionError: Job failed
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.hadoop.tools.rumen.TestRumenJobTraces.testCurrentJHParser(TestRumenJobTraces.java:430)
testProcessInputArgument(org.apache.hadoop.tools.rumen.TestRumenJobTraces)
Time elapsed: 0.081 sec FAILURE!
java.lang.AssertionError: Some of the history log files do not match the
expected.
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.hadoop.tools.rumen.TestRumenJobTraces.validateHistoryLogPaths(TestRumenJobTraces.java:315)
at
org.apache.hadoop.tools.rumen.TestRumenJobTraces.testProcessInputArgument(TestRumenJobTraces.java:278)
testRumenAnonymization(org.apache.hadoop.tools.rumen.TestRumenAnonymization)
Time elapsed: 6.799 sec ERROR!
org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:693)
at
org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster$JobHistoryServerWrapper.serviceStart(MiniMRYarnCluster.java:214)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:80)
at
org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:41)
at
org.apache.hadoop.tools.rumen.TestRumenAnonymization.testRumenAnonymization(TestRumenAnonymization.java:786)
{code}
I'm running Linux 3.9 with Sun JDK1.7
[Rumen] Bring back the removed Rumen unit tests
---
Key: MAPREDUCE-3860
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tools/rumen
Reporter: Ravi Gummadi
Assignee: Andrey Klochkov
Attachments: MAPREDUCE-3860.patch, rumen-test-data.tar.gz
MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder
and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need
to be brought back:
TestZombieJob.java
TestRumenJobTraces.java
TestRumenFolder.java
TestRumenAnonymization.java
TestParsedLine.java
TestConcurrentRead.java
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-5587) TestTextOutputFormat fails on JDK7
[
https://issues.apache.org/jira/browse/MAPREDUCE-5587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798553#comment-13798553
]
Hadoop QA commented on MAPREDUCE-5587:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609020/MAPREDUCE-5587.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapred.TestClientRedirect
The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.v2.TestUberAM
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4123//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4123//console
This message is automatically generated.
TestTextOutputFormat fails on JDK7
--
Key: MAPREDUCE-5587
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5587
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 3.0.0, 2.3.0, 0.23.10
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles
Attachments: MAPREDUCE-5587.patch
the test method run order on JDK7 is not fixed causing test method
inter-dependencies to show themselves.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[ https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated MAPREDUCE-3860: --- Attachment: MAPREDUCE-3860--n2.patch [Rumen] Bring back the removed Rumen unit tests --- Key: MAPREDUCE-3860 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860 Project: Hadoop Map/Reduce Issue Type: Bug Components: tools/rumen Reporter: Ravi Gummadi Assignee: Andrey Klochkov Attachments: MAPREDUCE-3860--n2.patch, MAPREDUCE-3860.patch, rumen-test-data.tar.gz MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need to be brought back: TestZombieJob.java TestRumenJobTraces.java TestRumenFolder.java TestRumenAnonymization.java TestParsedLine.java TestConcurrentRead.java -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (MAPREDUCE-3860) [Rumen] Bring back the removed Rumen unit tests
[
https://issues.apache.org/jira/browse/MAPREDUCE-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798598#comment-13798598
]
Andrey Klochkov commented on MAPREDUCE-3860:
Ravi,
I reproduced the bug in {{testProcessInputArgument}} on a Linux machine, will
attach a fixed patch shortly. Also I tried to reproduce the job failure in
{{testCurrentJHParser}} and made a few runs of the test on 2 machines with 2.x
and 3.x Linux kernels and different flavors of JDK7, but all runs succeeded.
Worked on OSX too. Didn't see any issues caused by OOM. If it happens again,
try giving more memory to Maven.
[Rumen] Bring back the removed Rumen unit tests
---
Key: MAPREDUCE-3860
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3860
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tools/rumen
Reporter: Ravi Gummadi
Assignee: Andrey Klochkov
Attachments: MAPREDUCE-3860--n2.patch, MAPREDUCE-3860.patch,
rumen-test-data.tar.gz
MAPREDUCE-3582 did not move some of the Rumen unit tests to the new folder
and then MAPREDUCE-3705 deleted those unit tests. These Rumen unit tests need
to be brought back:
TestZombieJob.java
TestRumenJobTraces.java
TestRumenFolder.java
TestRumenAnonymization.java
TestParsedLine.java
TestConcurrentRead.java
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4980) Parallel test execution of hadoop-mapreduce-client-core
[
https://issues.apache.org/jira/browse/MAPREDUCE-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13798642#comment-13798642
]
Hadoop QA commented on MAPREDUCE-4980:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609036/MAPREDUCE-4980--n7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 119
new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-tools/hadoop-extras hadoop-tools/hadoop-gridmix:
org.apache.hadoop.mapred.TestJobCleanup
The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-tools/hadoop-extras hadoop-tools/hadoop-gridmix:
org.apache.hadoop.mapreduce.v2.TestUberAM
The test build failed in
hadoop-tools/hadoop-gridmix hadoop-tools/hadoop-extras
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4124//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4124//console
This message is automatically generated.
Parallel test execution of hadoop-mapreduce-client-core
---
Key: MAPREDUCE-4980
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4980
Project: Hadoop Map/Reduce
Issue Type: Test
Components: test
Affects Versions: 3.0.0
Reporter: Tsuyoshi OZAWA
Assignee: Andrey Klochkov
Attachments: MAPREDUCE-4980.1.patch, MAPREDUCE-4980--n3.patch,
MAPREDUCE-4980--n4.patch, MAPREDUCE-4980--n5.patch, MAPREDUCE-4980--n6.patch,
MAPREDUCE-4980--n7.patch, MAPREDUCE-4980.patch
The maven surefire plugin supports parallel testing feature. By using it, the
tests can be run more faster.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
