Hadoop-Mapreduce-trunk - Build # 1467 - Still Failing
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1467/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 31171 lines...] testChild(org.apache.hadoop.mapreduce.TestChild) Tests in error: testClassPath(org.apache.hadoop.mapred.TestMiniMRClasspath): Job failed! testExternalWritable(org.apache.hadoop.mapred.TestMiniMRClasspath): Job failed! testMapReduce(org.apache.hadoop.mapred.TestClusterMapReduceTestCase): Job failed! testMapReduceRestarting(org.apache.hadoop.mapred.TestClusterMapReduceTestCase): Job failed! testDistinctUsers(org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers): Job failed! testMultipleSpills(org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers): Job failed! testMR(org.apache.hadoop.mapred.TestClusterMRNotification): Job failed! testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem): Job failed! testWithDFS(org.apache.hadoop.mapred.TestJobSysDirWithDFS): Job failed! testMerge(org.apache.hadoop.mapred.TestMerge): File /testplugin/output does not exist. testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem): Job failed! testLazyOutput(org.apache.hadoop.mapred.TestLazyOutput): Job failed! testComplexName(org.apache.hadoop.mapred.TestJobName): Job failed! testComplexNameWithRegex(org.apache.hadoop.mapred.TestJobName): Job failed! testFailingMapper(org.apache.hadoop.mapreduce.v2.TestMRJobs): 0 Tests run: 465, Failures: 29, Errors: 15, Skipped: 11 [INFO] [INFO] Reactor Summary: [INFO] [INFO] hadoop-mapreduce-client ... SUCCESS [1.668s] [INFO] hadoop-mapreduce-client-core .. SUCCESS [54.057s] [INFO] hadoop-mapreduce-client-common SUCCESS [23.767s] [INFO] hadoop-mapreduce-client-shuffle ... SUCCESS [2.228s] [INFO] hadoop-mapreduce-client-app ... SUCCESS [5:41.896s] [INFO] hadoop-mapreduce-client-hs SUCCESS [1:25.494s] [INFO] hadoop-mapreduce-client-jobclient . FAILURE [21:21.768s] [INFO] hadoop-mapreduce-client-hs-plugins SKIPPED [INFO] Apache Hadoop MapReduce Examples .. SKIPPED [INFO] hadoop-mapreduce .. SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 29:51.512s [INFO] Finished at: Mon Jun 24 13:45:59 UTC 2013 [INFO] Final Memory: 21M/114M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-mapreduce-client-jobclient: There are test failures. [ERROR] [ERROR] Please refer to /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports for the individual test results. [ERROR] - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn goals -rf :hadoop-mapreduce-client-jobclient Build step 'Execute shell' marked build as failure [FINDBUGS] Skipping publisher since build result is FAILURE Archiving artifacts Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Resolved] (MAPREDUCE-3502) Review all Service.stop() operations and make sure that they work before a service is started
[ https://issues.apache.org/jira/browse/MAPREDUCE-3502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-3502. --- Resolution: Fixed Fix Version/s: 2.1.0-beta 3.0.0 this is fixed now Review all Service.stop() operations and make sure that they work before a service is started - Key: MAPREDUCE-3502 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3502 Project: Hadoop Map/Reduce Issue Type: Task Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Steve Loughran Assignee: Steve Loughran Fix For: 3.0.0, 2.1.0-beta Attachments: MAPREDUCE-3502.patch, MAPREDUCE-3502.patch Original Estimate: 24h Time Spent: 2.5h Remaining Estimate: 21.5h MAPREDUCE-3431 has shown that some of the key services's shutdown operations are not robust against being invoked before the service is started. They need to be by # not calling other things if the other things are null # not being re-entrant (i.e. make synchronized if possible), Maybe # have a StopService operation that only stops a service if it is live # factor out the is-running test from the base service class and make it a pre-check for all the child services, so they bail out sooner rather than later. This would be the best as it would be the one guaranteed to work consistently across all instances, so only one or two would need testing my first iteration will skip the sync though it's something to consider. Testing: try to create each instance; call stop() straight after construction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5349) TestClusterMapReduceTestCase and TestJobName fail on Windows in branch-2
Chuan Liu created MAPREDUCE-5349:
Summary: TestClusterMapReduceTestCase and TestJobName fail on
Windows in branch-2
Key: MAPREDUCE-5349
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5349
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.1.0-beta
Reporter: Chuan Liu
Assignee: Chuan Liu
Priority: Minor
The two unit tests fails due to MiniMRCluster use test class fullname in
branch-2, instead of simple name as in trunk, to construct the MiniMRCluster
identifier. Full name in the identifier almost always leads to a command script
path with length larger than 260 characters which will generate an exception
{{DefaultContainerExecutor.launchContainer()}} when launching the container
script.
The exception looks like the follows:
{noformat}
2013-06-24 09:45:03,060 WARN [ContainersLauncher #0] launcher.ContainerLaunch
(ContainerLaunch.java:call(262)) - Failed to launch container.
java.io.IOException: Cannot launch container using script at path
C:/Users/chuanliu/AppData/Local/Temp/1/1372092295656/org.apache.hadoop.mapred.ClusterMapReduceTestCaseConfigurableMiniMRCluster_1106798455-localDir-nm-0_1/usercache/chuanliu/appcache/application_1372092193505_0001/container_1372092193505_0001_01_01/default_container_executor.cmd,
because it exceeds the maximum supported path length of 260 characters.
Consider configuring shorter directories in yarn.nodemanager.local-dirs.
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:159)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:257)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:1)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5350) Expose Fair Scheduler-specific queue metrics
Sandy Ryza created MAPREDUCE-5350: - Summary: Expose Fair Scheduler-specific queue metrics Key: MAPREDUCE-5350 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5350 Project: Hadoop Map/Reduce Issue Type: Improvement Components: scheduler Affects Versions: 2.0.5-alpha Reporter: Sandy Ryza Assignee: Sandy Ryza When the Fair Scheduler is enabled, QueueMetrics should include fair share, minimum share, and maximum share. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5351) JobTracker memory leak caused by CleanupQueue reopening FileSystem
Sandy Ryza created MAPREDUCE-5351: - Summary: JobTracker memory leak caused by CleanupQueue reopening FileSystem Key: MAPREDUCE-5351 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5351 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 1.1.2 Reporter: Sandy Ryza Assignee: Sandy Ryza Priority: Critical When a job is completed, closeAllForUGI is called to close all the cached FileSystems in the FileSystem cache. However, the CleanupQueue may run after this occurs and call FileSystem.get() to delete the staging directory, adding a FileSystem to the cache that will never be closed. People on the user-list have reported this causing their JobTrackers to OOME every two weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5352) Optimize node local splits generated by CombineFileInputFormat
Siddharth Seth created MAPREDUCE-5352: - Summary: Optimize node local splits generated by CombineFileInputFormat Key: MAPREDUCE-5352 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5352 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.0.5-alpha Reporter: Siddharth Seth Assignee: Siddharth Seth CombineFileInputFormat currently walks through all available nodes and generates multiple (maxSplitsPerNode) splits on a single node before attempting to generate splits on subsequent nodes. This ends up reducing the possibility of generating splits for subsequent nodes - since these blocks will no longer be available for subsequent nodes. Allowing splits to go 1 block above the max-split-size makes this worse. Allocating a single split per node in one iteration, should help increase the distribution of splits across nodes - so the subsequent nodes will have more blocks to choose from. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
