Hadoop-Mapreduce-0.23-Commit - Build # 1 - Failure
See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/1/ ### ## LAST 60 LINES OF THE CONSOLE ### Started by user gkesavan Building remotely on hadoop7 Location 'http://svn.apache.org/repos/asf/hadoop/common/braches/branch-0.23' does not exist Cleaning workspace /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-0.23-Commit Checking out http://svn.apache.org/repos/asf/hadoop/common/braches/branch-0.23 ERROR: Failed to check out http://svn.apache.org/repos/asf/hadoop/common/braches/branch-0.23 org.tmatesoft.svn.core.SVNException: svn: URL 'http://svn.apache.org/repos/asf/hadoop/common/braches/branch-0.23' doesn't exist at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64) at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51) at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:910) at hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:90) at hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:136) at hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:773) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:754) at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:738) at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1995) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:287) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: org.tmatesoft.svn.core.SVNErrorMessage: svn: URL 'http://svn.apache.org/repos/asf/hadoop/common/braches/branch-0.23' doesn't exist at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:163) at org.tmatesoft.svn.core.SVNErrorMessage.create(SVNErrorMessage.java:118) at org.tmatesoft.svn.core.wc.SVNUpdateClient.doCheckout(SVNUpdateClient.java:909) ... 15 more Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Resolved] (MAPREDUCE-3107) DFSIO tests are failing intermittently
[ https://issues.apache.org/jira/browse/MAPREDUCE-3107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John George resolved MAPREDUCE-3107. Resolution: Cannot Reproduce It seems like this was fixed somehow. I confirmed with Rajit and he does not see it anymore as well. Hence, closing this as cannot be reproduced. > DFSIO tests are failing intermittently > -- > > Key: MAPREDUCE-3107 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3107 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks, mrv2 >Affects Versions: 0.23.0 >Reporter: Rajit Saha >Assignee: John George >Priority: Blocker > Fix For: 0.23.0 > > > Intermittently DFSIO tests are failing either in "read" or "write" operations. > $HADOOP_COMMON_HOME/bin/hadoop --config $HADOOP_CONF_DIR jar > $HADOOP_MAPRED_HOME/hadoop-mapreduce-test-*.jar TestDFSIO -read -nrFiles 36 > -fileSize 320 > 11/09/27 03:21:03 INFO fs.TestDFSIO: TestDFSIO.0.0.6 > 11/09/27 03:21:03 INFO fs.TestDFSIO: nrFiles = 36 > 11/09/27 03:21:03 INFO fs.TestDFSIO: fileSize (MB) = 320.0 > 11/09/27 03:21:03 INFO fs.TestDFSIO: bufferSize = 100 > 11/09/27 03:21:03 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO > 11/09/27 03:21:03 INFO fs.TestDFSIO: creating control file: 335544320 bytes, > 36 files > 11/09/27 03:21:05 INFO fs.TestDFSIO: created control files for: 36 files > .. > .. > .. > .. > .. > 11/09/27 03:24:04 INFO mapreduce.Job: map 88% reduce 25% > 11/09/27 03:24:05 INFO mapreduce.Job: map 90% reduce 25% > 11/09/27 03:24:06 INFO mapreduce.Job: map 92% reduce 25% > 11/09/27 03:24:09 INFO mapreduce.Job: map 93% reduce 25% > 11/09/27 03:24:10 INFO mapreduce.Job: map 93% reduce 26% > 11/09/27 03:24:42 INFO mapreduce.Job: map 94% reduce 26% > 11/09/27 03:24:43 INFO mapreduce.Job: map 94% reduce 27% > 11/09/27 03:24:56 INFO mapreduce.Job: map 95% reduce 27% > 11/09/27 03:24:58 INFO mapreduce.Job: map 95% reduce 28% > 11/09/27 03:25:13 INFO mapreduce.Job: map 96% reduce 28% > 11/09/27 03:25:15 INFO mapreduce.Job: map 97% reduce 28% > 11/09/27 03:25:16 INFO mapreduce.Job: map 97% reduce 30% > 11/09/27 03:25:16 INFO mapreduce.Job: map 98% reduce 30% > 11/09/27 03:25:16 INFO mapreduce.Job: Job job_1317092846056_0012 failed with > state FAILED > 11/09/27 03:25:16 INFO mapreduce.Job: Counters: 44 > File System Counters > FILE: BYTES_READ=20931 > FILE: BYTES_WRITTEN=2192966 > FILE: READ_OPS=0 > FILE: LARGE_READ_OPS=0 > FILE: WRITE_OPS=0 > HDFS: BYTES_READ=10454250228 > HDFS: BYTES_WRITTEN=0 > HDFS: READ_OPS=172 > HDFS: LARGE_READ_OPS=0 > HDFS: WRITE_OPS=0 > org.apache.hadoop.mapreduce.JobCounter > NUM_FAILED_MAPS=6 > TOTAL_LAUNCHED_MAPS=44 > TOTAL_LAUNCHED_REDUCES=1 > DATA_LOCAL_MAPS=10 > RACK_LOCAL_MAPS=34 > SLOTS_MILLIS_MAPS=2340307 > org.apache.hadoop.mapreduce.TaskCounter > MAP_INPUT_RECORDS=36 > MAP_OUTPUT_RECORDS=140 > MAP_OUTPUT_BYTES=2103 > MAP_OUTPUT_MATERIALIZED_BYTES=2579 > SPLIT_RAW_BYTES=5210 > COMBINE_INPUT_RECORDS=0 > COMBINE_OUTPUT_RECORDS=0 > REDUCE_INPUT_GROUPS=0 > REDUCE_SHUFFLE_BYTES=3054 > REDUCE_INPUT_RECORDS=0 > REDUCE_OUTPUT_RECORDS=0 > SPILLED_RECORDS=140 > SHUFFLED_MAPS=33 > FAILED_SHUFFLE=0 > MERGED_MAP_OUTPUTS=0 > GC_TIME_MILLIS=59933 > CPU_MILLISECONDS=159470 > PHYSICAL_MEMORY_BYTES=11310596096 > VIRTUAL_MEMORY_BYTES=31425290240 > COMMITTED_HEAP_BYTES=12728664064 > Shuffle Errors > BAD_ID=0 > CONNECTION=0 > IO_ERROR=0 > WRONG_LENGTH=0 > WRONG_MAP=0 > WRONG_REDUCE=0 > org.apache.hadoop.mapreduce.lib.input.FileInputFormatCounter > BYTES_READ=4058 > org.apache.hadoop.mapreduce.lib.output.FileOutputFormatCounter > BYTES_WRITTEN=0 > java.io.IOException: Job failed! > at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:781) > at org.apache.hadoop.fs.TestDFSIO.runIOTest(TestDFSIO.java:340) > at org.apache.hadoop.fs.TestDFSIO.readTest(TestDFSIO.java:418) > at org.apache.hadoop.fs.TestDFSIO.run(TestDFSIO.java:522) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83) > at org.apache.hadoop.fs.TestDFSIO.main(TestDFSIO.java:445) > a
[jira] [Resolved] (MAPREDUCE-3160) Merge -r 1177530:1177531 from trunk to branch-0.23 to fix MAPREDUCE-2996 broke ant test compilation
[
https://issues.apache.org/jira/browse/MAPREDUCE-3160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash resolved MAPREDUCE-3160.
-
Resolution: Invalid
This was a problem with my ivy cache. After clearing it, and rebuilding
everything, this problem went away.
> Merge -r 1177530:1177531 from trunk to branch-0.23 to fix MAPREDUCE-2996
> broke ant test compilation
> ---
>
> Key: MAPREDUCE-3160
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3160
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Ravi Prakash
>
> I git bisected and the problem starts from
> adb810babaf25b9f9dae75b43d4beac782deaa01 . ant
> {noformat}
> [jsp-compile] log4j:WARN See
> http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> [javac]
> /home/raviprak/Code/hadoop/hadoop-all/hadoop-mapreduce-project/build.xml:398:
> warning: 'includeantruntime' was not set, defaulting to
> build.sysclasspath=last; set to false for repeatable builds
> [javac] Compiling 2 source files to
> /home/raviprak/Code/hadoop/hadoop-all/hadoop-mapreduce-project/build/classes
> [javac]
> /home/raviprak/Code/hadoop/hadoop-all/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java:697:
> cannot find symbol
> [javac] symbol : constructor
> JobInitedEvent(org.apache.hadoop.mapred.JobID,long,int,int,java.lang.String,boolean)
> [javac] location: class
> org.apache.hadoop.mapreduce.jobhistory.JobInitedEvent
> [javac] JobInitedEvent jie = new JobInitedEvent(
> [javac] ^
> [javac] Note:
> /home/raviprak/Code/hadoop/hadoop-all/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/JobInProgress.java
> uses or overrides a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] 1 error
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: 0.23 & trunk tars, we'll we publishing 1 tar per component or a single tar? What about source tar?
Other approach would be asking which tar to build. mapred-tar (mapred and common) hdfs-tar (hdfs and common) hadoop-tar (all) In this case, hbase can just use hdfs-tar. -Bharath From: Ravi Teja To: mapreduce-dev@hadoop.apache.org; common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org Sent: Wednesday, October 12, 2011 9:43 PM Subject: RE: 0.23 & trunk tars, we'll we publishing 1 tar per component or a single tar? What about source tar? I feel #4 as a better option. Regards, Ravi Teja -Original Message- From: Alejandro Abdelnur [mailto:t...@cloudera.com] Sent: Wednesday, October 12, 2011 9:38 PM To: common-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; hdfs-...@hadoop.apache.org Subject: 0.23 & trunk tars, we'll we publishing 1 tar per component or a single tar? What about source tar? Currently common, hdfs and mapred create partial tars which are not usable unless they are stitched together into a single tar. With HADOOP-7642 the stitching happens as part of the build. The build currently produces the following tars: 1* common TAR 2* hdfs (partial) TAR 3* mapreduce (partial) TAR 4* hadoop (full, the stitched one) TAR #1 on its own does not run anything, #2 and #3 on their own don't run. #4 runs hdfs & mapreduce. Questions: Q1. Does it make sense to publish #1, #2 & #3? Or #4 is sufficient and you start the services you want (i.e. Hbase would just use HDFS)? Q2. And what about a source TAR, does it make sense to have source TAR per component or a single TAR for the whole? For simplicity (for the build system and for users) I'd prefer a single binary TAR and a single source TAR. Thanks. Alejandro
[jira] [Created] (MAPREDUCE-3192) Fix Javadoc warning in JobClient.java and Cluster.java
Fix Javadoc warning in JobClient.java and Cluster.java -- Key: MAPREDUCE-3192 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3192 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Javadoc warnings in JobClient.java and Cluster.java need to be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3191) docs for map output compression incorrectly reference SequenceFile
docs for map output compression incorrectly reference SequenceFile -- Key: MAPREDUCE-3191 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3191 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Todd Lipcon Priority: Trivial The documentation currently says that map output compression uses SequenceFile compression. This hasn't been true in several years, since we use IFile for intermediate data now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3190) bin/yarn should barf early if HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set
bin/yarn should barf early if HADOOP_COMMON_HOME or HADOOP_HDFS_HOME are not set
Key: MAPREDUCE-3190
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3190
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 0.23.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Attachments: mr-3190.txt
Currently, if these env vars are not set when you run bin/yarn, it will crash
with various ClassNotFoundExceptions, having added {{/share/hadoop/hdfs}} to
the classpath. Rather, we should check for these env vars in the wrapper script
and display a reasonable error message.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3189) Add link decoration back to MR2's CSS
Add link decoration back to MR2's CSS - Key: MAPREDUCE-3189 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3189 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Attachments: mr-3189.txt I found the MRv2 web UI very difficult to use because it's not clear which items are links and which aren't. I'd like to change the CSS so that links are underlined, making it easier to see them (since they're also not in any different color) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3188) Lots of errors in logs when daemon startup fails
Lots of errors in logs when daemon startup fails Key: MAPREDUCE-3188 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3188 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Todd Lipcon Since the MR2 daemons are made up of lots of component services, if one of those components fails to start, it will cause the others to shut down as well, even if they haven't fully finished starting up. Currently, this causes the error output to have a bunch of NullPointerExceptions, IllegalStateExceptions, etc, which mask the actual root cause error at the top. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3187) Add names for various unnamed threads in MR2
Add names for various unnamed threads in MR2 Key: MAPREDUCE-3187 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3187 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Minor Simple patch to add thread names for all the places we use Executors, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3186) User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed.
User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed. -- Key: MAPREDUCE-3186 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3186 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Environment: linux Reporter: Ramgopal N If the resource manager is restarted while the job execution is in progress, the job is getting hanged. UI shows the job as running. In the RM log, it is throwing an error "ERROR org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: AppAttemptId doesnt exist in cache appattempt_1318579738195_0004_01" In the console MRAppMaster and Runjar processes are not getting killed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
