Hadoop-Mapreduce-trunk - Build # 1517 - Still Failing
See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1517/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 34435 lines...] Running org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedComparator Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.15 sec Running org.apache.hadoop.mapreduce.lib.partition.TestKeyFieldHelper Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.138 sec Running org.apache.hadoop.mapreduce.lib.partition.TestBinaryPartitioner Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.474 sec Running org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedPartitioner Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.417 sec Running org.apache.hadoop.mapreduce.TestChild Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.337 sec Running org.apache.hadoop.mapreduce.filecache.TestURIFragments Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 sec Running org.apache.hadoop.mapreduce.TestMapReduce Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.875 sec Results : Tests run: 466, Failures: 0, Errors: 0, Skipped: 11 [INFO] [INFO] Reactor Summary: [INFO] [INFO] hadoop-mapreduce-client ... SUCCESS [1.731s] [INFO] hadoop-mapreduce-client-core .. SUCCESS [37.798s] [INFO] hadoop-mapreduce-client-common SUCCESS [24.119s] [INFO] hadoop-mapreduce-client-shuffle ... SUCCESS [2.301s] [INFO] hadoop-mapreduce-client-app ... SUCCESS [5:50.968s] [INFO] hadoop-mapreduce-client-hs SUCCESS [1:38.958s] [INFO] hadoop-mapreduce-client-jobclient . FAILURE [59:12.437s] [INFO] hadoop-mapreduce-client-hs-plugins SKIPPED [INFO] Apache Hadoop MapReduce Examples .. SKIPPED [INFO] hadoop-mapreduce .. SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 1:07:48.955s [INFO] Finished at: Tue Aug 13 14:25:57 UTC 2013 [INFO] Final Memory: 39M/100M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-mapreduce-client-jobclient: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ? - [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn goals -rf :hadoop-mapreduce-client-jobclient Build step 'Execute shell' marked build as failure [FINDBUGS] Skipping publisher since build result is FAILURE Archiving artifacts Updating HADOOP-9847 Updating HADOOP-9848 Updating HADOOP-9845 Updating HADOOP-9583 Updating MAPREDUCE-5454 Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Hi Arun, Would it be possible to include YARN-1056 in the next RC - it is a straight-forward config change. I marked it as a blocker for 2.1.0. Thanks Karthik On Thu, Aug 8, 2013 at 8:14 AM, Kihwal Lee kih...@yahoo-inc.com wrote: Another blocker, HADOOP-9850, has been committed. Kihwal From: Arun C Murthy a...@hortonworks.com To: Daryn Sharp da...@yahoo-inc.com Cc: hdfs-...@hadoop.apache.org hdfs-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org yarn-...@hadoop.apache.org; common-...@hadoop.apache.org common-...@hadoop.apache.org Sent: Thursday, August 1, 2013 1:30 PM Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP. Thanks again! Arun On Aug 1, 2013, at 9:38 AM, Daryn Sharp da...@yahoo-inc.com wrote: I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix. The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting. Daryn On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote: Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight. thanks, Arun On Jul 21, 2013, at 12:43 AM, Devaraj Das d...@hortonworks.com wrote: I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This bug can easily be reproduced by some HBase tests. I'd like this to be considered before we make a beta release. Have spoken about this with some hdfs folks offline and I am told that it is being worked on. Thanks Devaraj On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur tuc...@gmail.com wrote: As I've mentioned in my previous email, if we get YARN-701 in, we should also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta. Else is a regression of a functionality it is already working. Because of that, to avoid continuing delaying the release, I'm suggesting to mention in the release notes the API changes and behavior changes that YARN-918 and YARN-701 will bring into the next beta or GA release. thx On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote: * YARN-701 It should be addressed before a GA release. Still, as it is this breaks unmanaged AMs and to me that would be a blocker for the beta. YARN-701 and the unmanaged AMs fix should be committed in tandem. * YARN-918 It is a consequence of YARN-701 and depends on it. YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0. * YARN-926 It would be nice to have it addressed before GA release. Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one. Thanks, +Vino -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: [UPDATE] Upgrade to protobuf 2.5.0 for the 2.1.0 release, HADOOP-9845
There is no indication that protoc 2.5.0 is breaking anything. Hadoop-trunk builds have been failing way before 1/2 way with: --- [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-yarn-client: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ? - [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-yarn-client: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ? --- The Hadoop-trunk #480 build failed with a JVM abort in a testcase towards the end of mapreduce tests. Until then there were no failures at all. I've increased heap size and tried a second run and the failure was earlier. I've looked a Hadoop-trunk builds prior to the HADOOP-9845 and it has been failing the same way in all the kept builds. We need to fix Hadoop-trunk builds independently of this. Any objection to commit HADOOP-9845 to branch-2 and the 2.1.0-beta branches to get all the other jenkins jobs working? I'll wait till tomorrow morning before proceeding. Thx On Mon, Aug 12, 2013 at 8:35 PM, Alejandro Abdelnur t...@cloudera.comwrote: Jenkins is running a full test run on trunk using protoc 2.5.0. https://builds.apache.org/job/Hadoop-trunk/480 And it seems go be going just fine. If everything looks OK, I'm planing to backport HADOOP-9845 to the 2.1.0-beta branch midday PST tomorrow. This will normalize all builds failures do the protoc mismatch. Thanks. Alejandro On Mon, Aug 12, 2013 at 5:53 PM, Alejandro Abdelnur t...@cloudera.comwrote: shooting to get it i n for 2.1.0. at moment is in trunk till the nightly finishes. then we'll decide in the mean time, you can have multiple versions installed in diff dirs and set the right one in the path thx Alejandro (phone typing) On Aug 12, 2013, at 17:47, Konstantin Shvachko shv.had...@gmail.com wrote: Ok. After installing protobuf 2.5.0 I can compile trunk. But now I cannot compile Hadoop-2 branches. None of them. So if I switch between branches I need to reinstall protobuf? Is there a consensus about going towards protobuf 2.5.0 upgrade in ALL versions? I did not get definite impression there is. If not it could be a pretty big disruption. Thanks, --Konst On Mon, Aug 12, 2013 at 3:19 PM, Alejandro Abdelnur t...@cloudera.com wrote: I've just committed HADOOP-9845 to trunk (only trunk at the moment). To build trunk now you need protoc 2.5.0 (the build will fail with a warning if you don't have it). We'd propagate this to the 2 branches once the precommit build is back to normal and see things are OK. Thanks. On Mon, Aug 12, 2013 at 2:57 PM, Alejandro Abdelnur t...@cloudera.com wrote: About to commit HADOOP-9845 to trunk, in 5 mins. This will make trunk use protoc 2.5.0. thx On Mon, Aug 12, 2013 at 11:47 AM, Giridharan Kesavan gkesa...@hortonworks.com wrote: I can take care of re-installing 2.4 and installing 2.5 in a different location. This would fix 2.0 branch builds as well. Thoughts? -Giri On Mon, Aug 12, 2013 at 11:37 AM, Alejandro Abdelnur t...@cloudera.com wrote: Giri, first of all, thanks for installing protoc 2.5.0. I didn't know we were installing them as the only version and not driven by env/path settings. Now we have a bit of a problem, precommit builds are broken because of mismatch of protoc (2.5.0) and protobuf JAR( 2.4.1). We have to options: 1* commit HADOOP-9845 that will bring protobuf to 2.5.0 and iron out any follow up issues. 2* reinstall protoc 2.4.1 in the jenkins machines and have 2.4.1 and 2.5.0 coexisting My take would be to commit HADOOP-9845 in trunk, iron out any issues an then merge it to the other branches. We need to sort this out quickly as precommits are not working. I'll wait till 3PM today for objections to option #1, if none I'll commit it to trunk. Thanks. Alejandro On Mon, Aug 12, 2013 at 11:30 AM, Giridharan Kesavan gkesa...@hortonworks.com wrote: Like I said protoc is upgraded from 2.4 to 2.5. 2.5 is in the default path. If we still need 2.4 I may have to install it. Let me know -Giri On Sat, Aug 10, 2013 at 7:01 AM, Alejandro Abdelnur t...@cloudera.com wrote: thanks giri, how do we set 2.4 or 2.5., what is the path to both so we can use and env to set it in the jobs? thx Alejandro (phone typing) On Aug 9, 2013, at 23:10, Giridharan Kesavan gkesa...@hortonworks.com wrote:
[jira] [Created] (MAPREDUCE-5457) Add a KeyOnlyTextOutputFormat to enable streaming write out text files without separators
Sandy Ryza created MAPREDUCE-5457: - Summary: Add a KeyOnlyTextOutputFormat to enable streaming write out text files without separators Key: MAPREDUCE-5457 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5457 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.1.0-beta Reporter: Sandy Ryza MR jobs sometimes want to just output lines of text, not key/value pairs. TextOutputFormat handles this by, if a null value is given, outputting only the key with no separator. Streaming jobs are unable to take advantage of this, because they can't output null values. A text output format that ignores values and only outputs keys would allow streaming jobs to output lines of text. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
