Re: [VOTE] Release Apache Hadoop 2.2.0
+1 Deployed on secure and nonsecure clusters. Ran some hdfs, MR and yarn tests. Seems good to me. On Mon, Oct 7, 2013 at 12:21 PM, Alejandro Abdelnur t...@cloudera.comwrote: +1 * downloaded source tarball * verified MD5 * verified signature * verified CHANGES.txt files, release # and date * run 'mvn apache-rat:check' successfully * built distribution * setup speudo cluster * started HDFS/YARN * run some HTTFS tests * run a couple of MR examples * run a few tests using Llama AM On Mon, Oct 7, 2013 at 12:07 PM, Tassapol Athiapinya tathiapi...@hortonworks.com wrote: +1 for the release. I have deployed a multinode cluster and extensively tested MR speculative execution, YARN CLI and YARN distributed shell. There were couple of issues I encountered while testing MAPREDUCE-5533, YARN-1168, YARN-1167, YARN-1157, YARN-1131, YARN-1118, YARN-1117 and all of them have been fixed. Thanks, Tassapol On Oct 7, 2013, at 12:00 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.2.0 that I would like to get released - this release fixes a small number of bugs and some protocol/api issues which should ensure they are now stable and will not change in hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.2.0-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun P.S.: Thanks to Colin, Andrew, Daryn, Chris and others for helping nail down the symlinks-related issues. I'll release note the fact that we have disabled it in 2.2. Also, thanks to Vinod for some heavy-lifting on the YARN side in the last couple of weeks. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Alejandro -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
-1. Some of the cli and distcp system tests which use hftp:// and webhdfs:// are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is a regression and we need to make sure they work before we call a release. On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released. This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including: # HDFS Snapshots # Windows support # YARN API stabilization # MapReduce Binary Compatibility with hadoop-1.x # Substantial amount of integration testing with rest of projects in the ecosystem The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: Heads up: branch-2.1-beta
We have started testing branch-2.1-beta and for most parts the code looks very stable. We have deployed both secure and non-secure multinode clusters. We had some minor hiccups with some of our e2e tests breaking due to additional setsid info being logged by the bin scripts and errors while building with snappy turned on. Both are now fixed. Some of the e2e tests had to be tweaked due to configuration changes in branch-2.1-beta such as yarn.nodemanager.delete.debug-delay-sec has to be set to 0 to invoke the cleanup of container tmp files, fs.trash.interval is now a server side config etc. We are now seeing failures while configuring short circuit read and job summary logs(most likely configuration issues) and tests which use hftp:// and webhdfs:// are failing on secure clusters. Will file jiras after initial investigation. Thanks, Ramya On Tue, Jun 4, 2013 at 8:32 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, The vast majority of of the planned features and API work is complete, thanks to everyone who contributed! I've created a branch-2.1-beta branch from which I anticipate I can make the first of our beta releases very shortly. For now the remaining work is to wrap up loose ends i.e. last minute api work (e.g. YARN-759 showed up last night for consideration), bug-fixes etc.; then run this through a battery of unit/system/integration tests and do a final review before we ship. There is more work remaining on documentation (e.g. HADOOP-9517) and I plan to personally focus on it this week - obviously help reviewing docs is very welcome. Committers, from now, please please exercise your judgement on where you commit. Typically, features should go into branch-2 with 2.3.0 as the version on jira (fix-version 2.3.0 is ready). The expectation is that 2.2.0 will be limited to content in branch-2.1-beta and we stick to stabilizing it henceforth (I've deliberately not created 2.2.0 fix-version on jira yet). thanks, Arun
Re: [Vote] Merge branch-trunk-win to trunk
+1 for the merge. As someone who has been testing the code for many months now, both on singlenode and multinode clusters, I am very confident about the stability and the quality of the code. I have run several regression tests to verify distributed cache, streaming, compression, capacity scheduler, job history and many more features in HDFS and MR. - Ramya On Thu, Feb 28, 2013 at 3:08 PM, sanjay Radia san...@hortonworks.comwrote: +1 Java has done the bulk of the work in making Hadoop multi-platform. Windows specific code is a tiny percentage of the code. Jeninks support for windows is going help us keep the platform portable going forward. I expect that the vast majority of new commits have no problems. I propose that we start by fixing problems that Jenkins raises but not block new commits for too long if the author does not have a windows box or if a volunteer does not step up. sanjay
[jira] [Created] (HADOOP-8583) Globbing is not correctly handled in a few cases on Windows
Ramya Sunil created HADOOP-8583:
---
Summary: Globbing is not correctly handled in a few cases on
Windows
Key: HADOOP-8583
URL: https://issues.apache.org/jira/browse/HADOOP-8583
Project: Hadoop Common
Issue Type: Bug
Environment: Windows
Reporter: Ramya Sunil
Glob handling fails in a few cases on a Windows environment.
For example:
{noformat}
c:\ hadoop dfs -ls /
Found 2 items
drwxrwxrwx - Administrator supergroup 0 2012-07-06 15:00 /tmp
drwxr-xr-x - Administrator supergroup 0 2012-07-06 18:52 /user
c:\ hadoop dfs -ls /tmpInvalid*
Found 2 items
drwxr-xr-x - Administrator supergroup 0 2012-07-10 18:50
/user/Administrator/sortInputDir
drwxr-xr-x - Administrator supergroup 0 2012-07-10 18:50
/user/Administrator/sortOutputDir
c:\ hadoop dfs -rmr /tmp/*
Usage: java FsShell [-rmr [-skipTrash] src ]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-7941) NoClassDefFoundError while running distcp/archive
NoClassDefFoundError while running distcp/archive
-
Key: HADOOP-7941
URL: https://issues.apache.org/jira/browse/HADOOP-7941
Project: Hadoop Common
Issue Type: Bug
Components: build
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Fix For: 0.23.1
bin/hadoop distcp
{noformat}
Exception in thread main java.lang.NoClassDefFoundError:
org/apache/hadoop/tools/DistCp
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.tools.DistCp
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: org.apache.hadoop.tools.DistCp. Program will
exit.
{noformat}
Same is the case while running 'bin/hadoop archive'
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-7815) Map memory mb is being incorrectly set by hadoop-setup-conf.sh
Map memory mb is being incorrectly set by hadoop-setup-conf.sh -- Key: HADOOP-7815 URL: https://issues.apache.org/jira/browse/HADOOP-7815 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 0.20.205.1, 0.24.0 Reporter: Ramya Sunil Assignee: Ramya Sunil Priority: Minor Fix For: 0.20.205.1, 0.23.0, 0.24.0 HADOOP-7728 enabled task memory management to be configurable in the hadoop-setup-conf.sh. However, the default value for mapred.job.map.memory.mb is being set incorrectly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] 0.20.205.0 Release Candidate 2
+1 (non binding) Deployed on a single node and a 20node cluster with security turned on and off. Ran Hadoop benchmarks, a few end-end tests and functional tests covering distributed cache, capacity scheduler, decommissioning etc Also ran end-end tests on Pig 0.9.x and a few system tests on Hbase 0.90.x and Hive 0.7.x. Works convincingly fine. - Ramya On Thu, Oct 13, 2011 at 7:03 PM, Eric Yang eric...@gmail.com wrote: +1 (non-binding) Exercised Hadoop 0.20.205 with HBase 0.90.4+, Pig 0.9.1+, Chukwa trunk on 20 nodes clusters, works fine. regards, Eric On Oct 13, 2011, at 5:46 PM, Aaron T. Myers wrote: +1 (non-binding) I downloaded the tar ball and deployed on a 6 node cluster with security disabled. I put a few files and ran a few MR jobs, mostly to calculate the value of Pi. All worked fine. Curiously, the estimated value of Pi was once determined to be 4., but I suspect this is a function of the parameters I gave the Pi job, and not of any deficiency of the RC of 0.20.205. :) -- Aaron T. Myers Software Engineer, Cloudera On Fri, Oct 7, 2011 at 5:10 PM, Matt Foley ma...@apache.org wrote: Many thanks to the community members who tried out RC1, and found several critical or blocker bugs. These have been resolved and I have now posted RC2 at http://people.apache.org/~mattf/hadoop-0.20.205.0-rc2/ A copy of the Release Notes are provided at the top level of that directory. Appropriate artifacts have also been pushed to the Maven staging repo. Please download and evaluate it, then vote on accepting it as a public release. Voting will close at 1am GMT on Saturday 15 October (6pm PDT, Fri 14 Oct). Best regards, --Matt RM for 0.20.205.0
[jira] [Created] (HADOOP-7728) hadoop-setup-conf.sh should be modified to enable task memory manager
hadoop-setup-conf.sh should be modified to enable task memory manager - Key: HADOOP-7728 URL: https://issues.apache.org/jira/browse/HADOOP-7728 Project: Hadoop Common Issue Type: Bug Components: conf Affects Versions: 0.20.205.1 Reporter: Ramya Sunil Assignee: Ramya Sunil The hadoop-setup-conf.sh should be modified such that task memory management is enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
