Re: [VOTE] - Release 2.0.5-beta

2013-05-22 Thread Mayank Bansal
Hi Guys, +1 We @ ebay would like to see snapshots before we start testing/deploying hadoop 2.0 next month. Thanks, Mayank On Wed, May 15, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, A considerable number of people have expressed confusion regarding the recent vote

Re: [VOTE] - Release 2.0.5-beta

2013-05-21 Thread sanjay Radia
+1 on 2.0.5 defined in this thread with the new features. But I am supportive of an earlier release that has ALL the compatibility changes, without the features. sanjay On May 15, 2013, at 10:57 AM, Arun C Murthy wrote: Folks, ... I propose we continue the original plan and make a

Re: [VOTE] - Release 2.0.5-beta

2013-05-21 Thread Konstantin Shvachko
-1 for the record. This is a great plan for 2.1, which I would gladly support, but not for 2.0.5. I do not see how the previous vote could have been confusing, as it contained a direct quotation of the relative clause of Bylaws. Arun, the format of this vote remains confusing. What is the

Re: [VOTE] - Release 2.0.5-beta

2013-05-21 Thread Konstantin Shvachko
Chris, I find you are contradicting yourself within this message and with some other of yours. But I want to address only one thing here This has exposed a bug in our bylaws, which we can fix. This could be a bug, and we may need to fix it. But until then it is a bylaw, which is the only rule

Re: [VOTE] - Release 2.0.5-beta

2013-05-21 Thread Matt Foley
I've now started a separate discussion thread in common-dev@, titled [PROPOSAL] change in bylaws to remove Release Plan vote. If it achieves consensus, I'll put it to a vote to so change the bylaws. Best, --Matt On Sat, May 18, 2013 at 4:22 PM, Chris Douglas cdoug...@apache.org wrote: The

Re: [VOTE] - Release 2.0.5-beta

2013-05-18 Thread Chris Douglas
The release plan vote is not binding in any way. Nobody lost a vote, or risks having an outcome reversed, because there are no consequences to these exercises. Konstantin, I've been trying to tell you for more than a week that you can go forward without anyone's blessing or consent. There are no

Re: [VOTE] - Release 2.0.5-beta

2013-05-17 Thread Vinod Kumar Vavilapalli
Thanks a bunch Nathan, for clearly letting us know the Yahoo! team's perspective. We are getting started on rolling upgrades from YARN side (Sid opened YARN-666) and I hear HDFS side is too. We definitely need compatibility and testing kits. Have to get started on this. Work-preserving

Re: [VOTE] - Release 2.0.5-beta

2013-05-17 Thread Roman Shaposhnik
Apologies for a bunch of delayed responses (and as such adding even more emails to this thread). On Wed, May 15, 2013 at 4:47 PM, Arun C Murthy a...@hortonworks.com wrote: My reading of your response is that while you appreciate the feedback Bigtop is providing you're not of an opinion that

Re: [VOTE] - Release 2.0.5-beta

2013-05-17 Thread Konstantin Shvachko
BCC: general@ Since we recognize now that this is a vote to overrule previous decision, I am referring to Vinod's note on general *http://s.apache.org/h7x* should this be brought to the attention of the Board? I don't remember any precedents of this kind in Hadoop history. But other projects may

Re: [VOTE] - Release 2.0.5-beta

2013-05-17 Thread Doug Cutting
On Wed, May 15, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com wrote: To get past all of this confusion, I'd like to present an alternate, specific proposal for consideration. I propose we continue the original plan and make a 2.0.5-beta release by May end with the following content:

Re: [VOTE] - Release 2.0.5-beta

2013-05-17 Thread Roman Shaposhnik
Guys, this is a pretty long email with all the details I can think of on how Bigtop can help stabilization efforts of Hadoop 2.x. A lot of this information is required background. I really, really encourage everyone who's thinking of contributing to this effort to read it up. Once again, I do

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Konstantin Boudnik
Guys, I guess what you're missing is that Bigtop isn't a testing framework for Hadoop. It is stack framework that verifies that components are dealing with each other nicely. Every single stack is different: Bigtop 0.5.0 differs from 0.6.0, and so on. Bigtop - as any other ASF project - has its

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Vinod Kumar Vavilapalli
On May 15, 2013, at 2:54 PM, Roman Shaposhnik wrote: This is not my argument at all. I apologize if somehow I failed to communicate it, but here's what my argument boils down to: given *my* experience with Hadoop 2.0.x series and Bigtop release every time I try a different release of Hadoop

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Konstantin Boudnik
On Wed, May 15, 2013 at 10:52PM, Suresh Srinivas wrote: Assuming that you are talking about HDFS features when you say features going into a beta on a very short short timetable and laundry list etc, No, that would not be a correct assumption. So these features are not

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Robert Evans
-0 (Binding) I have made my opinion known in the previous thread/vote, but I have spent enough time discussing this and need to get back to my day job. If the community is able to get snapshots and everything else in this list merged and stable without breaking the stack above it in two weeks it

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Steve Loughran
On 15 May 2013 23:19, Konstantin Boudnik c...@apache.org wrote: Guys, I guess what you're missing is that Bigtop isn't a testing framework for Hadoop. It is stack framework that verifies that components are dealing with each other nicely. which to me means Some form of integration test

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Arun C Murthy
Cos, On May 15, 2013, at 11:38 PM, Konstantin Boudnik wrote: What I am seeing times and again in these endless discussion threads is this: a) downstream or bigtop: we are seeing a bunch of integration issues with every new feature introduced/something even a commit made b) feature

Re: [VOTE] - Release 2.0.5-beta

2013-05-16 Thread Nathan Roberts
(initially respond on general@, sorry about that. copied here) +1 (non-binding) From my perspective: * The key feature that will drive me to adopt 2.x is Rolling Upgrades * In order to get to rolling upgrades, we need a compatibility story that is significantly better than we have today ** We

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Suresh Srinivas
This is the course that we were taking before the unfortunate disruption. We should be able to meet both the stabilization goals and compatibility goals quickly with this proposal. I personally am willing to invest a lot of time in testing, code reviews and work on adding missing functionality to

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Vinod Kumar Vavilapalli
Seems like you forgot to bcc. Forwarding this to general. Thanks, +Vinod On May 15, 2013, at 10:57 AM, Arun C Murthy wrote: Folks, A considerable number of people have expressed confusion regarding the recent vote on 2.0.5, beta status etc. given lack of specifics, the voting itself

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Amir Sanjar
good, glad we are back on track again. BTW, we have already started build (IBM and OpenJDK SDK), unit test, and limited integration testing on x86 and POWER, results are promising. Best Regards Amir Sanjar System Management Architect PowerLinux Open Source Hadoop development lead IBM Senior

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Karthik Kambatla
Hi Arun, Can we add HADOOP-9517 to the list - having compatibility guidelines should help us support users and downstream projects better? Thanks Karthik On Wed, May 15, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, A considerable number of people have expressed

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Alejandro Abdelnur
Do we need to add YARN-397? Thanks. On Wed, May 15, 2013 at 11:23 AM, Karthik Kambatla ka...@cloudera.comwrote: Hi Arun, Can we add HADOOP-9517 to the list - having compatibility guidelines should help us support users and downstream projects better? Thanks Karthik On Wed, May 15,

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Vinod Kumar Vavilapalli
Thanks for laying out a very specific release plan, easy to vote on. I am watching most of YARN and MAPREDUCE changes, glad that those are called out specifically. Apart from that, we have - RM restart which is mostly already committed but needs a couple more in - a couple of scheduling

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Vinod Kumar Vavilapalli
- RM restart which is mostly already committed but needs a couple more in - a couple of scheduling related APIs which fall under the protocol changes you mentioned, that are close to commit - a couple of security issues which aren't exactly features. I should have been clearer: - RM

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Arun C Murthy
Yes to all. As long as we are making timely and compatible progress, we don't need to debate individual issues here. Let's continue discussion on relevant jiras. thanks, Arun On May 15, 2013, at 12:11 PM, Vinod Kumar Vavilapalli wrote: - RM restart which is mostly already committed but

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Vinod Kumar Vavilapalli
I also feel that some of YARN-397 should go in. If you also feel so, please put in a +1 to state your intention. Thanks, +Vinod On May 15, 2013, at 11:32 AM, Alejandro Abdelnur wrote: Do we need to add YARN-397? Thanks. On Wed, May 15, 2013 at 11:23 AM, Karthik Kambatla

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread eric baldeschwieler
+1 On May 15, 2013, at 10:57 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, A considerable number of people have expressed confusion regarding the recent vote on 2.0.5, beta status etc. given lack of specifics, the voting itself (validity of the vote itself, whose votes are

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Matt Foley
[mailto:vino...@hortonworks.com] Sent: Wednesday, May 15, 2013 12:20 PM To: common-dev@hadoop.apache.org Subject: Re: [VOTE] - Release 2.0.5-beta I also feel that some of YARN-397 should go in. If you also feel so, please put in a +1 to state your intention. Thanks, +Vinod On May 15

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Sandy Ryza
+1 (non-binding) Agreed with Bikas that we should get the scheduler API enhancements (YARN-397) in we are able, but they don't need to be blockers because they will be backwards compatible. Arun, not sure whether your Yes to all already covered this, but I'd like to throw in support for the

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Matt Foley
Arun, not sure whether your Yes to all already covered this, but I'd like to throw in support for the compatibility guidelines being a blocker. +1 to that. Definitely an overriding concern for me. On Wed, May 15, 2013 at 1:25 PM, Sandy Ryza sandy.r...@cloudera.com wrote: +1 (non-binding)

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Matt Foley
lets fork this thread into the appropriate ML and discuss the practical, achievable steps that can be included into the release criteria of Hadoop 2.0.5-beta Seems to me common-dev is the appropriate ML, and Arun has invited Jiras to include. Open a Jira with your suggested list, and we carry

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Devaraj Das
+1 (binding) on the proposal. 2-3 weeks doesn't sound too long a time, and we have many committers willing to be on-call to fix issues when they are discovered. On Wed, May 15, 2013 at 10:57 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, A considerable number of people have expressed

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Eli Collins
On Wed, May 15, 2013 at 1:29 PM, Matt Foley mfo...@hortonworks.com wrote: Arun, not sure whether your Yes to all already covered this, but I'd like to throw in support for the compatibility guidelines being a blocker. +1 to that. Definitely an overriding concern for me. +1 Likewise.

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Zhijie Shen
+1 (non-binding) on the proposal. On Wed, May 15, 2013 at 1:43 PM, Eli Collins e...@cloudera.com wrote: On Wed, May 15, 2013 at 1:29 PM, Matt Foley mfo...@hortonworks.com wrote: Arun, not sure whether your Yes to all already covered this, but I'd like to throw in support for the

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Roman Shaposhnik
On Wed, May 15, 2013 at 1:36 PM, Matt Foley mfo...@hortonworks.com wrote: lets fork this thread into the appropriate ML and discuss the practical, achievable steps that can be included into the release criteria of Hadoop 2.0.5-beta Seems to me common-dev is the appropriate ML, Thanks. I'll

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Vinod Kumar Vavilapalli
Roman, I keep this same argument again and again. Should've refuted earlier. Please list down all the issues that BigTop ran into *because of* new features. You continue to argue that new features are destabilizing 2.0.*, which I don't agree with at all. 2.0.3-alpha was the last time major

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Vinod Kumar Vavilapalli
Typo, keep hearing* Thanks, +Vinod On May 15, 2013, at 2:14 PM, Vinod Kumar Vavilapalli wrote: Roman, I keep this same argument again and again. Should've refuted earlier.

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Arun C Murthy
Great summary, thanks Vinod. On May 15, 2013, at 2:14 PM, Vinod Kumar Vavilapalli wrote: Roman, I keep this same argument again and again. Should've refuted earlier. Please list down all the issues that BigTop ran into *because of* new features. You continue to argue that new features

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Steve Loughran
On 15 May 2013 10:57, Arun C Murthy a...@hortonworks.com wrote: Folks, A considerable number of people have expressed confusion regarding the recent vote on 2.0.5, beta status etc. given lack of specifics, the voting itself (validity of the vote itself, whose votes are binding) etc. IMHO

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Roman Shaposhnik
On Wed, May 15, 2013 at 2:14 PM, Vinod Kumar Vavilapalli vino...@hortonworks.com wrote: Please list down all the issues that BigTop ran into *because of* new features. Whether the bug is *because of* new feature or not is a red herring for my argument. Please lets drop this distinction. I

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Arun C Murthy
Roman, Furthermore, before we rush into finding flaws and scaring kids at night it would be useful to remember one thing: Software has *bugs*. We can't block any release till the entire universe validates it, in fact they won't validate it if we don't release since are at the bottom of the

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Chris Douglas
+1 (binding) on the proposal. However, the value we get from these release plan votes is dubious, to put it mildly. The surrounding discussion has cost more than it is worth, and votes on executive summaries of releases discourage the sort of detailed collaboration we're trying to create. It

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Steve Loughran
On 15 May 2013 15:02, Arun C Murthy a...@hortonworks.com wrote: Roman, Furthermore, before we rush into finding flaws and scaring kids at night it would be useful to remember one thing: Software has *bugs*. We can't block any release till the entire universe validates it, in fact they won't

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Roman Shaposhnik
Arun, am I reading yours answer to my binary question correctly? It is a 'no'. My reading of your response is that while you appreciate the feedback Bigtop is providing you're not of an opinion that investigating the level of stability of Hadoop wrt. downstream any further than what is currently

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Konstantin Boudnik
Indeed. I think the root of the issue is deeper. ASF software practices are great to deal with isolated, relatively contained projects like httpd, libreoffice, trac, etc. However, Hadoop based stack - essentially, software aimed at enterprises with bigger scale operations - is a different animal,

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Arun C Murthy
On May 15, 2013, at 3:27 PM, Chris Douglas wrote: +1 (binding) on the proposal. However, the value we get from these release plan votes is dubious, to put it mildly. The surrounding discussion has cost more than it is worth, and votes on executive summaries of releases discourage the sort

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Arun C Murthy
On May 15, 2013, at 3:50 PM, Roman Shaposhnik wrote: Arun, am I reading yours answer to my binary question correctly? It is a 'no'. No. My reading of your response is that while you appreciate the feedback Bigtop is providing you're not of an opinion that investigating the level of

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Matt Foley
Roman, what is your model for how test results from Bigtop should feed back into Hadoop-2 development? With the understanding that (a) software does have bugs, and (b) you're not going to get an SLA on community-sponsored software, what are your ideas for how to close the loop better? Would CI

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Matt Foley
I'm actually drafting such a proposal. Will open the discussion as a [PROPOSAL] in general@ --Matt On Wed, May 15, 2013 at 4:44 PM, Arun C Murthy a...@hortonworks.com wrote: On May 15, 2013, at 3:27 PM, Chris Douglas wrote: +1 (binding) on the proposal. However, the value we get from

Re: [VOTE] - Release 2.0.5-beta

2013-05-15 Thread Suresh Srinivas
On Wed, May 15, 2013 at 9:25 PM, Andrew Purtell apurt...@apache.org wrote: The other thread or vote or whatever at least served the purpose in fresh surfacing of concerns. Talk of new features going in to a beta on a very short short timetable is concerning for anyone with experience working