Re: Heads up: branch-2.1-beta

2013-06-26 Thread Arun C Murthy
Ok, the last of the blockers is done. I'll roll an RC tonight. thanks, Arun On Jun 20, 2013, at 2:40 PM, Arun C Murthy a...@hortonworks.com wrote: I think I've shared this before, but here you go again… http://s.apache.org/hadoop-2.1.0-beta-blockers At this point, HADOOP-9421 seems like

Re: Heads up: branch-2.1-beta

2013-06-26 Thread Alejandro Abdelnur
Have anybody from the HDFS side of things had a chance to look at the OOM that Roman was reporting? Thanks. On Wed, Jun 26, 2013 at 12:26 AM, Arun C Murthy a...@hortonworks.com wrote: Ok, the last of the blockers is done. I'll roll an RC tonight. thanks, Arun On Jun 20, 2013, at 2:40

Re: Heads up: branch-2.1-beta

2013-06-20 Thread Alejandro Abdelnur
Arun, It seems there are still a few things to iron out before getting 2.1 out of the door. As RM for the release, would you mind sharing the current state of things and your estimate on when it could happen? Thanks. On Wed, Jun 19, 2013 at 4:29 PM, Roman Shaposhnik r...@apache.org wrote:

Re: Heads up: branch-2.1-beta

2013-06-20 Thread Arun C Murthy
I think I've shared this before, but here you go again… http://s.apache.org/hadoop-2.1.0-beta-blockers At this point, HADOOP-9421 seems like the most important. thanks, Arun On Jun 20, 2013, at 8:31 AM, Alejandro Abdelnur t...@cloudera.com wrote: Arun, It seems there are still a few

Re: Heads up: branch-2.1-beta

2013-06-19 Thread Arun C Murthy
Ping. Any luck? On Jun 17, 2013, at 4:06 PM, Roman Shaposhnik r...@apache.org wrote: On Sun, Jun 16, 2013 at 5:14 PM, Arun C Murthy a...@hortonworks.com wrote: Roman, Is there a chance you can run the tests with the full stack built against branch-2.1-beta and help us know where we are?

Re: Heads up: branch-2.1-beta

2013-06-19 Thread Steve Loughran
Arun, What do I do if I have I minor patch that I'd like to get out into 2.1, but where I don't want to introduce instability into that 2.1 branch if you are trying to make a beta release? For example, HADOOP-9651https://issues.apache.org/jira/browse/HADOOP-9651 changes the exception that

RE: Heads up: branch-2.1-beta

2013-06-19 Thread Devaraj k
; yarn-...@hadoop.apache.org Subject: Re: Heads up: branch-2.1-beta Ping. Any luck? On Jun 17, 2013, at 4:06 PM, Roman Shaposhnik r...@apache.org wrote: On Sun, Jun 16, 2013 at 5:14 PM, Arun C Murthy a...@hortonworks.com wrote: Roman, Is there a chance you can run the tests with the full

Re: Heads up: branch-2.1-beta

2013-06-19 Thread Roman Shaposhnik
On Tue, Jun 18, 2013 at 11:58 PM, Arun C Murthy a...@hortonworks.com wrote: Ping. Any luck? The full bigtop build failed because of Giraph: http://bigtop01.cloudera.org:8080/view/Upstream-tests/job/Hadoop-2.1.0/label=centos5/4/console Now, I'm pretty sure that's an issue with Giraph

Re: Heads up: branch-2.1-beta

2013-06-19 Thread Roman Shaposhnik
On Tue, Jun 18, 2013 at 11:58 PM, Arun C Murthy a...@hortonworks.com wrote: Ping. Any luck? Unfortunately I've ran into: https://issues.apache.org/jira/browse/HADOOP-9654 which was caused by the unrelated memory pressure on the NN, but it had an unfortunate side effect of making the rest of

Re: Heads up: branch-2.1-beta

2013-06-17 Thread Roman Shaposhnik
Hi Arun, thanks for chiming in! On Sun, Jun 16, 2013 at 4:50 PM, Arun C Murthy a...@hortonworks.com wrote: But there's a bit of bad news too (or at least the news that need to be triaged). At this point I don't know whether the Hadoop code is to blame or the tests/components themselves --

Re: Heads up: branch-2.1-beta

2013-06-17 Thread Roman Shaposhnik
On Sun, Jun 16, 2013 at 5:14 PM, Arun C Murthy a...@hortonworks.com wrote: Roman, Is there a chance you can run the tests with the full stack built against branch-2.1-beta and help us know where we are? I will try to kick off the full build today. And deploy/test tomorrow. It is all pretty

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Arun C Murthy
Which one are you talking about Ralph? This doesn't show up on any blocker list. http://s.apache.org/hadoop-blocker-bugs Arun On Jun 15, 2013, at 4:44 PM, Ralph Castain wrote: Not trying to be a pain, but I am trying to get clarification. The protocol buffer support is still broken. Do you

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Arun C Murthy
On Jun 15, 2013, at 8:19 AM, Alejandro Abdelnur wrote: Of the JIRAs in my laundry list for 2.1 the ones I would really want in are YARN-752, MAPREDUCE-5171 YARN-787. I agree YARN-787 needs to go in ASAP is a blocker - I'm looking at it right now. I'm ok with YARN-752 MAPREDUCE-5171

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Arun C Murthy
Karthik, My apologies - I'll definitely spend time on this *today*. Arun On Jun 15, 2013, at 4:18 PM, Karthik Kambatla wrote: Re-posting here to the wider audience: HADOOP-9517 (Document Hadoop Compatibility): I believe I have incorporated all suggestions so far. It would be great if

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Alejandro Abdelnur
Thanks Arun, I'll take care of committing YARN-752 and MAPREDUCE-5171 around noon today (SUN noon PST). What is your take on YARN-791 MAPREDUCE-5130? On Sun, Jun 16, 2013 at 7:02 AM, Arun C Murthy a...@hortonworks.com wrote: On Jun 16, 2013, at 5:39 AM, Arun C Murthy wrote: On Jun 15,

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Ralph Castain
HDFS-4866 is marked as a blocker. A patch was submitted and applied, but it only fixes compilation. As I marked on the ticket, the results remain unusable due to linker issues. On Jun 16, 2013, at 5:33 AM, Arun C Murthy a...@hortonworks.com wrote: Which one are you talking about Ralph? This

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Arun C Murthy
Alejandro, Can you please take a look at MAPREDUCE-5327? This is related to YARN-787. thanks, Arun On Jun 16, 2013, at 8:56 AM, Alejandro Abdelnur wrote: Thanks Arun, I'll take care of committing YARN-752 and MAPREDUCE-5171 around noon today (SUN noon PST). What is your take on YARN-791

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Roman Shaposhnik
On Sat, Jun 15, 2013 at 8:19 AM, Alejandro Abdelnur t...@cloudera.com wrote: If the intention is to get the release out in time for the Hadoop Summit we have a very tight schedule. Because the release vote runs for 7 days, we should have an RC latest Monday afternoon, and we should encourage

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Alejandro Abdelnur
Arun, actually YARN-787 fixed the problem. Run those tests from trunk and branch-2.1-beta HEADs without issues. Without YARN-787 they fail. Thx On Sun, Jun 16, 2013 at 12:03 PM, Arun C Murthy a...@hortonworks.com wrote: Alejandro, Can you please take a look at MAPREDUCE-5327? This is

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Arun C Murthy
Responses inline: On Jun 16, 2013, at 1:04 PM, Roman Shaposhnik wrote: But there's a bit of bad news too (or at least the news that need to be triaged). At this point I don't know whether the Hadoop code is to blame or the tests/components themselves -- all I know is that these tests passed

Re: Heads up: branch-2.1-beta

2013-06-16 Thread Arun C Murthy
Roman, Is there a chance you can run the tests with the full stack built against branch-2.1-beta and help us know where we are? thanks, Arun On Jun 16, 2013, at 4:50 PM, Arun C Murthy wrote: Responses inline: On Jun 16, 2013, at 1:04 PM, Roman Shaposhnik wrote: But there's a bit of

Re: Heads up: branch-2.1-beta

2013-06-15 Thread Ralph Castain
Just curious of your procedures. Given that there is at least one blocker JIRA out there that has yet to be fully resolved, do you intend to release anyway? On Jun 15, 2013, at 8:19 AM, Alejandro Abdelnur t...@cloudera.com wrote: If the intention is to get the release out in time for the

Re: Heads up: branch-2.1-beta

2013-06-15 Thread Karthik Kambatla
Re-posting here to the wider audience: HADOOP-9517 (Document Hadoop Compatibility): I believe I have incorporated all suggestions so far. It would be great if people could take another look at it. I ll iterate fast on any comments so we get this in by the time rest of the code pieces are

Re: Heads up: branch-2.1-beta

2013-06-14 Thread Arun C Murthy
As Ramya noted, things are looking good on branch-2.1-beta ATM. Henceforth, can I please ask committers to hold off non-blocker fixes for the final set of tests? thanks, Arun On Jun 4, 2013, at 8:32 AM, Arun C Murthy wrote: Folks, The vast majority of of the planned features and API work

Re: Heads up: branch-2.1-beta

2013-06-14 Thread Alejandro Abdelnur
Arun, This sounds great. Following is the list of JIRAs I'd like to get in. Note that the are ready or almost ready, my estimate is that they can be taken care of in a couple of day. Thanks. * YARN-752: In AMRMClient, automatically add corresponding rack requests for requested nodes impact:

Re: Heads up: branch-2.1-beta

2013-06-14 Thread Roman Shaposhnik
On Thu, Jun 6, 2013 at 4:48 AM, Arun C Murthy a...@hortonworks.com wrote: On Jun 5, 2013, at 11:04 AM, Roman Shaposhnik wrote On the Bigtop side of things, once we have stable Bigtop 0.6.0 platform based on Hadoop 2.0.x codeline we plan to start running the same battery of integration tests

Re: Heads up: branch-2.1-beta

2013-06-12 Thread Arpit Gupta
We are running into an issue where link to the AM leads to a 500 https://issues.apache.org/jira/browse/YARN-800 We should fix this before we release 2.1.0 beta. -- Arpit Gupta Hortonworks Inc. http://hortonworks.com/ On Jun 10, 2013, at 4:49 AM, Arun C Murthy a...@hortonworks.com wrote: On

Re: Heads up: branch-2.1-beta

2013-06-10 Thread Arun C Murthy
On Jun 4, 2013, at 8:32 AM, Arun C Murthy wrote: Folks, The vast majority of of the planned features and API work is complete, thanks to everyone who contributed! I've created a branch-2.1-beta branch from which I anticipate I can make the first of our beta releases very shortly.

Re: Heads up: branch-2.1-beta

2013-06-10 Thread Ramya Sunil
We have started testing branch-2.1-beta and for most parts the code looks very stable. We have deployed both secure and non-secure multinode clusters. We had some minor hiccups with some of our e2e tests breaking due to additional setsid info being logged by the bin scripts and errors while

Re: Heads up: branch-2.1-beta

2013-06-10 Thread Roman Shaposhnik
On Thu, Jun 6, 2013 at 4:48 AM, Arun C Murthy a...@hortonworks.com wrote: On Jun 5, 2013, at 11:04 AM, Roman Shaposhnik wrote On the Bigtop side of things, once we have stable Bigtop 0.6.0 platform based on Hadoop 2.0.x codeline we plan to start running the same battery of integration tests

Re: Heads up: branch-2.1-beta

2013-06-10 Thread Roman Shaposhnik
On Mon, Jun 10, 2013 at 9:35 AM, Ramya Sunil ra...@hortonworks.com wrote: We had some minor hiccups with some of our e2e tests breaking due to additional setsid info being logged by the bin scripts and errors while building with snappy turned on. Any chance some (all?) of these tests can be

Re: Heads up: branch-2.1-beta

2013-06-06 Thread Arun C Murthy
On Jun 5, 2013, at 11:04 AM, Roman Shaposhnik wrote On the Bigtop side of things, once we have stable Bigtop 0.6.0 platform based on Hadoop 2.0.x codeline we plan to start running the same battery of integration tests on the branch-2.1-beta. We plan to simply file JIRAs if anything gets