Re: [DISCUSS] Looking to a 2.8.0 release

> > > maybe discuss having a list @ release time. As an example, s3 and > encryption at rest shipped in beta stage... what's in 2.8 that "we don't > yet trust ourselves?". Me, I'd put erasure coding in there just because > I've no familiarity with it > > Quick clarification, EC isn't scheduled

Re: [DISCUSS] Looking to a 2.8.0 release

> On 25 Nov 2015, at 22:01, Vinod Kumar Vavilapalli wrote: > > Tx for your comments, Andrew! > > I did talk about it in a few discussions in the past related to this but yes, > we never codified the feature-level alpha/beta tags. Part of the reason why I > never pushed

Re: [DISCUSS] Looking to a 2.8.0 release

> On Nov 25, 2015, at 11:23 AM, Vinod Kumar Vavilapalli > wrote: > > There are 40 odd incompatible changes in 3.x: >

Re: [DISCUSS] Looking to a 2.8.0 release

This is the current state from the feedback I gathered. - Support priorities across applications within the same queue YARN-1963 — Can push as an alpha / beta feature per Sunil - YARN-1197 Support changing resources of an allocated container: — Can push as an alpha/beta feature per

Re: [DISCUSS] Looking to a 2.8.0 release

Hey Vinod, I'm fine with the idea of alpha/beta marking in the abstract, but had a question: do we define these terms in our compatibility policy or elsewhere? I think it's commonly understood among us developers (alpha means not fully tested and API unstable, beta means it's not fully tested but

Re: [DISCUSS] Looking to a 2.8.0 release

There are 40 odd incompatible changes in 3.x:

Re: [DISCUSS] Looking to a 2.8.0 release

I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap I plan to create a 2.8 branch EOD today. The goal for all of us should be to restrict improvements & fixes to only (a) the

Re: [DISCUSS] Looking to a 2.8.0 release

Hi Vinod, Thanks for driving this. Can you add YARN-2573 which includes the work done to integrate ReservationSystem with the RM failover mechanism to your list. This can be reviewed and committed (branch-2) also about a month back. Cheers, Subru On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar

Re: [DISCUSS] Looking to a 2.8.0 release

Okay, tx for this clarification Chris! I dug more into this and now realized the actual scope of this. Given the the limited nature of this feature (non-Namenode etc) and the WIP nature of the larger umbrella HADOOP-11744, we will ship the feature but I’ll stop calling this out as a notable

Re: [DISCUSS] Looking to a 2.8.0 release

Tx for your comments, Andrew! I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors

Re: [DISCUSS] Looking to a 2.8.0 release

+1. Thanks, Vinod. --Chris Nauroth On 11/25/15, 1:45 PM, "Vinod Kumar Vavilapalli" wrote: >Okay, tx for this clarification Chris! I dug more into this and now >realized the actual scope of this. Given the the limited nature of this >feature (non-Namenode etc) and the

Re: [DISCUSS] Looking to a 2.8.0 release

SGTM, thanks Vinod! LMK if you need reviews on any of that. Regarding the release checklist, another item I'd add is updating the release notes in the project documentation, we've forgotten in the past. On Wed, Nov 25, 2015 at 2:01 PM, Vinod Kumar Vavilapalli wrote: > Tx

Re: [DISCUSS] Looking to a 2.8.0 release

Hi Vinod, The HDFS-8155 work is complete in branch-2 already, so feel free to include it in the roadmap. For those watching the thread that aren't familiar with HDFS-8155, I want to call out that it was a client-side change only. The WebHDFS client is capable of obtaining OAuth2 tokens and

Re: [DISCUSS] Looking to a 2.8.0 release

I reviewed the current state of the YARN-2928 changes regarding its impact if the timeline service v.2 is disabled. It does appear that there are a lot of things that still do get created and enabled unconditionally regardless of configuration. While this is understandable when we were working to

Re: [DISCUSS] Looking to a 2.8.0 release

I am really against the notion of calling x.y.0 releases alpha/beta; it is very confusing. If we think a release is alpha/beta quality, why not release it as x.y.0-alpha or x.y.0-beta, and follow it up eventually with x.y.0 GA. I am in favor of labeling any of the newer features to be of

Re: [DISCUSS] Looking to a 2.8.0 release

Did we consider cutting a branch-3 that borrows relatively compatible patches from trunk to run the 3.x line? That said, I would like for us to really tighten our compatibility policies and actually stick to them starting the next major release. On Wed, Nov 11, 2015 at 1:11 PM, Vinod Vavilapalli

Re: [DISCUSS] Looking to a 2.8.0 release

On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli wrote: > — YARN Timeline Service Next generation: YARN-2928: Lots of momentum, > but clearly a work in progress. Two options here > — If it is safe to ship it into 2.8 in a disable manner, we can > get the

Re: [DISCUSS] Looking to a 2.8.0 release

Agreed on not mixing this with major release discussions. Okay, I just finished my review of 2.8 content. A quick summary follows. Current state of originally planned items - Nearly Done / Done and so need to close down quickly — Support *both* JDK7 and JDK8 runtimes HADOOP-11090 —

Re: [DISCUSS] Looking to a 2.8.0 release

I’ll let others comment on specific features. Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release. Therefore, as I was proposing before, we should

Fwd: [DISCUSS] Looking to a 2.8.0 release

Started a thread on 2.8.0. Please give feedback etc. I’ll need help from both the teams to get this done right. +Vinod Begin forwarded message: From: Vinod Kumar Vavilapalli <vino...@hortonworks.com<mailto:vino...@hortonworks.com>> Subject: Re: [DISCUSS] Looking to a 2.8.0

Re: [DISCUSS] Looking to a 2.8.0 release

Thanks to Vinod for starting this discussion! +1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested. I'd prefer to push it as an Alpha feature before wilder testing. And also agree to call first release of 2.8 as an Alpha release according to the number of new features /

Re: [DISCUSS] Looking to a 2.8.0 release

teams to get this done right. +Vinod Begin forwarded message: From: Vinod Kumar Vavilapalli <vino...@hortonworks.com<mailto:vino...@hortonworks.com>> Subject: Re: [DISCUSS] Looking to a 2.8.0 release Date: November 11, 2015 at 12:13:54 PM PST To: Hadoop Common <common-...@hado

Re: [DISCUSS] Looking to a 2.8.0 release

> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli > wrote: > >— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement > - no dimension of alpha/betaness here. IMO: this feels like a massive break in backwards compatibility. Anyone who

Re: [DISCUSS] Looking to a 2.8.0 release

> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli > wrote: > > I’ll let others comment on specific features. > > Regarding the 3.x vs 2.x point, as I noted before on other threads, given all > the incompatibilities in trunk it will be ways off before users can run

Re: [DISCUSS] Looking to a 2.8.0 release

I think it makes sense to have a 2.8 release since there are a tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x release seems like something we should consider separately since it would not have the same compatibility guarantees as a 2.8 release. There's a pretty big delta

Re: [DISCUSS] Looking to a 2.8.0 release

As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the unusually long 2.6.1 release. With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8. I’ll do a quick survey of where the

Re: [DISCUSS] Looking to a 2.8.0 release

With two active sustaining branches (2.6, 2.7), what would you think of releasing trunk as 3.x instead of pushing 2.8? There are many new features (EC, Y1197, etc.), and trunk could be the source of several alpha/beta releases before we fork the 3.x line. -C On Sat, Sep 26, 2015 at 12:49 PM,

Re: [DISCUSS] Looking to a 2.8.0 release

-...@hadoop.apache.org Cc: vino...@apache.org Subject: [DISCUSS] Looking to a 2.8.0 release With 2.7.0 out of the way, and with more maintenance releases to stabilize it, I propose we start thinking about 2.8.0. Here's my first cut of the proposal, will update the Roadmap wiki. - Support *both* JDK7

[DISCUSS] Looking to a 2.8.0 release

With 2.7.0 out of the way, and with more maintenance releases to stabilize it, I propose we start thinking about 2.8.0. Here's my first cut of the proposal, will update the Roadmap wiki. - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090 - Compatibility tools to catch backwards, forwards

Re: [DISCUSS] Looking to a 2.8.0 release

I would also like to support Karthik's proposal on the release thread about version numbering. 2.7.0 being alpha is pretty confusing since all of the other 2.x releases since GA have been stable. I think users would prefer a version like 2.8.0-alpha1 instead, which is very clear and similar to

Re: [DISCUSS] Looking to a 2.8.0 release

Thanks for your comment Karthik. Here's my take. Feature based release is going to put us back into the same prolonged release cycle. That is why I am proposing that we look at 2-3 months time and get whatever is ready. If we don't have even a single feature in by then, clearly we can drop the

Re: [DISCUSS] Looking to a 2.8.0 release

The feature set here seems pretty long, even for 2 - 3 months. Can we come up with a minimum set of features (or a number of features) that justify a new minor release, and start stabilizing as soon as those are in? On Tue, Apr 21, 2015 at 2:39 PM, Vinod Kumar Vavilapalli vino...@apache.org