Re: DISCUSS: How can we have less branches?

[Andrew Purtell]

I'm going to do a monthly cadence with 1.4. 

It's a bit delayed getting off the ground sadly because I need to address 
Francis' feedback and otherwise get RSGroups in before cutting the first RC, 
and I was recently out for two weeks on vacation. Soon, though. 

Or, we can start spinning minors off of branch-1 with the same cadence: 1.4.0, 
then 1.5.0, 1.6.0. Monthly or bi-monthly. Versions are cheap. Releasing minors 
would keep branch-1 in a good state and give us more flexibility for making 
changes (at the possible detriment of downstreamers we might break with LP 
changes, so hopefully we can be better about that). If we go this route I Will 
probably also maintain patch versions for 1.4 because our production will move 
up on it for a while. 

We will have to convince ourselves the issues with locking introduced in 1.3 
have settled before moving the stable pointer beyond 1.2. 


> On Sep 18, 2017, at 9:07 PM, Nick Dimiduk  wrote:
> 
> Reviving this thread. I know our interest is around branch-2 right now, but
> I think we are close to an actionable resolution here.
> 
> I'm a fan of Andrew's proposal of stabilizing our branch-1 for maintenance
> releases, getting away from all the minors if we can. Do we have a quorum
> of folks able and willing to do that work? Seems like it would center
> around the 1.4 release.
> 
> If branch-1.1 retires in the next 2-6 months, are we happy with branch-1.2,
> branch-1.3, or branch-1.4 carrying on the monthly cadence? Do we have a
> strong need to continue 1.x once 2.x is out? Are we calling 1.2 the LTS
> with quarterly patch releases, 1.3 aborted on arrival, and pushing folks
> onto 2.x? Do we want to settle on quarterly or bi-annual minor releases and
> only patch monthly or as necessary in between?
> 
> Thanks,
> Nick
> 
>> On Wed, Aug 9, 2017 at 8:00 AM, Mike Drob  wrote:
>> 
>> One thing that we need to be super careful about after releasing 2.0 is
>> that we don't accidentally introduce any incompatible changes. It's much
>> harder to reason about forwards compatibility than backwards compatibility
>> in my experience. So even though a hypothetical 1.5 would be released after
>> 2.0, users would still have the expectation that a 1.5->2.0 upgrade would
>> be smooth (unless we explicitly message that it isn't but that is a whole
>> new set of problems).
>> 
>> Also, to clarify some context here -- what is the difference between an LTS
>> branch and a Stable branch? Don't they convey the same thing to users? As a
>> relative newcomer, I feel like I missed an earlier conversation about this.
>> 
>> Mike
>> 
>>> On Tue, Aug 8, 2017 at 10:45 PM, Phil Yang  wrote:
>>> 
>>> Another option is no 1.5/branch-1 any more. What new features we are
>> going
>>> to have in HBase 1.5 (if it will be released)? Backporting from branch-2
>> to
>>> branch-1 is not easy, so maybe we will not have any big feature in
>> branch-1
>>> after releasing 1.4. And if we release 1.5 after releasing 2.0, it may
>>> confuse users. So IMO we can focus on branch-2 for new features.
>>> 
>>> I think it will be good if we have fixed logic for EOL branches, for
>>> example(just an example, can discuss):
>>> 1:Release a final version and EOL 1.x after we think 1.x+1 is stable.
>>> 2:Cut off new 1.x+2 branch after 1.x+1 is stable, don't cut too early.
>>> 
>>> Then we will not need discussing each time for each branch EOL and we
>> will
>>> have a fixed number of maintaining branches.
>>> 
>>> Thanks,
>>> Phil
>>> 
>>> 
>>> 2017-08-09 1:44 GMT+08:00 Andrew Purtell :
>>> 
 Well you are not wrong that branching was premature, it turns out. But
>>> I'll
 roll with it...
 
 
 On Tue, Aug 8, 2017 at 10:43 AM, Zach York <
>> zyork.contribut...@gmail.com
 
 wrote:
 
>> I made branch-1.4 a few weeks ago only.
> 
> Whoops, sorry for that! For some reason I thought I had seen it
>> months
 ago.
> 
> On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell >> 
> wrote:
> 
>> +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are
>> candidates. I
> think
>> 1.1 has the edge because it lacks locking changes introduced into
>>> 1.2,
> just
>> like 1.2 lacks locking changes introduced in 1.3 - the latter of
>>> which
> has
>> had far reaching consequences, and the former not an insignificant
 change
>> either.
>> 
>> 
>> 
>> On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk 
> wrote:
>> 
>>> On Tue, Aug 8, 2017 at 9:15 AM Mike Drob 
>> wrote:
>>> 
> The discussion also brought up the notion of an LTS release
>>> line.
> I'm
>>> not
> sure how this jives with the more fequent minors, but would
 require
>>> some
> branch that's so stable that an RM can effectively spin
>>> releases
>> blind.
 
 Seems 

Re: DISCUSS: How can we have less branches?

[Nick Dimiduk]

Reviving this thread. I know our interest is around branch-2 right now, but
I think we are close to an actionable resolution here.

I'm a fan of Andrew's proposal of stabilizing our branch-1 for maintenance
releases, getting away from all the minors if we can. Do we have a quorum
of folks able and willing to do that work? Seems like it would center
around the 1.4 release.

If branch-1.1 retires in the next 2-6 months, are we happy with branch-1.2,
branch-1.3, or branch-1.4 carrying on the monthly cadence? Do we have a
strong need to continue 1.x once 2.x is out? Are we calling 1.2 the LTS
with quarterly patch releases, 1.3 aborted on arrival, and pushing folks
onto 2.x? Do we want to settle on quarterly or bi-annual minor releases and
only patch monthly or as necessary in between?

Thanks,
Nick

On Wed, Aug 9, 2017 at 8:00 AM, Mike Drob  wrote:

> One thing that we need to be super careful about after releasing 2.0 is
> that we don't accidentally introduce any incompatible changes. It's much
> harder to reason about forwards compatibility than backwards compatibility
> in my experience. So even though a hypothetical 1.5 would be released after
> 2.0, users would still have the expectation that a 1.5->2.0 upgrade would
> be smooth (unless we explicitly message that it isn't but that is a whole
> new set of problems).
>
> Also, to clarify some context here -- what is the difference between an LTS
> branch and a Stable branch? Don't they convey the same thing to users? As a
> relative newcomer, I feel like I missed an earlier conversation about this.
>
> Mike
>
> On Tue, Aug 8, 2017 at 10:45 PM, Phil Yang  wrote:
>
> > Another option is no 1.5/branch-1 any more. What new features we are
> going
> > to have in HBase 1.5 (if it will be released)? Backporting from branch-2
> to
> > branch-1 is not easy, so maybe we will not have any big feature in
> branch-1
> > after releasing 1.4. And if we release 1.5 after releasing 2.0, it may
> > confuse users. So IMO we can focus on branch-2 for new features.
> >
> > I think it will be good if we have fixed logic for EOL branches, for
> > example(just an example, can discuss):
> > 1:Release a final version and EOL 1.x after we think 1.x+1 is stable.
> > 2:Cut off new 1.x+2 branch after 1.x+1 is stable, don't cut too early.
> >
> > Then we will not need discussing each time for each branch EOL and we
> will
> > have a fixed number of maintaining branches.
> >
> > Thanks,
> > Phil
> >
> >
> > 2017-08-09 1:44 GMT+08:00 Andrew Purtell :
> >
> > > Well you are not wrong that branching was premature, it turns out. But
> > I'll
> > > roll with it...
> > >
> > >
> > > On Tue, Aug 8, 2017 at 10:43 AM, Zach York <
> zyork.contribut...@gmail.com
> > >
> > > wrote:
> > >
> > > > > I made branch-1.4 a few weeks ago only.
> > > >
> > > > Whoops, sorry for that! For some reason I thought I had seen it
> months
> > > ago.
> > > >
> > > > On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell  >
> > > > wrote:
> > > >
> > > > > +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are
> candidates. I
> > > > think
> > > > > 1.1 has the edge because it lacks locking changes introduced into
> > 1.2,
> > > > just
> > > > > like 1.2 lacks locking changes introduced in 1.3 - the latter of
> > which
> > > > has
> > > > > had far reaching consequences, and the former not an insignificant
> > > change
> > > > > either.
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk 
> > > > wrote:
> > > > >
> > > > > > On Tue, Aug 8, 2017 at 9:15 AM Mike Drob 
> wrote:
> > > > > >
> > > > > > > > The discussion also brought up the notion of an LTS release
> > line.
> > > > I'm
> > > > > > not
> > > > > > > > sure how this jives with the more fequent minors, but would
> > > require
> > > > > > some
> > > > > > > > branch that's so stable that an RM can effectively spin
> > releases
> > > > > blind.
> > > > > > >
> > > > > > > Seems to me like this branch would necessarily need to be very
> > > > > > > backport-light? Only the top of the highest priority issues
> would
> > > be
> > > > > > > backportable to it, no?
> > > > > >
> > > > > >
> > > > > > The LTS is as 1.1 is today -- bug fixes only. The difference here
> > is
> > > > we'd
> > > > > > "formally" recognize the LTS designation somehow, perhaps with a
> > > > symlink
> > > > > > marker as we do for the "stable" designation.
> > > > > >
> > > > > > On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk <
> ndimi...@apache.org
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Last time we DISCUSSed EOL of 1.1 was back in November. At
> that
> > > > > time, a
> > > > > > > > litany of issues were raised re: 1.2. Have those concerns
> been
> > > > > > addressed?
> > > > > > > > It seems to me that making this one the last release is too
> > > abrupt
> > > > to
> > > > > > > folks
> > > > > > > > tracking Apache. Would be 

Re: DISCUSS: How can we have less branches?

[Mike Drob]

One thing that we need to be super careful about after releasing 2.0 is
that we don't accidentally introduce any incompatible changes. It's much
harder to reason about forwards compatibility than backwards compatibility
in my experience. So even though a hypothetical 1.5 would be released after
2.0, users would still have the expectation that a 1.5->2.0 upgrade would
be smooth (unless we explicitly message that it isn't but that is a whole
new set of problems).

Also, to clarify some context here -- what is the difference between an LTS
branch and a Stable branch? Don't they convey the same thing to users? As a
relative newcomer, I feel like I missed an earlier conversation about this.

Mike

On Tue, Aug 8, 2017 at 10:45 PM, Phil Yang  wrote:

> Another option is no 1.5/branch-1 any more. What new features we are going
> to have in HBase 1.5 (if it will be released)? Backporting from branch-2 to
> branch-1 is not easy, so maybe we will not have any big feature in branch-1
> after releasing 1.4. And if we release 1.5 after releasing 2.0, it may
> confuse users. So IMO we can focus on branch-2 for new features.
>
> I think it will be good if we have fixed logic for EOL branches, for
> example(just an example, can discuss):
> 1:Release a final version and EOL 1.x after we think 1.x+1 is stable.
> 2:Cut off new 1.x+2 branch after 1.x+1 is stable, don't cut too early.
>
> Then we will not need discussing each time for each branch EOL and we will
> have a fixed number of maintaining branches.
>
> Thanks,
> Phil
>
>
> 2017-08-09 1:44 GMT+08:00 Andrew Purtell :
>
> > Well you are not wrong that branching was premature, it turns out. But
> I'll
> > roll with it...
> >
> >
> > On Tue, Aug 8, 2017 at 10:43 AM, Zach York  >
> > wrote:
> >
> > > > I made branch-1.4 a few weeks ago only.
> > >
> > > Whoops, sorry for that! For some reason I thought I had seen it months
> > ago.
> > >
> > > On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell 
> > > wrote:
> > >
> > > > +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are candidates. I
> > > think
> > > > 1.1 has the edge because it lacks locking changes introduced into
> 1.2,
> > > just
> > > > like 1.2 lacks locking changes introduced in 1.3 - the latter of
> which
> > > has
> > > > had far reaching consequences, and the former not an insignificant
> > change
> > > > either.
> > > >
> > > >
> > > >
> > > > On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk 
> > > wrote:
> > > >
> > > > > On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:
> > > > >
> > > > > > > The discussion also brought up the notion of an LTS release
> line.
> > > I'm
> > > > > not
> > > > > > > sure how this jives with the more fequent minors, but would
> > require
> > > > > some
> > > > > > > branch that's so stable that an RM can effectively spin
> releases
> > > > blind.
> > > > > >
> > > > > > Seems to me like this branch would necessarily need to be very
> > > > > > backport-light? Only the top of the highest priority issues would
> > be
> > > > > > backportable to it, no?
> > > > >
> > > > >
> > > > > The LTS is as 1.1 is today -- bug fixes only. The difference here
> is
> > > we'd
> > > > > "formally" recognize the LTS designation somehow, perhaps with a
> > > symlink
> > > > > marker as we do for the "stable" designation.
> > > > >
> > > > > On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk  >
> > > > wrote:
> > > > > >
> > > > > > > Last time we DISCUSSed EOL of 1.1 was back in November. At that
> > > > time, a
> > > > > > > litany of issues were raised re: 1.2. Have those concerns been
> > > > > addressed?
> > > > > > > It seems to me that making this one the last release is too
> > abrupt
> > > to
> > > > > > folks
> > > > > > > tracking Apache. Would be better to give some notice.
> > > > > > >
> > > > > > > Had a nice hallway conversation a couple months back (at
> > > PhoenixCon,
> > > > as
> > > > > > it
> > > > > > > happens; they feel the pain as well) about our branch
> situation.
> > > I'll
> > > > > let
> > > > > > > the others chime in with more details, but the gist as I recall
> > is
> > > > that
> > > > > > we
> > > > > > > should be doing more frequent minor releases with fewer patch
> > > > releases.
> > > > > > > This pushes stabilization efforts closer to master and also
> > imposes
> > > > > more
> > > > > > > strict stability requirements on big new features before they
> can
> > > be
> > > > > > merged
> > > > > > > off the feature branch.
> > > > > > >
> > > > > > > The discussion also brought up the notion of an LTS release
> line.
> > > I'm
> > > > > not
> > > > > > > sure how this jives with the more fequent minors, but would
> > require
> > > > > some
> > > > > > > branch that's so stable that an RM can effectively spin
> releases
> > > > blind.
> > > > > > >
> > > > > > > On Tue, Aug 8, 2017 at 12:14 AM Stack 

Re: DISCUSS: How can we have less branches?

[Andrew Purtell]

Branch 2 doesn't have a release yet never mind enough production experience for 
anyone other than bleeding edge to consider it. Some of us are rebasing 
production on 1.x and once there intend for mid to long term stable operations. 
So I think it is very likely we as a project will be maintaining branch-1 for a 
long time. Perhaps as long as 3 years. (The lifetime of 0.98). Maybe it ends 
with 1.4 but that's probably unlikely. 

> On Aug 8, 2017, at 8:45 PM, Phil Yang  wrote:
> 
> Another option is no 1.5/branch-1 any more. What new features we are going
> to have in HBase 1.5 (if it will be released)? Backporting from branch-2 to
> branch-1 is not easy, so maybe we will not have any big feature in branch-1
> after releasing 1.4. And if we release 1.5 after releasing 2.0, it may
> confuse users. So IMO we can focus on branch-2 for new features.
> 
> I think it will be good if we have fixed logic for EOL branches, for
> example(just an example, can discuss):
> 1:Release a final version and EOL 1.x after we think 1.x+1 is stable.
> 2:Cut off new 1.x+2 branch after 1.x+1 is stable, don't cut too early.
> 
> Then we will not need discussing each time for each branch EOL and we will
> have a fixed number of maintaining branches.
> 
> Thanks,
> Phil
> 
> 
> 2017-08-09 1:44 GMT+08:00 Andrew Purtell :
> 
>> Well you are not wrong that branching was premature, it turns out. But I'll
>> roll with it...
>> 
>> 
>> On Tue, Aug 8, 2017 at 10:43 AM, Zach York 
>> wrote:
>> 
 I made branch-1.4 a few weeks ago only.
>>> 
>>> Whoops, sorry for that! For some reason I thought I had seen it months
>> ago.
>>> 
>>> On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell 
>>> wrote:
>>> 
 +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are candidates. I
>>> think
 1.1 has the edge because it lacks locking changes introduced into 1.2,
>>> just
 like 1.2 lacks locking changes introduced in 1.3 - the latter of which
>>> has
 had far reaching consequences, and the former not an insignificant
>> change
 either.
 
 
 
 On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk 
>>> wrote:
 
> On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:
> 
>>> The discussion also brought up the notion of an LTS release line.
>>> I'm
> not
>>> sure how this jives with the more fequent minors, but would
>> require
> some
>>> branch that's so stable that an RM can effectively spin releases
 blind.
>> 
>> Seems to me like this branch would necessarily need to be very
>> backport-light? Only the top of the highest priority issues would
>> be
>> backportable to it, no?
> 
> 
> The LTS is as 1.1 is today -- bug fixes only. The difference here is
>>> we'd
> "formally" recognize the LTS designation somehow, perhaps with a
>>> symlink
> marker as we do for the "stable" designation.
> 
> On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk 
 wrote:
>> 
>>> Last time we DISCUSSed EOL of 1.1 was back in November. At that
 time, a
>>> litany of issues were raised re: 1.2. Have those concerns been
> addressed?
>>> It seems to me that making this one the last release is too
>> abrupt
>>> to
>> folks
>>> tracking Apache. Would be better to give some notice.
>>> 
>>> Had a nice hallway conversation a couple months back (at
>>> PhoenixCon,
 as
>> it
>>> happens; they feel the pain as well) about our branch situation.
>>> I'll
> let
>>> the others chime in with more details, but the gist as I recall
>> is
 that
>> we
>>> should be doing more frequent minor releases with fewer patch
 releases.
>>> This pushes stabilization efforts closer to master and also
>> imposes
> more
>>> strict stability requirements on big new features before they can
>>> be
>> merged
>>> off the feature branch.
>>> 
>>> The discussion also brought up the notion of an LTS release line.
>>> I'm
> not
>>> sure how this jives with the more fequent minors, but would
>> require
> some
>>> branch that's so stable that an RM can effectively spin releases
 blind.
>>> 
 On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
 
 (This came up during dev meeting in Shenzhen) We are running
>> too
 many
 branches and/or when applying patches, we do not do a good job
>>> backporting
 to all active branches, especially fixes.
 
 We have master, branch-2, branch-1, branch-1.4, branch-1.3,
> branch-1.2,
>>> and
 branch-1.1 active currently. If a dirty bug fix, the applier is
>>> backporting
 to 7 branches. It takes a while applying to all especially if
>> the
>>> backport
 doesn't go in clean. I suppose the RM could monitor 

Re: DISCUSS: How can we have less branches?

[Phil Yang]

Another option is no 1.5/branch-1 any more. What new features we are going
to have in HBase 1.5 (if it will be released)? Backporting from branch-2 to
branch-1 is not easy, so maybe we will not have any big feature in branch-1
after releasing 1.4. And if we release 1.5 after releasing 2.0, it may
confuse users. So IMO we can focus on branch-2 for new features.

I think it will be good if we have fixed logic for EOL branches, for
example(just an example, can discuss):
1:Release a final version and EOL 1.x after we think 1.x+1 is stable.
2:Cut off new 1.x+2 branch after 1.x+1 is stable, don't cut too early.

Then we will not need discussing each time for each branch EOL and we will
have a fixed number of maintaining branches.

Thanks,
Phil


2017-08-09 1:44 GMT+08:00 Andrew Purtell :

> Well you are not wrong that branching was premature, it turns out. But I'll
> roll with it...
>
>
> On Tue, Aug 8, 2017 at 10:43 AM, Zach York 
> wrote:
>
> > > I made branch-1.4 a few weeks ago only.
> >
> > Whoops, sorry for that! For some reason I thought I had seen it months
> ago.
> >
> > On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell 
> > wrote:
> >
> > > +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are candidates. I
> > think
> > > 1.1 has the edge because it lacks locking changes introduced into 1.2,
> > just
> > > like 1.2 lacks locking changes introduced in 1.3 - the latter of which
> > has
> > > had far reaching consequences, and the former not an insignificant
> change
> > > either.
> > >
> > >
> > >
> > > On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk 
> > wrote:
> > >
> > > > On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:
> > > >
> > > > > > The discussion also brought up the notion of an LTS release line.
> > I'm
> > > > not
> > > > > > sure how this jives with the more fequent minors, but would
> require
> > > > some
> > > > > > branch that's so stable that an RM can effectively spin releases
> > > blind.
> > > > >
> > > > > Seems to me like this branch would necessarily need to be very
> > > > > backport-light? Only the top of the highest priority issues would
> be
> > > > > backportable to it, no?
> > > >
> > > >
> > > > The LTS is as 1.1 is today -- bug fixes only. The difference here is
> > we'd
> > > > "formally" recognize the LTS designation somehow, perhaps with a
> > symlink
> > > > marker as we do for the "stable" designation.
> > > >
> > > > On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk 
> > > wrote:
> > > > >
> > > > > > Last time we DISCUSSed EOL of 1.1 was back in November. At that
> > > time, a
> > > > > > litany of issues were raised re: 1.2. Have those concerns been
> > > > addressed?
> > > > > > It seems to me that making this one the last release is too
> abrupt
> > to
> > > > > folks
> > > > > > tracking Apache. Would be better to give some notice.
> > > > > >
> > > > > > Had a nice hallway conversation a couple months back (at
> > PhoenixCon,
> > > as
> > > > > it
> > > > > > happens; they feel the pain as well) about our branch situation.
> > I'll
> > > > let
> > > > > > the others chime in with more details, but the gist as I recall
> is
> > > that
> > > > > we
> > > > > > should be doing more frequent minor releases with fewer patch
> > > releases.
> > > > > > This pushes stabilization efforts closer to master and also
> imposes
> > > > more
> > > > > > strict stability requirements on big new features before they can
> > be
> > > > > merged
> > > > > > off the feature branch.
> > > > > >
> > > > > > The discussion also brought up the notion of an LTS release line.
> > I'm
> > > > not
> > > > > > sure how this jives with the more fequent minors, but would
> require
> > > > some
> > > > > > branch that's so stable that an RM can effectively spin releases
> > > blind.
> > > > > >
> > > > > > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> > > > > >
> > > > > > > (This came up during dev meeting in Shenzhen) We are running
> too
> > > many
> > > > > > > branches and/or when applying patches, we do not do a good job
> > > > > > backporting
> > > > > > > to all active branches, especially fixes.
> > > > > > >
> > > > > > > We have master, branch-2, branch-1, branch-1.4, branch-1.3,
> > > > branch-1.2,
> > > > > > and
> > > > > > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > > > > > backporting
> > > > > > > to 7 branches. It takes a while applying to all especially if
> the
> > > > > > backport
> > > > > > > doesn't go in clean. I suppose the RM could monitor all
> upstream
> > of
> > > > > them
> > > > > > > and then pull wanted patches back (we could do that too) but
> > seems
> > > > > like a
> > > > > > > burden on an RMer.
> > > > > > >
> > > > > > > We should EOL a few?
> > > > > > >
> > > > > > > Nick is on branch-1.1 release at the moment. Perhaps this could
> > be
> > > > the
> > > > > > last
> > > > > > 

Re: DISCUSS: How can we have less branches?

[Andrew Purtell]

Well you are not wrong that branching was premature, it turns out. But I'll
roll with it...


On Tue, Aug 8, 2017 at 10:43 AM, Zach York 
wrote:

> > I made branch-1.4 a few weeks ago only.
>
> Whoops, sorry for that! For some reason I thought I had seen it months ago.
>
> On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell 
> wrote:
>
> > +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are candidates. I
> think
> > 1.1 has the edge because it lacks locking changes introduced into 1.2,
> just
> > like 1.2 lacks locking changes introduced in 1.3 - the latter of which
> has
> > had far reaching consequences, and the former not an insignificant change
> > either.
> >
> >
> >
> > On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk 
> wrote:
> >
> > > On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:
> > >
> > > > > The discussion also brought up the notion of an LTS release line.
> I'm
> > > not
> > > > > sure how this jives with the more fequent minors, but would require
> > > some
> > > > > branch that's so stable that an RM can effectively spin releases
> > blind.
> > > >
> > > > Seems to me like this branch would necessarily need to be very
> > > > backport-light? Only the top of the highest priority issues would be
> > > > backportable to it, no?
> > >
> > >
> > > The LTS is as 1.1 is today -- bug fixes only. The difference here is
> we'd
> > > "formally" recognize the LTS designation somehow, perhaps with a
> symlink
> > > marker as we do for the "stable" designation.
> > >
> > > On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk 
> > wrote:
> > > >
> > > > > Last time we DISCUSSed EOL of 1.1 was back in November. At that
> > time, a
> > > > > litany of issues were raised re: 1.2. Have those concerns been
> > > addressed?
> > > > > It seems to me that making this one the last release is too abrupt
> to
> > > > folks
> > > > > tracking Apache. Would be better to give some notice.
> > > > >
> > > > > Had a nice hallway conversation a couple months back (at
> PhoenixCon,
> > as
> > > > it
> > > > > happens; they feel the pain as well) about our branch situation.
> I'll
> > > let
> > > > > the others chime in with more details, but the gist as I recall is
> > that
> > > > we
> > > > > should be doing more frequent minor releases with fewer patch
> > releases.
> > > > > This pushes stabilization efforts closer to master and also imposes
> > > more
> > > > > strict stability requirements on big new features before they can
> be
> > > > merged
> > > > > off the feature branch.
> > > > >
> > > > > The discussion also brought up the notion of an LTS release line.
> I'm
> > > not
> > > > > sure how this jives with the more fequent minors, but would require
> > > some
> > > > > branch that's so stable that an RM can effectively spin releases
> > blind.
> > > > >
> > > > > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> > > > >
> > > > > > (This came up during dev meeting in Shenzhen) We are running too
> > many
> > > > > > branches and/or when applying patches, we do not do a good job
> > > > > backporting
> > > > > > to all active branches, especially fixes.
> > > > > >
> > > > > > We have master, branch-2, branch-1, branch-1.4, branch-1.3,
> > > branch-1.2,
> > > > > and
> > > > > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > > > > backporting
> > > > > > to 7 branches. It takes a while applying to all especially if the
> > > > > backport
> > > > > > doesn't go in clean. I suppose the RM could monitor all upstream
> of
> > > > them
> > > > > > and then pull wanted patches back (we could do that too) but
> seems
> > > > like a
> > > > > > burden on an RMer.
> > > > > >
> > > > > > We should EOL a few?
> > > > > >
> > > > > > Nick is on branch-1.1 release at the moment. Perhaps this could
> be
> > > the
> > > > > last
> > > > > > off branch-1.1.
> > > > > >
> > > > > > 1.2 hosts our current stable release though 1.3 is out. How about
> > we
> > > > cut
> > > > > a
> > > > > > release off 1.3, call it stable, and then EOL 1.2 after another
> > > release
> > > > > or
> > > > > > so?
> > > > > >
> > > > > > What you all think?
> > > > > >
> > > > > > Thanks,
> > > > > > S
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: DISCUSS: How can we have less branches?

[Zach York]

> I made branch-1.4 a few weeks ago only.

Whoops, sorry for that! For some reason I thought I had seen it months ago.

On Tue, Aug 8, 2017 at 10:40 AM, Andrew Purtell  wrote:

> +1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are candidates. I think
> 1.1 has the edge because it lacks locking changes introduced into 1.2, just
> like 1.2 lacks locking changes introduced in 1.3 - the latter of which has
> had far reaching consequences, and the former not an insignificant change
> either.
>
>
>
> On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk  wrote:
>
> > On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:
> >
> > > > The discussion also brought up the notion of an LTS release line. I'm
> > not
> > > > sure how this jives with the more fequent minors, but would require
> > some
> > > > branch that's so stable that an RM can effectively spin releases
> blind.
> > >
> > > Seems to me like this branch would necessarily need to be very
> > > backport-light? Only the top of the highest priority issues would be
> > > backportable to it, no?
> >
> >
> > The LTS is as 1.1 is today -- bug fixes only. The difference here is we'd
> > "formally" recognize the LTS designation somehow, perhaps with a symlink
> > marker as we do for the "stable" designation.
> >
> > On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk 
> wrote:
> > >
> > > > Last time we DISCUSSed EOL of 1.1 was back in November. At that
> time, a
> > > > litany of issues were raised re: 1.2. Have those concerns been
> > addressed?
> > > > It seems to me that making this one the last release is too abrupt to
> > > folks
> > > > tracking Apache. Would be better to give some notice.
> > > >
> > > > Had a nice hallway conversation a couple months back (at PhoenixCon,
> as
> > > it
> > > > happens; they feel the pain as well) about our branch situation. I'll
> > let
> > > > the others chime in with more details, but the gist as I recall is
> that
> > > we
> > > > should be doing more frequent minor releases with fewer patch
> releases.
> > > > This pushes stabilization efforts closer to master and also imposes
> > more
> > > > strict stability requirements on big new features before they can be
> > > merged
> > > > off the feature branch.
> > > >
> > > > The discussion also brought up the notion of an LTS release line. I'm
> > not
> > > > sure how this jives with the more fequent minors, but would require
> > some
> > > > branch that's so stable that an RM can effectively spin releases
> blind.
> > > >
> > > > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> > > >
> > > > > (This came up during dev meeting in Shenzhen) We are running too
> many
> > > > > branches and/or when applying patches, we do not do a good job
> > > > backporting
> > > > > to all active branches, especially fixes.
> > > > >
> > > > > We have master, branch-2, branch-1, branch-1.4, branch-1.3,
> > branch-1.2,
> > > > and
> > > > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > > > backporting
> > > > > to 7 branches. It takes a while applying to all especially if the
> > > > backport
> > > > > doesn't go in clean. I suppose the RM could monitor all upstream of
> > > them
> > > > > and then pull wanted patches back (we could do that too) but seems
> > > like a
> > > > > burden on an RMer.
> > > > >
> > > > > We should EOL a few?
> > > > >
> > > > > Nick is on branch-1.1 release at the moment. Perhaps this could be
> > the
> > > > last
> > > > > off branch-1.1.
> > > > >
> > > > > 1.2 hosts our current stable release though 1.3 is out. How about
> we
> > > cut
> > > > a
> > > > > release off 1.3, call it stable, and then EOL 1.2 after another
> > release
> > > > or
> > > > > so?
> > > > >
> > > > > What you all think?
> > > > >
> > > > > Thanks,
> > > > > S
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

Re: DISCUSS: How can we have less branches?

[Andrew Purtell]

+1 from me on making 1.1 our LTS. Either 1.1 or 1.2 are candidates. I think
1.1 has the edge because it lacks locking changes introduced into 1.2, just
like 1.2 lacks locking changes introduced in 1.3 - the latter of which has
had far reaching consequences, and the former not an insignificant change
either.



On Tue, Aug 8, 2017 at 10:31 AM, Nick Dimiduk  wrote:

> On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:
>
> > > The discussion also brought up the notion of an LTS release line. I'm
> not
> > > sure how this jives with the more fequent minors, but would require
> some
> > > branch that's so stable that an RM can effectively spin releases blind.
> >
> > Seems to me like this branch would necessarily need to be very
> > backport-light? Only the top of the highest priority issues would be
> > backportable to it, no?
>
>
> The LTS is as 1.1 is today -- bug fixes only. The difference here is we'd
> "formally" recognize the LTS designation somehow, perhaps with a symlink
> marker as we do for the "stable" designation.
>
> On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk  wrote:
> >
> > > Last time we DISCUSSed EOL of 1.1 was back in November. At that time, a
> > > litany of issues were raised re: 1.2. Have those concerns been
> addressed?
> > > It seems to me that making this one the last release is too abrupt to
> > folks
> > > tracking Apache. Would be better to give some notice.
> > >
> > > Had a nice hallway conversation a couple months back (at PhoenixCon, as
> > it
> > > happens; they feel the pain as well) about our branch situation. I'll
> let
> > > the others chime in with more details, but the gist as I recall is that
> > we
> > > should be doing more frequent minor releases with fewer patch releases.
> > > This pushes stabilization efforts closer to master and also imposes
> more
> > > strict stability requirements on big new features before they can be
> > merged
> > > off the feature branch.
> > >
> > > The discussion also brought up the notion of an LTS release line. I'm
> not
> > > sure how this jives with the more fequent minors, but would require
> some
> > > branch that's so stable that an RM can effectively spin releases blind.
> > >
> > > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> > >
> > > > (This came up during dev meeting in Shenzhen) We are running too many
> > > > branches and/or when applying patches, we do not do a good job
> > > backporting
> > > > to all active branches, especially fixes.
> > > >
> > > > We have master, branch-2, branch-1, branch-1.4, branch-1.3,
> branch-1.2,
> > > and
> > > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > > backporting
> > > > to 7 branches. It takes a while applying to all especially if the
> > > backport
> > > > doesn't go in clean. I suppose the RM could monitor all upstream of
> > them
> > > > and then pull wanted patches back (we could do that too) but seems
> > like a
> > > > burden on an RMer.
> > > >
> > > > We should EOL a few?
> > > >
> > > > Nick is on branch-1.1 release at the moment. Perhaps this could be
> the
> > > last
> > > > off branch-1.1.
> > > >
> > > > 1.2 hosts our current stable release though 1.3 is out. How about we
> > cut
> > > a
> > > > release off 1.3, call it stable, and then EOL 1.2 after another
> release
> > > or
> > > > so?
> > > >
> > > > What you all think?
> > > >
> > > > Thanks,
> > > > S
> > > >
> > >
> >
>



-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: DISCUSS: How can we have less branches?

[Andrew Purtell]

I made branch-1.4 a few weeks ago only. I did it because I had hoped to RC
quite quickly. Then the reality of compatibility breaks and failing unit
tests hit, as well as insufficient time for me to make progress on my
must-have for the release, which is a backport to RSGroups. I will get to
all of this as soon as I can given work commitments. Maybe we can have a RC
of 1.4.0 by the end of next month.

I don't disagree with you, though. If there were more active scrutiny of
branch-1 then we would have found the compat breaks and addressed the test
problems near when they happened and my work as RM of 1.4 will have been
lessened considerably.

I would like to see us move to a model where we have three main code lines:

master: 3.x
branch-2: 2.x
branch-1: 1.x

and we migrate our RM model away from RMs curating patch branches to a
model where there are three people actively monitoring the state of the
above three branches, and, when considering releasing, we prefer to cut new
minors from these three branches (modulo master) over patch release work.
There will be times when patch releases are necessary. I don't think a
dedicated RM for a patch branch is necessary, though. If there is a
critical bug fix for branch-1, the "RM" for branch-1 can spin patch
releases of all 1.x code lines we have decided to keep alive and run votes
on all of them. This is incremental additional effort after getting set up
to make releases in the first place.

There are two huge benefits:

1. We split precious RM resources in fewer ways.

2. We have fewer code lines receiving the majority of commits, so
committers do less work.


On Tue, Aug 8, 2017 at 10:19 AM, Zach York 
wrote:

> One problem that I see is that it appears we are cutting branches extremely
> early.
> I'm going to pick on branch-1.4 for a moment. This branch has existed (as
> far as I can remember) for at least 3-6 months, yet we still have not had a
> 1.4.0 release. I understand that releases can take time, but this adds pain
> for developers (If I'm developing a new feature or bug fix, do I put it on
> branch-1 and it will be pulled into branch-1.4 or do I need to explicitly
> put it on branch-1.4?). It seems that this branch should be the same as
> branch-1 at the moment.
>
> So my suggestion would be use branch-1 (or branch-2 now) for all
> development and only create a branch when the first release of a line is
> imminent.
>
> Perhaps I am missing some context, but that is my 2 cents.
>
> Thanks,
> Zach
>
> On Tue, Aug 8, 2017 at 9:14 AM, Mike Drob  wrote:
>
> > > The discussion also brought up the notion of an LTS release line. I'm
> not
> > > sure how this jives with the more fequent minors, but would require
> some
> > > branch that's so stable that an RM can effectively spin releases blind.
> >
> > Seems to me like this branch would necessarily need to be very
> > backport-light? Only the top of the highest priority issues would be
> > backportable to it, no?
> >
> > On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk 
> wrote:
> >
> > > Last time we DISCUSSed EOL of 1.1 was back in November. At that time, a
> > > litany of issues were raised re: 1.2. Have those concerns been
> addressed?
> > > It seems to me that making this one the last release is too abrupt to
> > folks
> > > tracking Apache. Would be better to give some notice.
> > >
> > > Had a nice hallway conversation a couple months back (at PhoenixCon, as
> > it
> > > happens; they feel the pain as well) about our branch situation. I'll
> let
> > > the others chime in with more details, but the gist as I recall is that
> > we
> > > should be doing more frequent minor releases with fewer patch releases.
> > > This pushes stabilization efforts closer to master and also imposes
> more
> > > strict stability requirements on big new features before they can be
> > merged
> > > off the feature branch.
> > >
> > > The discussion also brought up the notion of an LTS release line. I'm
> not
> > > sure how this jives with the more fequent minors, but would require
> some
> > > branch that's so stable that an RM can effectively spin releases blind.
> > >
> > > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> > >
> > > > (This came up during dev meeting in Shenzhen) We are running too many
> > > > branches and/or when applying patches, we do not do a good job
> > > backporting
> > > > to all active branches, especially fixes.
> > > >
> > > > We have master, branch-2, branch-1, branch-1.4, branch-1.3,
> branch-1.2,
> > > and
> > > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > > backporting
> > > > to 7 branches. It takes a while applying to all especially if the
> > > backport
> > > > doesn't go in clean. I suppose the RM could monitor all upstream of
> > them
> > > > and then pull wanted patches back (we could do that too) but seems
> > like a
> > > > burden on an RMer.
> > > >
> > > > We should EOL a few?
> > 

Re: DISCUSS: How can we have less branches?

[Nick Dimiduk]

On Tue, Aug 8, 2017 at 9:15 AM Mike Drob  wrote:

> > The discussion also brought up the notion of an LTS release line. I'm not
> > sure how this jives with the more fequent minors, but would require some
> > branch that's so stable that an RM can effectively spin releases blind.
>
> Seems to me like this branch would necessarily need to be very
> backport-light? Only the top of the highest priority issues would be
> backportable to it, no?


The LTS is as 1.1 is today -- bug fixes only. The difference here is we'd
"formally" recognize the LTS designation somehow, perhaps with a symlink
marker as we do for the "stable" designation.

On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk  wrote:
>
> > Last time we DISCUSSed EOL of 1.1 was back in November. At that time, a
> > litany of issues were raised re: 1.2. Have those concerns been addressed?
> > It seems to me that making this one the last release is too abrupt to
> folks
> > tracking Apache. Would be better to give some notice.
> >
> > Had a nice hallway conversation a couple months back (at PhoenixCon, as
> it
> > happens; they feel the pain as well) about our branch situation. I'll let
> > the others chime in with more details, but the gist as I recall is that
> we
> > should be doing more frequent minor releases with fewer patch releases.
> > This pushes stabilization efforts closer to master and also imposes more
> > strict stability requirements on big new features before they can be
> merged
> > off the feature branch.
> >
> > The discussion also brought up the notion of an LTS release line. I'm not
> > sure how this jives with the more fequent minors, but would require some
> > branch that's so stable that an RM can effectively spin releases blind.
> >
> > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> >
> > > (This came up during dev meeting in Shenzhen) We are running too many
> > > branches and/or when applying patches, we do not do a good job
> > backporting
> > > to all active branches, especially fixes.
> > >
> > > We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2,
> > and
> > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > backporting
> > > to 7 branches. It takes a while applying to all especially if the
> > backport
> > > doesn't go in clean. I suppose the RM could monitor all upstream of
> them
> > > and then pull wanted patches back (we could do that too) but seems
> like a
> > > burden on an RMer.
> > >
> > > We should EOL a few?
> > >
> > > Nick is on branch-1.1 release at the moment. Perhaps this could be the
> > last
> > > off branch-1.1.
> > >
> > > 1.2 hosts our current stable release though 1.3 is out. How about we
> cut
> > a
> > > release off 1.3, call it stable, and then EOL 1.2 after another release
> > or
> > > so?
> > >
> > > What you all think?
> > >
> > > Thanks,
> > > S
> > >
> >
>

Re: DISCUSS: How can we have less branches?

[Zach York]

One problem that I see is that it appears we are cutting branches extremely
early.
I'm going to pick on branch-1.4 for a moment. This branch has existed (as
far as I can remember) for at least 3-6 months, yet we still have not had a
1.4.0 release. I understand that releases can take time, but this adds pain
for developers (If I'm developing a new feature or bug fix, do I put it on
branch-1 and it will be pulled into branch-1.4 or do I need to explicitly
put it on branch-1.4?). It seems that this branch should be the same as
branch-1 at the moment.

So my suggestion would be use branch-1 (or branch-2 now) for all
development and only create a branch when the first release of a line is
imminent.

Perhaps I am missing some context, but that is my 2 cents.

Thanks,
Zach

On Tue, Aug 8, 2017 at 9:14 AM, Mike Drob  wrote:

> > The discussion also brought up the notion of an LTS release line. I'm not
> > sure how this jives with the more fequent minors, but would require some
> > branch that's so stable that an RM can effectively spin releases blind.
>
> Seems to me like this branch would necessarily need to be very
> backport-light? Only the top of the highest priority issues would be
> backportable to it, no?
>
> On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk  wrote:
>
> > Last time we DISCUSSed EOL of 1.1 was back in November. At that time, a
> > litany of issues were raised re: 1.2. Have those concerns been addressed?
> > It seems to me that making this one the last release is too abrupt to
> folks
> > tracking Apache. Would be better to give some notice.
> >
> > Had a nice hallway conversation a couple months back (at PhoenixCon, as
> it
> > happens; they feel the pain as well) about our branch situation. I'll let
> > the others chime in with more details, but the gist as I recall is that
> we
> > should be doing more frequent minor releases with fewer patch releases.
> > This pushes stabilization efforts closer to master and also imposes more
> > strict stability requirements on big new features before they can be
> merged
> > off the feature branch.
> >
> > The discussion also brought up the notion of an LTS release line. I'm not
> > sure how this jives with the more fequent minors, but would require some
> > branch that's so stable that an RM can effectively spin releases blind.
> >
> > On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
> >
> > > (This came up during dev meeting in Shenzhen) We are running too many
> > > branches and/or when applying patches, we do not do a good job
> > backporting
> > > to all active branches, especially fixes.
> > >
> > > We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2,
> > and
> > > branch-1.1 active currently. If a dirty bug fix, the applier is
> > backporting
> > > to 7 branches. It takes a while applying to all especially if the
> > backport
> > > doesn't go in clean. I suppose the RM could monitor all upstream of
> them
> > > and then pull wanted patches back (we could do that too) but seems
> like a
> > > burden on an RMer.
> > >
> > > We should EOL a few?
> > >
> > > Nick is on branch-1.1 release at the moment. Perhaps this could be the
> > last
> > > off branch-1.1.
> > >
> > > 1.2 hosts our current stable release though 1.3 is out. How about we
> cut
> > a
> > > release off 1.3, call it stable, and then EOL 1.2 after another release
> > or
> > > so?
> > >
> > > What you all think?
> > >
> > > Thanks,
> > > S
> > >
> >
>

Re: DISCUSS: How can we have less branches?

[Mike Drob]

> The discussion also brought up the notion of an LTS release line. I'm not
> sure how this jives with the more fequent minors, but would require some
> branch that's so stable that an RM can effectively spin releases blind.

Seems to me like this branch would necessarily need to be very
backport-light? Only the top of the highest priority issues would be
backportable to it, no?

On Tue, Aug 8, 2017 at 11:09 AM, Nick Dimiduk  wrote:

> Last time we DISCUSSed EOL of 1.1 was back in November. At that time, a
> litany of issues were raised re: 1.2. Have those concerns been addressed?
> It seems to me that making this one the last release is too abrupt to folks
> tracking Apache. Would be better to give some notice.
>
> Had a nice hallway conversation a couple months back (at PhoenixCon, as it
> happens; they feel the pain as well) about our branch situation. I'll let
> the others chime in with more details, but the gist as I recall is that we
> should be doing more frequent minor releases with fewer patch releases.
> This pushes stabilization efforts closer to master and also imposes more
> strict stability requirements on big new features before they can be merged
> off the feature branch.
>
> The discussion also brought up the notion of an LTS release line. I'm not
> sure how this jives with the more fequent minors, but would require some
> branch that's so stable that an RM can effectively spin releases blind.
>
> On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:
>
> > (This came up during dev meeting in Shenzhen) We are running too many
> > branches and/or when applying patches, we do not do a good job
> backporting
> > to all active branches, especially fixes.
> >
> > We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2,
> and
> > branch-1.1 active currently. If a dirty bug fix, the applier is
> backporting
> > to 7 branches. It takes a while applying to all especially if the
> backport
> > doesn't go in clean. I suppose the RM could monitor all upstream of them
> > and then pull wanted patches back (we could do that too) but seems like a
> > burden on an RMer.
> >
> > We should EOL a few?
> >
> > Nick is on branch-1.1 release at the moment. Perhaps this could be the
> last
> > off branch-1.1.
> >
> > 1.2 hosts our current stable release though 1.3 is out. How about we cut
> a
> > release off 1.3, call it stable, and then EOL 1.2 after another release
> or
> > so?
> >
> > What you all think?
> >
> > Thanks,
> > S
> >
>

Re: DISCUSS: How can we have less branches?

[Nick Dimiduk]

Last time we DISCUSSed EOL of 1.1 was back in November. At that time, a
litany of issues were raised re: 1.2. Have those concerns been addressed?
It seems to me that making this one the last release is too abrupt to folks
tracking Apache. Would be better to give some notice.

Had a nice hallway conversation a couple months back (at PhoenixCon, as it
happens; they feel the pain as well) about our branch situation. I'll let
the others chime in with more details, but the gist as I recall is that we
should be doing more frequent minor releases with fewer patch releases.
This pushes stabilization efforts closer to master and also imposes more
strict stability requirements on big new features before they can be merged
off the feature branch.

The discussion also brought up the notion of an LTS release line. I'm not
sure how this jives with the more fequent minors, but would require some
branch that's so stable that an RM can effectively spin releases blind.

On Tue, Aug 8, 2017 at 12:14 AM Stack  wrote:

> (This came up during dev meeting in Shenzhen) We are running too many
> branches and/or when applying patches, we do not do a good job backporting
> to all active branches, especially fixes.
>
> We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2, and
> branch-1.1 active currently. If a dirty bug fix, the applier is backporting
> to 7 branches. It takes a while applying to all especially if the backport
> doesn't go in clean. I suppose the RM could monitor all upstream of them
> and then pull wanted patches back (we could do that too) but seems like a
> burden on an RMer.
>
> We should EOL a few?
>
> Nick is on branch-1.1 release at the moment. Perhaps this could be the last
> off branch-1.1.
>
> 1.2 hosts our current stable release though 1.3 is out. How about we cut a
> release off 1.3, call it stable, and then EOL 1.2 after another release or
> so?
>
> What you all think?
>
> Thanks,
> S
>

Re: DISCUSS: How can we have less branches?

[Mikhail Antonov]

I can also start separate discussion thread on what prevents 1.3 (and 1.4
subsequently, though Andrew would be the right person here) from being
marked stable in our books, so this issue gets more focused attention.

-Mikhail

On Tue, Aug 8, 2017 at 8:10 AM, Mikhail Antonov 
wrote:

> I think in the dev community interest and ultimately in interest of our
> users to have a fewer maintained branches.
>
> For a variety of reasons.
>
> Efforts put in the maintenance of old release lines is the effort not put
> into releasing new ones, since community resources are
> finite. Active backporting (beyond critical security fixes and such) of
> new things somewhat reduces users incentive to upgrade often. Then we can
> get in the loop situation when new major features aren't deemed very stable
> - but it's really hard to get features of that complexity stable and robust
> if they're not used beyond integration tests.
>
> So before we go too deeply into discussion on how to make backporting
> easier, I'd rather see us focusing on releasing new branches and making
> them stable.
>
> -Mikhail
>
>
>
> On Tue, Aug 8, 2017 at 7:10 AM, Sean Busbey  wrote:
>
>> Are we approaching this problem wrong though?
>>
>> Most of the cherry picks I have to do to the maintenance release lines are
>> clean.
>>
>> What if we get more tooling to help with the everything-goes-right case?
>>
>> My last read of asf policy and infra folks (from back in the website
>> automation) is they won't look favorably on Jenkins pushing back ports
>> into
>> our repo.
>>
>> But! We could make something similar to the website job from back when a
>> committer still had to do the git push. A nightly job that tries to cherry
>> pick and gives us a set of back ports that worked.
>>
>> We could either have it look to jira for fix versions to try backporting
>> to, or we could have it try everything and update jira with what worked.
>>
>> Thoughts?
>>
>> On Aug 8, 2017 02:14, "Stack"  wrote:
>>
>> > (This came up during dev meeting in Shenzhen) We are running too many
>> > branches and/or when applying patches, we do not do a good job
>> backporting
>> > to all active branches, especially fixes.
>> >
>> > We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2,
>> and
>> > branch-1.1 active currently. If a dirty bug fix, the applier is
>> backporting
>> > to 7 branches. It takes a while applying to all especially if the
>> backport
>> > doesn't go in clean. I suppose the RM could monitor all upstream of them
>> > and then pull wanted patches back (we could do that too) but seems like
>> a
>> > burden on an RMer.
>> >
>> > We should EOL a few?
>> >
>> > Nick is on branch-1.1 release at the moment. Perhaps this could be the
>> last
>> > off branch-1.1.
>> >
>> > 1.2 hosts our current stable release though 1.3 is out. How about we
>> cut a
>> > release off 1.3, call it stable, and then EOL 1.2 after another release
>> or
>> > so?
>> >
>> > What you all think?
>> >
>> > Thanks,
>> > S
>> >
>>
>
>
>
> --
> Thanks,
> Michael Antonov
>



-- 
Thanks,
Michael Antonov

Re: DISCUSS: How can we have less branches?

[Mikhail Antonov]

I think in the dev community interest and ultimately in interest of our
users to have a fewer maintained branches.

For a variety of reasons.

Efforts put in the maintenance of old release lines is the effort not put
into releasing new ones, since community resources are
finite. Active backporting (beyond critical security fixes and such) of new
things somewhat reduces users incentive to upgrade often. Then we can get
in the loop situation when new major features aren't deemed very stable -
but it's really hard to get features of that complexity stable and robust
if they're not used beyond integration tests.

So before we go too deeply into discussion on how to make backporting
easier, I'd rather see us focusing on releasing new branches and making
them stable.

-Mikhail



On Tue, Aug 8, 2017 at 7:10 AM, Sean Busbey  wrote:

> Are we approaching this problem wrong though?
>
> Most of the cherry picks I have to do to the maintenance release lines are
> clean.
>
> What if we get more tooling to help with the everything-goes-right case?
>
> My last read of asf policy and infra folks (from back in the website
> automation) is they won't look favorably on Jenkins pushing back ports into
> our repo.
>
> But! We could make something similar to the website job from back when a
> committer still had to do the git push. A nightly job that tries to cherry
> pick and gives us a set of back ports that worked.
>
> We could either have it look to jira for fix versions to try backporting
> to, or we could have it try everything and update jira with what worked.
>
> Thoughts?
>
> On Aug 8, 2017 02:14, "Stack"  wrote:
>
> > (This came up during dev meeting in Shenzhen) We are running too many
> > branches and/or when applying patches, we do not do a good job
> backporting
> > to all active branches, especially fixes.
> >
> > We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2,
> and
> > branch-1.1 active currently. If a dirty bug fix, the applier is
> backporting
> > to 7 branches. It takes a while applying to all especially if the
> backport
> > doesn't go in clean. I suppose the RM could monitor all upstream of them
> > and then pull wanted patches back (we could do that too) but seems like a
> > burden on an RMer.
> >
> > We should EOL a few?
> >
> > Nick is on branch-1.1 release at the moment. Perhaps this could be the
> last
> > off branch-1.1.
> >
> > 1.2 hosts our current stable release though 1.3 is out. How about we cut
> a
> > release off 1.3, call it stable, and then EOL 1.2 after another release
> or
> > so?
> >
> > What you all think?
> >
> > Thanks,
> > S
> >
>



-- 
Thanks,
Michael Antonov

Re: DISCUSS: How can we have less branches?

[Sean Busbey]

Are we approaching this problem wrong though?

Most of the cherry picks I have to do to the maintenance release lines are
clean.

What if we get more tooling to help with the everything-goes-right case?

My last read of asf policy and infra folks (from back in the website
automation) is they won't look favorably on Jenkins pushing back ports into
our repo.

But! We could make something similar to the website job from back when a
committer still had to do the git push. A nightly job that tries to cherry
pick and gives us a set of back ports that worked.

We could either have it look to jira for fix versions to try backporting
to, or we could have it try everything and update jira with what worked.

Thoughts?

On Aug 8, 2017 02:14, "Stack"  wrote:

> (This came up during dev meeting in Shenzhen) We are running too many
> branches and/or when applying patches, we do not do a good job backporting
> to all active branches, especially fixes.
>
> We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2, and
> branch-1.1 active currently. If a dirty bug fix, the applier is backporting
> to 7 branches. It takes a while applying to all especially if the backport
> doesn't go in clean. I suppose the RM could monitor all upstream of them
> and then pull wanted patches back (we could do that too) but seems like a
> burden on an RMer.
>
> We should EOL a few?
>
> Nick is on branch-1.1 release at the moment. Perhaps this could be the last
> off branch-1.1.
>
> 1.2 hosts our current stable release though 1.3 is out. How about we cut a
> release off 1.3, call it stable, and then EOL 1.2 after another release or
> so?
>
> What you all think?
>
> Thanks,
> S
>

Re: DISCUSS: How can we have less branches?

[Sean Busbey]

I'd love to see us more aggressively updating the stable pointer.

I spent a but of time at the last hbasecon west socializing more EOL, but
as Mikhail mentions we can't make 1.3 the stable line while we know it's
broken. AFAIK, that same problem is in brach-1.4.

We could immediately EOL 1.3 and then declare 1.4.1 stable once the problem
is fixed. (Aiming the fix to 1.4.1 so that Andrew doesn't need to stop
before making a 1.4.0 if his other stabilization efforts finish first.)

This gets us fewer branches now and a plan to move the stable pointer.


On Aug 8, 2017 05:17, "Mikhail Antonov"  wrote:

I totally agree we have too many branches to maintain.

I think we should EOL 1.1. 1.2 has been out and stable for a long time,
doesn't have any outstanding major problems preventing upgrade that I'm
aware of, though obviously Sean would know better.

Speaking of 1.3, there's one issue that I feel needs to be addressed before
we move stable pointer to 1.3 and start EOL-ing 1.2

HBASE-18397, TL;DR - major optimization changes in scanners locking schema
in 1.3 broke store file accounting, which could cause failed long scans,
failed splits and failed snapshots. Many people spent lots and lots of time
fixing various aspects of that, but there are still issues not yet fixed.
Any help in that area would be amazing. This specific issue probably
deserves separate discussion.

Thanks,
Mikhail

On Tue, Aug 8, 2017 at 12:14 AM, Stack  wrote:

> (This came up during dev meeting in Shenzhen) We are running too many
> branches and/or when applying patches, we do not do a good job backporting
> to all active branches, especially fixes.
>
> We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2,
and
> branch-1.1 active currently. If a dirty bug fix, the applier is
backporting
> to 7 branches. It takes a while applying to all especially if the backport
> doesn't go in clean. I suppose the RM could monitor all upstream of them
> and then pull wanted patches back (we could do that too) but seems like a
> burden on an RMer.
>
> We should EOL a few?
>
> Nick is on branch-1.1 release at the moment. Perhaps this could be the
last
> off branch-1.1.
>
> 1.2 hosts our current stable release though 1.3 is out. How about we cut a
> release off 1.3, call it stable, and then EOL 1.2 after another release or
> so?
>
> What you all think?
>
> Thanks,
> S
>



--
Thanks,
Michael Antonov

Re: DISCUSS: How can we have less branches?

[Mikhail Antonov]

I totally agree we have too many branches to maintain.

I think we should EOL 1.1. 1.2 has been out and stable for a long time,
doesn't have any outstanding major problems preventing upgrade that I'm
aware of, though obviously Sean would know better.

Speaking of 1.3, there's one issue that I feel needs to be addressed before
we move stable pointer to 1.3 and start EOL-ing 1.2

HBASE-18397, TL;DR - major optimization changes in scanners locking schema
in 1.3 broke store file accounting, which could cause failed long scans,
failed splits and failed snapshots. Many people spent lots and lots of time
fixing various aspects of that, but there are still issues not yet fixed.
Any help in that area would be amazing. This specific issue probably
deserves separate discussion.

Thanks,
Mikhail

On Tue, Aug 8, 2017 at 12:14 AM, Stack  wrote:

> (This came up during dev meeting in Shenzhen) We are running too many
> branches and/or when applying patches, we do not do a good job backporting
> to all active branches, especially fixes.
>
> We have master, branch-2, branch-1, branch-1.4, branch-1.3, branch-1.2, and
> branch-1.1 active currently. If a dirty bug fix, the applier is backporting
> to 7 branches. It takes a while applying to all especially if the backport
> doesn't go in clean. I suppose the RM could monitor all upstream of them
> and then pull wanted patches back (we could do that too) but seems like a
> burden on an RMer.
>
> We should EOL a few?
>
> Nick is on branch-1.1 release at the moment. Perhaps this could be the last
> off branch-1.1.
>
> 1.2 hosts our current stable release though 1.3 is out. How about we cut a
> release off 1.3, call it stable, and then EOL 1.2 after another release or
> so?
>
> What you all think?
>
> Thanks,
> S
>



-- 
Thanks,
Michael Antonov