Re: [openstack-dev] [tripleo] Upgrades, Releases & Branches

[Steven Hardy]

On Tue, Aug 18, 2015 at 02:28:39PM -0400, James Slagle wrote:
> On Tue, Aug 18, 2015 at 2:10 PM, Steven Hardy  wrote:
> > On Mon, Aug 17, 2015 at 03:29:07PM -0400, James Slagle wrote:
> >> On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy  wrote:
> >> > Hi all,
> >> >
> >> > Recently I had some discussion with folks around the future strategy for
> >> > TripleO wrt upgrades, releases and branches, specifically:
> >> >
> >> > - How do we support a "stable" TripleO release/branch that enables folks 
> >> > to
> >> >   easily deploy the current stable release of OpenStack
> >> > - Related to the above, how do we allow development of TripleO components
> >> >   (and in particular t-h-t) to proceed without imposing undue constraints
> >> >   on what new features may be used (e.g new-for-liberty Heat features 
> >> > which
> >> >   aren't present in the current released OpenStack version)
> >> > - We're aiming to provide upgrade support, thus from and to which 
> >> > versions?
> >> >
> >> > I know historically TripleO has taken something of a developer and/or
> >> > continuous deployment model for granted, but I'd like to propose that we
> >> > revisit that discusion, such that we move towards something that's more
> >> > consumable by users/operators that are consuming the OpenStack 
> >> > coordinated
> >> > releases.
> >> >
> >> > The obvious answer is a stable branch for certain TripleO components, and
> >> > in particular for t-h-t, but this has disadvantages if we take the
> >> > OpenStack wide "no feature backports" approach - for example "upgrade
> >> > support to liberty" could be considered a feature, and many other TripleO
> >> > "features" are really more about making features of the deployed 
> >> > OpenStack
> >> > services consumable.
> >> >
> >> > I'd like propose we take a somewhat modified "release branch" approach,
> >> > which combines many of the advantages of the stable-branch model, but
> >> > allows for a somewhat more liberal approach to backports, where most 
> >> > things
> >> > are considered valid backports provided they work with the currently
> >> > released OpenStack services (e.g right now, a t-h-t release/kilo branch
> >> > would have to maintain compatibility with a kilo Heat in the undercloud)
> >>
> >> I like the idea, it seems reasonable to me.
> >>
> >> I do think we should clarify if the rule is:
> >>
> >> We *can* backport anything to release/kilo that doesn't break
> >> compatibility with kilo Heat.
> >>
> >> Or:
> >>
> >> We *must* backport anything to release/kilo that doesn't break
> >> compatibility with kilo Heat.
> >
> > I think I was envisaging something closer to the "must", but as Zane said,
> > more a "should", which if automated would become an opt-out thing, e.g
> > through a commit tag "nobackport" or whatever.
> >
> > Ideally, for the upstream branch we should probably be backporting most
> > things which don't break compatibility with the currently released
> > OpenStack services, and don't introduce gratuitous interface changes or
> > other backwards incompatibilities.
> >
> > I know our template "interfaces" are fuzzily defined but here are some
> > ideas of things we might not backport in addition to the "must work with
> > kilo" rule:
> >
> > - Removing parameters or resource types used to hook in external/optional
> >   code (e.g *ExtraConfig etc) - we should advertise these as deprecated via
> >   the descriptions, docs and release notes, then have them removed only
> >   when moving between TripleO releases (same as deprecation policy for most
> >   other projects)
> >
> > - Adding support for new services which either don't exist or weren't
> >   considered stable in the current released version
> >
> >> If it's the former, I think we'd get into a lot of subjective
> >> discussions around if we want certain things backported or not.
> >> Essentially it's the same discussion that happens for stable/*, except
> >> we consider features as well. This could become quite difficult to
> >> manage, and lead to a lot of reviewer opinionated inconsistency into
> >> what actually ends up getting backported.
> >
> > True, but this decision making ends up happening sometime regardless, e.g
> > what patches do you carry in a downstream package etc?  But you're right
> > defining the process early should help with consistency.
> >
> >>
> >> For instance, there could be a very large and disruptive feature that
> >> doesn't break compatibility at all, but some users may not want to see
> >> it in release/kilo. Or, something like the recent proposed patch to
> >> rename a bunch of templates by dropping the "-puppet". That doesn't
> >> break compatibility with a kilo Heat at all, however it could break
> >> compatibility with someone's scripts or external tooling, and might be
> >> a considered an "API" incompatible change. The consuming downstreams
> >> (RDO) may not want to consume such a change. I know we don't have any
> >> 

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[Thierry Carrez]

Steven Hardy wrote:
 The obvious answer is a stable branch for certain TripleO components, and
 in particular for t-h-t, but this has disadvantages if we take the
 OpenStack wide no feature backports approach - for example upgrade
 support to liberty could be considered a feature, and many other TripleO
 features are really more about making features of the deployed OpenStack
 services consumable.
 
 I'd like propose we take a somewhat modified release branch approach,
 which combines many of the advantages of the stable-branch model, but
 allows for a somewhat more liberal approach to backports, where most things
 are considered valid backports provided they work with the currently
 released OpenStack services (e.g right now, a t-h-t release/kilo branch
 would have to maintain compatibility with a kilo Heat in the undercloud)

I think that makes sense.

The critical point being they should *not* be named stable/$SERIES
(which kind of indicates that you follow the stable branch policy), but
something else. release/$SERIES is slightly tainted (used to be the name
of the pre-release branches we created during RCs before release) but I
don't really have better suggestions (deploy/$SERIES ? support/$SERIES ?).

I agree with James that you'll need to clearly describe such branch
policy (everything should be backported ? everything can be backported ?
Only specific things can be backported ?) so that everyone is on the
same page.

Cheers,

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[marios]

On 17/08/15 22:29, James Slagle wrote:
 On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy sha...@redhat.com wrote:
 Hi all,

 Recently I had some discussion with folks around the future strategy for
 TripleO wrt upgrades, releases and branches, specifically:

 - How do we support a stable TripleO release/branch that enables folks to
   easily deploy the current stable release of OpenStack
 - Related to the above, how do we allow development of TripleO components
   (and in particular t-h-t) to proceed without imposing undue constraints
   on what new features may be used (e.g new-for-liberty Heat features which
   aren't present in the current released OpenStack version)
 - We're aiming to provide upgrade support, thus from and to which versions?

 I know historically TripleO has taken something of a developer and/or
 continuous deployment model for granted, but I'd like to propose that we
 revisit that discusion, such that we move towards something that's more
 consumable by users/operators that are consuming the OpenStack coordinated
 releases.

 The obvious answer is a stable branch for certain TripleO components, and
 in particular for t-h-t, but this has disadvantages if we take the
 OpenStack wide no feature backports approach - for example upgrade
 support to liberty could be considered a feature, and many other TripleO
 features are really more about making features of the deployed OpenStack
 services consumable.

 I'd like propose we take a somewhat modified release branch approach,
 which combines many of the advantages of the stable-branch model, but
 allows for a somewhat more liberal approach to backports, where most things
 are considered valid backports provided they work with the currently
 released OpenStack services (e.g right now, a t-h-t release/kilo branch
 would have to maintain compatibility with a kilo Heat in the undercloud)
 
 I like the idea, it seems reasonable to me.
 
 I do think we should clarify if the rule is:
 
 We *can* backport anything to release/kilo that doesn't break
 compatibility with kilo Heat.
 
 Or:
 
 We *must* backport anything to release/kilo that doesn't break
 compatibility with kilo Heat.
 
 If it's the former, I think we'd get into a lot of subjective
 discussions around if we want certain things backported or not.
 Essentially it's the same discussion that happens for stable/*, except
 we consider features as well. This could become quite difficult to
 manage, and lead to a lot of reviewer opinionated inconsistency into
 what actually ends up getting backported.
 
 For instance, there could be a very large and disruptive feature that
 doesn't break compatibility at all, but some users may not want to see
 it in release/kilo. Or, something like the recent proposed patch to
 rename a bunch of templates by dropping the -puppet. That doesn't
 break compatibility with a kilo Heat at all, however it could break
 compatibility with someone's scripts or external tooling, and might be
 a considered an API incompatible change. The consuming downstreams
 (RDO) may not want to consume such a change. I know we don't have any
 official published API for tripleo-heat-templates, I'm just trying
 to think about how people consume the templates, and what they might
 find surprising if they were to be using release/kilo.
 
 The question kind of comes down to if release/kilo is meant to imply
 any stability. Or, if it's just meant to imply that you will always
 be able to deploy OpenStack Kilo with release/kilo of
 tripleo-heat-templates.

yes, my understanding too is that the 'compatibility' guarantee would be
a big part of this idea - one problem we consistently hit in the last
period was not being able to use (latest master of) component x with
(latest master of) tripleo-heat-templates (for example). Using
release/latest across all the projects would mean you'd use the latest
versions of things that have actually been ci'd and seen to work with
each other to give you an OpenStack latest deployment.

 
 I think it's important to decide this up front so we can set the
 expectation. I'm leaning towards the latter (must backport) myself,
 but then I wonder if the release branch would really solve the
 downstream use.

+1 on the must (sorry gfidente, wanted to reply on yours but wanted to
quote slagle in response)

Your point about does it really solve the downstream use still stands in
the can case though (even more so, e.g. one downstream influencing which
patches can go to release branch).

 

 I know in the past stable branches have been discounted due to capacity
 concerns, but the reality atm is such branches are likely to be maintained
 downstream due to release-orientated efforts based on TripleO (e.g
 rdo-manager) anyway, so it could be better for the TripleO community if we
 maintained such branches upstream, where they can be consumed more directly
 by such downstream projects.
 
 I think getting tripleo-ci set up using the release branch would be a
 requirement. When 

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[Zane Bitter]

On 17/08/15 15:29, James Slagle wrote:


I'd like propose we take a somewhat modified release branch approach,
which combines many of the advantages of the stable-branch model, but
allows for a somewhat more liberal approach to backports, where most things
are considered valid backports provided they work with the currently
released OpenStack services (e.g right now, a t-h-t release/kilo branch
would have to maintain compatibility with a kilo Heat in the undercloud)

I like the idea, it seems reasonable to me.

I do think we should clarify if the rule is:

We*can*  backport anything to release/kilo that doesn't break
compatibility with kilo Heat.

Or:

We*must*  backport anything to release/kilo that doesn't break
compatibility with kilo Heat.


I think of it like this: there are theoretically 4 potential models for 
how to combine TripleO and OpenStack:


1) trunk TripleO deploys trunk OpenStack
2) trunk TripleO deploys stable OpenStack
3) stable TripleO deploys trunk OpenStack
4) stable TripleO deploys stable OpenStack

(You could double the size of this list by assuming a user was willing 
to use a different version of OpenStack in the undercloud than they're 
installing in the overcloud. However, I believe we are working on the 
assumption that they are not.)


To date, TripleO has supported only (1). This is a proposal to support 
(2). I'll hazard a guess that nobody cares about (3) right now. This is 
explicitly *not* a proposal to support (4) - i.e. it is not a stable/ 
branch (we'll continue to leave that job to downstream distros).


So we should think of this as a development branch that allows users to 
consume the latest TripleO features without breaking compatibility with 
the stable OpenStack packages. So, in RFC2119 terms, I believe the 
correct policy is we SHOULD backport anything to release/kilo (or 
whatever we call it) that doesn't break compatibility with kilo OpenStack.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[James Slagle]

On Tue, Aug 18, 2015 at 2:10 PM, Steven Hardy sha...@redhat.com wrote:
 On Mon, Aug 17, 2015 at 03:29:07PM -0400, James Slagle wrote:
 On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy sha...@redhat.com wrote:
  Hi all,
 
  Recently I had some discussion with folks around the future strategy for
  TripleO wrt upgrades, releases and branches, specifically:
 
  - How do we support a stable TripleO release/branch that enables folks to
easily deploy the current stable release of OpenStack
  - Related to the above, how do we allow development of TripleO components
(and in particular t-h-t) to proceed without imposing undue constraints
on what new features may be used (e.g new-for-liberty Heat features which
aren't present in the current released OpenStack version)
  - We're aiming to provide upgrade support, thus from and to which versions?
 
  I know historically TripleO has taken something of a developer and/or
  continuous deployment model for granted, but I'd like to propose that we
  revisit that discusion, such that we move towards something that's more
  consumable by users/operators that are consuming the OpenStack coordinated
  releases.
 
  The obvious answer is a stable branch for certain TripleO components, and
  in particular for t-h-t, but this has disadvantages if we take the
  OpenStack wide no feature backports approach - for example upgrade
  support to liberty could be considered a feature, and many other TripleO
  features are really more about making features of the deployed OpenStack
  services consumable.
 
  I'd like propose we take a somewhat modified release branch approach,
  which combines many of the advantages of the stable-branch model, but
  allows for a somewhat more liberal approach to backports, where most things
  are considered valid backports provided they work with the currently
  released OpenStack services (e.g right now, a t-h-t release/kilo branch
  would have to maintain compatibility with a kilo Heat in the undercloud)

 I like the idea, it seems reasonable to me.

 I do think we should clarify if the rule is:

 We *can* backport anything to release/kilo that doesn't break
 compatibility with kilo Heat.

 Or:

 We *must* backport anything to release/kilo that doesn't break
 compatibility with kilo Heat.

 I think I was envisaging something closer to the must, but as Zane said,
 more a should, which if automated would become an opt-out thing, e.g
 through a commit tag nobackport or whatever.

 Ideally, for the upstream branch we should probably be backporting most
 things which don't break compatibility with the currently released
 OpenStack services, and don't introduce gratuitous interface changes or
 other backwards incompatibilities.

 I know our template interfaces are fuzzily defined but here are some
 ideas of things we might not backport in addition to the must work with
 kilo rule:

 - Removing parameters or resource types used to hook in external/optional
   code (e.g *ExtraConfig etc) - we should advertise these as deprecated via
   the descriptions, docs and release notes, then have them removed only
   when moving between TripleO releases (same as deprecation policy for most
   other projects)

 - Adding support for new services which either don't exist or weren't
   considered stable in the current released version

 If it's the former, I think we'd get into a lot of subjective
 discussions around if we want certain things backported or not.
 Essentially it's the same discussion that happens for stable/*, except
 we consider features as well. This could become quite difficult to
 manage, and lead to a lot of reviewer opinionated inconsistency into
 what actually ends up getting backported.

 True, but this decision making ends up happening sometime regardless, e.g
 what patches do you carry in a downstream package etc?  But you're right
 defining the process early should help with consistency.


 For instance, there could be a very large and disruptive feature that
 doesn't break compatibility at all, but some users may not want to see
 it in release/kilo. Or, something like the recent proposed patch to
 rename a bunch of templates by dropping the -puppet. That doesn't
 break compatibility with a kilo Heat at all, however it could break
 compatibility with someone's scripts or external tooling, and might be
 a considered an API incompatible change. The consuming downstreams
 (RDO) may not want to consume such a change. I know we don't have any
 official published API for tripleo-heat-templates, I'm just trying
 to think about how people consume the templates, and what they might
 find surprising if they were to be using release/kilo.

 Yeah, it's a tricky one, I mean the aim of all this is to avoid having to
 maintain a fork downstream, and to improve the experience for folks wanting
 to consume upstream tripleo directly to deploy the coordinated release.

 So IMO we should consider the requirements of both those groups of users -
 some 

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[Steven Hardy]

On Mon, Aug 17, 2015 at 03:29:07PM -0400, James Slagle wrote:
 On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy sha...@redhat.com wrote:
  Hi all,
 
  Recently I had some discussion with folks around the future strategy for
  TripleO wrt upgrades, releases and branches, specifically:
 
  - How do we support a stable TripleO release/branch that enables folks to
easily deploy the current stable release of OpenStack
  - Related to the above, how do we allow development of TripleO components
(and in particular t-h-t) to proceed without imposing undue constraints
on what new features may be used (e.g new-for-liberty Heat features which
aren't present in the current released OpenStack version)
  - We're aiming to provide upgrade support, thus from and to which versions?
 
  I know historically TripleO has taken something of a developer and/or
  continuous deployment model for granted, but I'd like to propose that we
  revisit that discusion, such that we move towards something that's more
  consumable by users/operators that are consuming the OpenStack coordinated
  releases.
 
  The obvious answer is a stable branch for certain TripleO components, and
  in particular for t-h-t, but this has disadvantages if we take the
  OpenStack wide no feature backports approach - for example upgrade
  support to liberty could be considered a feature, and many other TripleO
  features are really more about making features of the deployed OpenStack
  services consumable.
 
  I'd like propose we take a somewhat modified release branch approach,
  which combines many of the advantages of the stable-branch model, but
  allows for a somewhat more liberal approach to backports, where most things
  are considered valid backports provided they work with the currently
  released OpenStack services (e.g right now, a t-h-t release/kilo branch
  would have to maintain compatibility with a kilo Heat in the undercloud)
 
 I like the idea, it seems reasonable to me.
 
 I do think we should clarify if the rule is:
 
 We *can* backport anything to release/kilo that doesn't break
 compatibility with kilo Heat.
 
 Or:
 
 We *must* backport anything to release/kilo that doesn't break
 compatibility with kilo Heat.

I think I was envisaging something closer to the must, but as Zane said,
more a should, which if automated would become an opt-out thing, e.g
through a commit tag nobackport or whatever.

Ideally, for the upstream branch we should probably be backporting most
things which don't break compatibility with the currently released
OpenStack services, and don't introduce gratuitous interface changes or
other backwards incompatibilities.

I know our template interfaces are fuzzily defined but here are some
ideas of things we might not backport in addition to the must work with
kilo rule:

- Removing parameters or resource types used to hook in external/optional
  code (e.g *ExtraConfig etc) - we should advertise these as deprecated via
  the descriptions, docs and release notes, then have them removed only
  when moving between TripleO releases (same as deprecation policy for most
  other projects)

- Adding support for new services which either don't exist or weren't
  considered stable in the current released version

 If it's the former, I think we'd get into a lot of subjective
 discussions around if we want certain things backported or not.
 Essentially it's the same discussion that happens for stable/*, except
 we consider features as well. This could become quite difficult to
 manage, and lead to a lot of reviewer opinionated inconsistency into
 what actually ends up getting backported.

True, but this decision making ends up happening sometime regardless, e.g
what patches do you carry in a downstream package etc?  But you're right
defining the process early should help with consistency.

 
 For instance, there could be a very large and disruptive feature that
 doesn't break compatibility at all, but some users may not want to see
 it in release/kilo. Or, something like the recent proposed patch to
 rename a bunch of templates by dropping the -puppet. That doesn't
 break compatibility with a kilo Heat at all, however it could break
 compatibility with someone's scripts or external tooling, and might be
 a considered an API incompatible change. The consuming downstreams
 (RDO) may not want to consume such a change. I know we don't have any
 official published API for tripleo-heat-templates, I'm just trying
 to think about how people consume the templates, and what they might
 find surprising if they were to be using release/kilo.

Yeah, it's a tricky one, I mean the aim of all this is to avoid having to
maintain a fork downstream, and to improve the experience for folks wanting
to consume upstream tripleo directly to deploy the coordinated release.

So IMO we should consider the requirements of both those groups of users -
some degree of stability probably makes sense, e.g not removing parameters
during the life of the 

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[Steven Hardy]

On Tue, Aug 18, 2015 at 02:28:39PM -0400, James Slagle wrote:
 On Tue, Aug 18, 2015 at 2:10 PM, Steven Hardy sha...@redhat.com wrote:
  On Mon, Aug 17, 2015 at 03:29:07PM -0400, James Slagle wrote:
  On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy sha...@redhat.com wrote:
   Hi all,
  
   Recently I had some discussion with folks around the future strategy for
   TripleO wrt upgrades, releases and branches, specifically:
  
   - How do we support a stable TripleO release/branch that enables folks 
   to
 easily deploy the current stable release of OpenStack
   - Related to the above, how do we allow development of TripleO components
 (and in particular t-h-t) to proceed without imposing undue constraints
 on what new features may be used (e.g new-for-liberty Heat features 
   which
 aren't present in the current released OpenStack version)
   - We're aiming to provide upgrade support, thus from and to which 
   versions?
  
   I know historically TripleO has taken something of a developer and/or
   continuous deployment model for granted, but I'd like to propose that we
   revisit that discusion, such that we move towards something that's more
   consumable by users/operators that are consuming the OpenStack 
   coordinated
   releases.
  
   The obvious answer is a stable branch for certain TripleO components, and
   in particular for t-h-t, but this has disadvantages if we take the
   OpenStack wide no feature backports approach - for example upgrade
   support to liberty could be considered a feature, and many other TripleO
   features are really more about making features of the deployed 
   OpenStack
   services consumable.
  
   I'd like propose we take a somewhat modified release branch approach,
   which combines many of the advantages of the stable-branch model, but
   allows for a somewhat more liberal approach to backports, where most 
   things
   are considered valid backports provided they work with the currently
   released OpenStack services (e.g right now, a t-h-t release/kilo branch
   would have to maintain compatibility with a kilo Heat in the undercloud)
 
  I like the idea, it seems reasonable to me.
 
  I do think we should clarify if the rule is:
 
  We *can* backport anything to release/kilo that doesn't break
  compatibility with kilo Heat.
 
  Or:
 
  We *must* backport anything to release/kilo that doesn't break
  compatibility with kilo Heat.
 
  I think I was envisaging something closer to the must, but as Zane said,
  more a should, which if automated would become an opt-out thing, e.g
  through a commit tag nobackport or whatever.
 
  Ideally, for the upstream branch we should probably be backporting most
  things which don't break compatibility with the currently released
  OpenStack services, and don't introduce gratuitous interface changes or
  other backwards incompatibilities.
 
  I know our template interfaces are fuzzily defined but here are some
  ideas of things we might not backport in addition to the must work with
  kilo rule:
 
  - Removing parameters or resource types used to hook in external/optional
code (e.g *ExtraConfig etc) - we should advertise these as deprecated via
the descriptions, docs and release notes, then have them removed only
when moving between TripleO releases (same as deprecation policy for most
other projects)
 
  - Adding support for new services which either don't exist or weren't
considered stable in the current released version
 
  If it's the former, I think we'd get into a lot of subjective
  discussions around if we want certain things backported or not.
  Essentially it's the same discussion that happens for stable/*, except
  we consider features as well. This could become quite difficult to
  manage, and lead to a lot of reviewer opinionated inconsistency into
  what actually ends up getting backported.
 
  True, but this decision making ends up happening sometime regardless, e.g
  what patches do you carry in a downstream package etc?  But you're right
  defining the process early should help with consistency.
 
 
  For instance, there could be a very large and disruptive feature that
  doesn't break compatibility at all, but some users may not want to see
  it in release/kilo. Or, something like the recent proposed patch to
  rename a bunch of templates by dropping the -puppet. That doesn't
  break compatibility with a kilo Heat at all, however it could break
  compatibility with someone's scripts or external tooling, and might be
  a considered an API incompatible change. The consuming downstreams
  (RDO) may not want to consume such a change. I know we don't have any
  official published API for tripleo-heat-templates, I'm just trying
  to think about how people consume the templates, and what they might
  find surprising if they were to be using release/kilo.
 
  Yeah, it's a tricky one, I mean the aim of all this is to avoid having to
  maintain a fork downstream, and to improve the 

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[Giulio Fidente]

On 08/17/2015 09:29 PM, James Slagle wrote:

On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy sha...@redhat.com wrote:

Hi all,

Recently I had some discussion with folks around the future strategy for
TripleO wrt upgrades, releases and branches, specifically:

- How do we support a stable TripleO release/branch that enables folks to
   easily deploy the current stable release of OpenStack
- Related to the above, how do we allow development of TripleO components
   (and in particular t-h-t) to proceed without imposing undue constraints
   on what new features may be used (e.g new-for-liberty Heat features which
   aren't present in the current released OpenStack version)
- We're aiming to provide upgrade support, thus from and to which versions?

I know historically TripleO has taken something of a developer and/or
continuous deployment model for granted, but I'd like to propose that we
revisit that discusion, such that we move towards something that's more
consumable by users/operators that are consuming the OpenStack coordinated
releases.

The obvious answer is a stable branch for certain TripleO components, and
in particular for t-h-t, but this has disadvantages if we take the
OpenStack wide no feature backports approach - for example upgrade
support to liberty could be considered a feature, and many other TripleO
features are really more about making features of the deployed OpenStack
services consumable.

I'd like propose we take a somewhat modified release branch approach,
which combines many of the advantages of the stable-branch model, but
allows for a somewhat more liberal approach to backports, where most things
are considered valid backports provided they work with the currently
released OpenStack services (e.g right now, a t-h-t release/kilo branch
would have to maintain compatibility with a kilo Heat in the undercloud)


I like the idea, it seems reasonable to me.


/me too

from what I understand this would apply only to the latest stable, the 
previous releases won't get automated updates when a new release is out, 
is this correct?



I do think we should clarify if the rule is:

We *can* backport anything to release/kilo that doesn't break
compatibility with kilo Heat.

Or:

We *must* backport anything to release/kilo that doesn't break
compatibility with kilo Heat.



[...]



I think it's important to decide this up front so we can set the
expectation. I'm leaning towards the latter (must backport) myself,
but then I wonder if the release branch would really solve the
downstream use.


I am for a must as well; a CI job deploying openstack/kilo code using 
the proposed tripleo/master changes might help exposing 
incompatibilities early on



Again, I just go back to the point of the branch. Does it exist to
provide some semblance of stability so that we make distros happy? Or
does it exist solely for the purpose so that we can iterate faster on
using new Heat features in master?


I am not a puppet expert but another reason for the branches could be to 
avoid cross-release incompatibilities due to updates of the 
manifests/modules, not only of the templates


Architectural (incompatible) changes might also benefit from having 
different branches; even though these would remain a hard problem to 
solve in an upgrade scenario



One way to minimise the overhead of maintaing such a branch could be to
implement a bot which automatically proposes commits which land in master
to the branch (using Depends-On to maintain the history order).


I'm not sure I'm following how this would work. Which patch depends on
which other one? If the master patch is A'd, is the release branch
automatically +A'd as well (as long as it's not -2'd)? It seems like
that might be necessary to maintain consistent looking history between
master and the release branch.

Likewise, if the release branch were to fail to merge, it would need
to block the master patch from merging so that there wasn't potential
for different patches to merge out of order in the 2 branches.


yeah looks like the automated process should try to backport the changes 
in the same order they are merged in the master branch and completely 
stop if one of them fails? then assuming manual intervention continue 
from more recent backport?



Interested to hear feedback on the idea, as well as if anyone has direct
experience of implementing the bot/automation pieces.


/me doesn't have experience but would think about the bot as a very 
'stupid' tool rather than an intelligent one; stopping the backports 
seems generally safer than an automated merge which breaks things out of 
immediate sight

--
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] Upgrades, Releases Branches

[Steven Hardy]

Hi all,

Recently I had some discussion with folks around the future strategy for
TripleO wrt upgrades, releases and branches, specifically:

- How do we support a stable TripleO release/branch that enables folks to
  easily deploy the current stable release of OpenStack
- Related to the above, how do we allow development of TripleO components
  (and in particular t-h-t) to proceed without imposing undue constraints
  on what new features may be used (e.g new-for-liberty Heat features which
  aren't present in the current released OpenStack version)
- We're aiming to provide upgrade support, thus from and to which versions?

I know historically TripleO has taken something of a developer and/or
continuous deployment model for granted, but I'd like to propose that we
revisit that discusion, such that we move towards something that's more
consumable by users/operators that are consuming the OpenStack coordinated
releases.

The obvious answer is a stable branch for certain TripleO components, and
in particular for t-h-t, but this has disadvantages if we take the
OpenStack wide no feature backports approach - for example upgrade
support to liberty could be considered a feature, and many other TripleO
features are really more about making features of the deployed OpenStack
services consumable.

I'd like propose we take a somewhat modified release branch approach,
which combines many of the advantages of the stable-branch model, but
allows for a somewhat more liberal approach to backports, where most things
are considered valid backports provided they work with the currently
released OpenStack services (e.g right now, a t-h-t release/kilo branch
would have to maintain compatibility with a kilo Heat in the undercloud)

I know in the past stable branches have been discounted due to capacity
concerns, but the reality atm is such branches are likely to be maintained
downstream due to release-orientated efforts based on TripleO (e.g
rdo-manager) anyway, so it could be better for the TripleO community if we
maintained such branches upstream, where they can be consumed more directly
by such downstream projects.

This could have several benefits:

- TripleO can much more easily offer an easy end-to-end deployment solution
  for released OpenStack versions, e.g you always use release/kilo to
  deploy kilo OpenStack, and in future steps may be defined for upgrading
  between branches when upgrades are implemented and we support upgrading
  e.g from kilo to liberty or whatever.
- RDO (and RDO Manager in particular) can consume the release branches to
  provide package-based TripleO, and effort won't be potentially duplicated
  over downstream efforts, since we maintain the release-orientated TripleO
  components directly upstream.  Obviously this benefits any projects
  downstream in a similar way.

One way to minimise the overhead of maintaing such a branch could be to
implement a bot which automatically proposes commits which land in master
to the branch (using Depends-On to maintain the history order).

Reviewers would then monitor the release/kilo queue, and -2 any changes
which aren't appropriate to backport (e.g use new Heat features or some
other backwards incompatiblity), which would cause the bot to drop the
patch and rebase.  An alternative to this would be a commit message tag
which caused the commit to be picked up (or not).

Obviously there will be merge conflicts which require manual fixup
sometimes (in which case the bot would commit with the conflicts block
intact and propose the patch for manual fixup).

Interested to hear feedback on the idea, as well as if anyone has direct
experience of implementing the bot/automation pieces.

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Upgrades, Releases Branches

[James Slagle]

On Mon, Aug 17, 2015 at 9:28 AM, Steven Hardy sha...@redhat.com wrote:
 Hi all,

 Recently I had some discussion with folks around the future strategy for
 TripleO wrt upgrades, releases and branches, specifically:

 - How do we support a stable TripleO release/branch that enables folks to
   easily deploy the current stable release of OpenStack
 - Related to the above, how do we allow development of TripleO components
   (and in particular t-h-t) to proceed without imposing undue constraints
   on what new features may be used (e.g new-for-liberty Heat features which
   aren't present in the current released OpenStack version)
 - We're aiming to provide upgrade support, thus from and to which versions?

 I know historically TripleO has taken something of a developer and/or
 continuous deployment model for granted, but I'd like to propose that we
 revisit that discusion, such that we move towards something that's more
 consumable by users/operators that are consuming the OpenStack coordinated
 releases.

 The obvious answer is a stable branch for certain TripleO components, and
 in particular for t-h-t, but this has disadvantages if we take the
 OpenStack wide no feature backports approach - for example upgrade
 support to liberty could be considered a feature, and many other TripleO
 features are really more about making features of the deployed OpenStack
 services consumable.

 I'd like propose we take a somewhat modified release branch approach,
 which combines many of the advantages of the stable-branch model, but
 allows for a somewhat more liberal approach to backports, where most things
 are considered valid backports provided they work with the currently
 released OpenStack services (e.g right now, a t-h-t release/kilo branch
 would have to maintain compatibility with a kilo Heat in the undercloud)

I like the idea, it seems reasonable to me.

I do think we should clarify if the rule is:

We *can* backport anything to release/kilo that doesn't break
compatibility with kilo Heat.

Or:

We *must* backport anything to release/kilo that doesn't break
compatibility with kilo Heat.

If it's the former, I think we'd get into a lot of subjective
discussions around if we want certain things backported or not.
Essentially it's the same discussion that happens for stable/*, except
we consider features as well. This could become quite difficult to
manage, and lead to a lot of reviewer opinionated inconsistency into
what actually ends up getting backported.

For instance, there could be a very large and disruptive feature that
doesn't break compatibility at all, but some users may not want to see
it in release/kilo. Or, something like the recent proposed patch to
rename a bunch of templates by dropping the -puppet. That doesn't
break compatibility with a kilo Heat at all, however it could break
compatibility with someone's scripts or external tooling, and might be
a considered an API incompatible change. The consuming downstreams
(RDO) may not want to consume such a change. I know we don't have any
official published API for tripleo-heat-templates, I'm just trying
to think about how people consume the templates, and what they might
find surprising if they were to be using release/kilo.

The question kind of comes down to if release/kilo is meant to imply
any stability. Or, if it's just meant to imply that you will always
be able to deploy OpenStack Kilo with release/kilo of
tripleo-heat-templates.

I think it's important to decide this up front so we can set the
expectation. I'm leaning towards the latter (must backport) myself,
but then I wonder if the release branch would really solve the
downstream use.


 I know in the past stable branches have been discounted due to capacity
 concerns, but the reality atm is such branches are likely to be maintained
 downstream due to release-orientated efforts based on TripleO (e.g
 rdo-manager) anyway, so it could be better for the TripleO community if we
 maintained such branches upstream, where they can be consumed more directly
 by such downstream projects.

I think getting tripleo-ci set up using the release branch would be a
requirement. When I didn't do this previously with the stable branches
attempt, that's when they ended up getting stale and cumbersome to
maintain.


 This could have several benefits:

 - TripleO can much more easily offer an easy end-to-end deployment solution
   for released OpenStack versions, e.g you always use release/kilo to
   deploy kilo OpenStack, and in future steps may be defined for upgrading
   between branches when upgrades are implemented and we support upgrading
   e.g from kilo to liberty or whatever.
 - RDO (and RDO Manager in particular) can consume the release branches to
   provide package-based TripleO, and effort won't be potentially duplicated
   over downstream efforts, since we maintain the release-orientated TripleO
   components directly upstream.  Obviously this benefits any projects
   downstream in a similar way.