On 11/9/2015 10:15 PM, Matthew Treinish wrote:
On Mon, Nov 09, 2015 at 05:05:36PM +1100, Tony Breeds wrote:
On Fri, Nov 06, 2015 at 10:12:21AM -0800, Clint Byrum wrote:

The argument in the original post, I think, is that we should not
stand in the way of the vendors continuing to collaborate on stable
maintenance in the upstream context after the EOL date. We already have
distro vendors doing work in the stable branches, but at EOL we push
them off to their respective distro-specific homes.

That is indeed a better summary than I started with.

I have a half formed idea that creates a state between the current EOL (where
we delete the branches) to what we have today (where we have full CI/VMT/Release

So this idea has come up before and it has been rejected for good reasons. If
vendors want to collaborate, but not to actually work to keep things working in
the gate is that really collaboration? It's just vendors pushing whatever random
backports they want into a shared repo. There is a reason we do all the testing
and it's deeply encoded into the openstack culture. If managing to keep things
verifiably working with the same set of test jobs when a branch was created is
too much of a burden for people then there isn't a reason to keep it around.

This is the crux of why we have shorter branch support windows. No matter how
much complaining people do about how they want LTS releases or longer support
windows, and we'll have world peace when they can use Grizzly with upstream
support for 17 years it doesn't change that barely anyone ever steps up to work
on keeping the gate working on stable branches.

Heh, speaking of Grizzly, I was just looking at backporting https://review.openstack.org/#/c/219301/ to Grizzly yesterday since we still support it. There is an unholy trifecta of things in that change that aren't in Grizzly: nova objects, an API method to get filtered resources from the DB, and a database migration in the change that added the virt driver method used as part of the fix (all added in Havana).

So fixing that in Grizzly is basically going to be a Frankenstein change to avoid the REST API backport and DB migration backport. Good times.

Why am I pointing this out? Just to give an example of the kind of mess that is involved in maintaining a branch that old.

Tony, as someone who has done a good job coming up to speed on fixing issues on 
stable branches you know this firsthand. We're always end up talking to the same
handful of people to debug issues. We're also almost always in firefighting mode
and regular failure rates on stable just keep going up when we look away. People
also burn out quickly debugging these issues all the time. Personally, I know I
don't keep an eye on things nearly as closely as I did before.

There is a massive ammount of trust involved in that simple statement and I 
underestimate that.

This would provide a place where interested vendors can work together.
We could disable grenade in juno which isn't awesome but removes the gotta land
patch in juno, kilo and liberty to unblock the master gate.
We could reduce the VMT impact by nominating a point of contact for juno and
granting that person access to the embargoed bugs.  Similarly we could
trust/delegate a certain about of the release management to that team (I say
team but so far we've only got one volunteer)

We can't ignore the fact that fixing things in Juno *may* still require fixes
in Kilo (and later releases) esp. given the mess that is requirements in

As much as I'd like everyone to get on the CD train, I think it might
make sense to enable the vendors to not diverge, but instead let them
show up with people and commitment and say "Hey we're going to keep
Juno/Mitaka/etc alive!".

So perhaps what would make sense is defining a process by which they can
make that happen.

Note that it's not just backporters though. It's infra resources too.

Sure.  CI and Python 2.6 I have a little understanding of.  I guess I can
extrapolate the additional burden on {system,project}-config.  I willingly
admit I don't have a detailed feel for what I'm asking here.

It's more than just that too, there is additional resources on Tempest and other
QA projects like devstack and grenade too. Tempest is branchless, so to keep
branches around longer you have to have additional jobs running on tempest to
ensure incoming changes work across all releases. In Tokyo we were discussing
doing the same thing on the client repos too because they make similar
guarantees about backwards compat but we never test it. There is a ton of extra
load generated by keeping things around for longer, it's not to be taken
lightly. Especially given the historical lack of contribution in this space.
This is honestly why our 1 experiment in a longer support ended in failure,
nobody stepped up to support the extra branch. To even attempt it again we need
proof that things have improved, which it clearly hasn't.

-Matt Treinish

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe



Matt Riedemann

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Reply via email to