On 11/11/2015 8:51 AM, Flavio Percoco wrote:
On 09/11/15 21:30 -0600, Matt Riedemann wrote:
On 11/9/2015 9:12 PM, Matthew Treinish wrote:
On Mon, Nov 09, 2015 at 10:54:43PM +0000, Kuvaja, Erno wrote:
On Mon, Nov 09, 2015 at 05:28:45PM -0500, Doug Hellmann wrote:
Excerpts from Matt Riedemann's message of 2015-11-09 16:05:29 -0600:
On 11/9/2015 10:41 AM, Thierry Carrez wrote:
Hi everyone,
A few cycles ago we set up the Release Cycle Management team which
was a bit of a frankenteam of the things I happened to be leading:
release management, stable branch maintenance and vulnerability
management.
While you could argue that there was some overlap between those
functions (as in, "all these things need to be released") logic
was not the primary reason they were put together.
When the Security Team was created, the VMT was spinned out of the
Release Cycle Management team and joined there. Now I think we
should spin out stable branch maintenance as well:
* A good chunk of the stable team work used to be stable point
release management, but as of stable/liberty this is now done by
the release management team and triggered by the project-specific
stable maintenance teams, so there is no more overlap in tooling
used there
* Following the kilo reform, the stable team is now focused on
defining and enforcing a common stable branch policy[1], rather
than approving every patch. Being more visible and having more
dedicated members can only help in that very specific mission
* The release team is now headed by Doug Hellmann, who is focused
on release management and does not have the history I had with
stable branch policy. So it might be the right moment to refocus
release management solely on release management and get the stable
team its own leadership
* Empowering that team to make its own decisions, giving it more
visibility and recognition will hopefully lead to more resources
being dedicated to it
* If the team expands, it could finally own stable branch health
and gate fixing. If that ends up all falling under the same roof,
that team could make decisions on support timeframes as well,
since it will be the primary resource to make that work
Isn't this kind of already what the stable maint team does? Well,
that and some QA people like mtreinish and sdague.
So.. good idea ? bad idea ? What do current stable-maint-core[2]
members think of that ? Who thinks they could step up to lead that
team ?
[1]
http://docs.openstack.org/project-team-guide/stable-branches.html
[2] https://review.openstack.org/#/admin/groups/530,members
With the decentralizing of the stable branch stuff in Liberty [1] it
seems like there would be less use for a PTL for stable branch
maintenance - the cats are now herding themselves, right? Or at
least that's the plan as far as I understood it. And the existing
stable branch wizards are more or less around for help and answering
questions.
The same might be said about releasing from master and the release
management team. There's still some benefit to having people
dedicated
to making sure projects all agree to sane policies and to keep up
with
deliverables that need to be released.
Except the distinction is that relmgt is actually producing
something. Relmgt
has the releases repo which does centralize library releases, reno
to do the
release notes, etc. What does the global stable core do? Right now
it's there
almost entirely to just add people to the project specific stable
core teams.
-Matt Treinish
I'd like to move the discussion from what are the roles of the
current stable-maint-core and more towards what the benefits would
be having a stable-maint team rather than the -core group alone.
Personally I think the stable maintenance should be quite a lot more
than unblocking gate and approving people allowed to merge to the
stable branches.
Sure, but that's not we're talking about here are we? The other
tasks, like
backporting changes for example, have been taken on by project teams.
Even in
your other email you mentioned that you've been doing backports and
other tasks
that you consider stable maint in a glance only context. That's
something we
changed in kilo which ttx referenced in [1] to enable that to happen,
and it was
the only way to scale things.
The discussion here is about the cross project effort around stable
branches,
which by design is a more limited scope now. Right now the cross
project effort
around stable branch policy is really 2 things (both of which ttx
already
mentioned):
1. Keeping the gates working on the stable branches
2. Defining and enforcing stable branch policy.
The only lever on #2 is that the global stable-maint-core is the only
group
which has add permissions to the per project stable core groups.
(also the
stable branch policy wiki, but that rarely changes) We specifically
shrunk it to
these 2 things in. [1] Well, really 3 things there, but since we're
not doing
integrated stable point releases in the future its now only 2.
This is my whole argument that creating a new team doesn't do
anything. Living
under rel-mgt, some other project, or creating a new one isn't going
to change
the fact that cross-project stable maint is the same 2 tasks which
basically
nobody cares about, which TBH your email is just an indication of.
Even if we
wanted to grow to something beyond these 2 tasks they would still be
the core of
whatever it becomes and a lack of people caring about them will just
block any
potential scope growth.
Frankly, this is the problem with any ML or open discussion about
stable branch
maint. Everyone has an opinion about everything but lacks actual
context or
never steps up to work on the cross-project side of this. I don't
think creating
a new team that has no repos or other artifacts and therefore no
stackalytics
credit (and by extension corporate chest thumping) is magically going
to create
people actually working on this.
The logical follow on idea to the above is to create a stable branch
policy doc
repo and a project team to own that. But, I still don't think that
solves the
problem, especially since the real priority issue in this space is #1
which
at most is a handful of us bothering to look at that, (well that and
the stable
policy barely ever changes) which honestly really isn't the majority
of the
stable-maint-core group. We also have a similar problem with gate
issues on
master and it's mostly the same people looking at these things there
too.
IMHO the only way for creating a separate project team to be useful
is to
re-centralize more responsibilities of stable maint, which is
something I don't
think we should do because we'll just hit the same scaling issues we
had before
kilo which prompted that change in the first place. It's the same
problem every
horizontal, cross project, or whatever you want to call it effort has
with
OpenStack's growth and why most have moved to the distributed self
service
models.
But, if the goal is just to not make Doug responsible for #2, which
is something
ttx was primarily doing before, then I guess it's fine but we should
be honest
about it and just make ttx the new team leader. :) I honestly don't
think
anything else discussed will change by explicitly making it a
separate team.
-Matt Treinish
[1]
http://lists.openstack.org/pipermail/openstack-dev/2015-November/078
281.html
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Matty T got all P&V in Tokyo! :)
I pretty much agree with everything said above. I don't think we make
many policy changes, the stable maint core team is already pretty
decentralized as it is. We rarely seem to communicate much except when
the (1) gate is wedged or (2) we're coming up on a release and need a
push for reviews.
And this is perhaps one of the problems. I don't think decentralizing
the stable team is enough to have proper stable maintenance. It's a
great step forward and I don't think the proposal is to remove that.
Just like for the releases (especially for projects following
coordinated releases), stable branches could benefit from having
someone looking after them (for those projects that apply to the tag
ttx mentioned in one of his previous emails) and coordinate releases,
status, etc.
I think there's more to it than enforcing policies and keeping the
gate sane. What about projects w/o backports? Are we encouraging ppl
to do enough backports to stable branches?
The small group of gate thumpers raise hell when (1) comes up, and the
stable branch champion coordinates (2).
I've asked this in other threads before, but do we need a PTL for
this? I'm assuming ttx doesn't want this since he's asking for
volunteers. Doug doesn't want it because of release mgmt duties. But
what does a stable branch PTL do? If we have policy changes on stable
that need discussing, can't we come together and discuss those as
needed as a group?
Here's a couple of things that I'd love to see the volunteer for that
job to do:
- Help coordinating stable releases when they are needed.
But isn't this why we decentralized the stable release stuff? So you
don't need a single person or small team herding all the cats?
- Help creating tools to monitor existing backports.
I'm not sure a PTL is needed for this. And I don't think a tool is
needed, we have gerrit. With decentralized stable maint to the projects
(including releasing), if I'm a nova person that cares about stable (and
I am), then I can find out what's open for stable/liberty backports
using gerrit:
- Help creating tools that could help identifying patches that might
be worth backporting.
Again, I'm not sure a PTL is needed for this, just some hackers. I could
see value in writing a script that scans launchpad for things like
'liberty-backport-potential' where the fix is merged in mitaka but not
proposed to stable/liberty.
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/liberty,n,z
Easy peasy. We actually have these same dashboards linked into our
weekly nova meeting agenda just for awareness.
- Be a contact for whenever things break in stable branches and
coordinate with folks that can help fixing those issues.
We have the #openstack-stable channel, people can ask for help there. We
have the mailing list where stable blocking issues are regularly brought
forward to the wider audience. I guess being PTL would mean there is a
lightning rod when something goes bad, but I'd think the majority of the
time they are going to defer to the PTL of whatever project is blocked
and get them to work on it, i.e. if trove stable/liberty is blocked b/c
of some issue in their own CI tooling.
Not sure if all the above points are worth it but I do see this as an
useful role to have.
Cheers,
Flavio
And for gate issues, the people that generally work on those are doing
it because it benefits them, either because it keeps them from having
to maintain forks out of tree or because it gets other changes landed
that are needed for their product/customers. I don't see a new team
making a difference in attracting new people to really dig into the
ugly job of unwedging the gate on stable - it takes quite a bit of
time to get up to speed. Tony is a great example of someone that has
kind of come out of left field and said he wants to help with this and
has stuck with it long enough that he makes a difference and it's
worth spending time mentoring him on the stable gate issues, but those
people are far and few between.
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
