On 11/10/2016 6:15 AM, Chris Dent wrote:
On Tue, 8 Nov 2016, Chris Friesen wrote:
That said, I don't know that there's an easy solution to this in nova
due to the fact that it's a distributed system with a central shared
data store. Enhancing a sched filter might require new data, which
means modifying the DB model and the DB migrations, and updating the
InstanceNUMATopology object and tweaking nova-api to request
additional attrs, and it might have implications for upgrade, etc.
The symptoms that you are describing here are pretty much the crux
of the biscuit.
I really hesitate to even respond to this but I feel the need to, but at
the same time feel as though I'm going to regret it, and it won't help
change anyone's mind, and I'll probably get in trouble from the
Foundation or TC or some other group that monitors what PTLs/cores/etc
say in public. But here goes!
As much as I think having additional cores or subsystem maintainers
with +2 but not +W would probably be useful, I don't think it will
do much to fix the fundamental problem -- the crux above -- so the
concerns that people have about the risks will continue, will continue
to be valid and can't just be dismissed.
That's a technology crux. There's also a social crux.
It was very clear in BCN that there is significant disagreement in
the community over the volume of code that nova ought to be accepting.
Even in an ideal world where a lot of the roadblocks that people are
experiencing aren't there, there would be disagreement between at least
two (idealized, not literal) camps: One that says any code which doesn't
break stuff and is vaguely in the domain of what nova is about should be
merged and another that says nova should have a clear and focused
vision and contributions that don't fit that vision should be rejected.
Believe it or not, those of us that say no to things aren't super happy
saying no to people all of the time. It gets pretty soul-crushing after
awhile. However, for those of us that have worked on Nova for several
years and have seen where saying, 'sure, what the hell, this magical
little vendor snowflake can't possibly harm us, right? btw, don't worry
about maintaining this or CI testing it...we'll trust you to do that out
of tree and report/fix those problems 4 years from now' - that creates
an innate hesitation to more and more magical unicorn vendor-specific
changes. Because it's those kinds of things that get in the way of us
being able to make broader necessary changes later, like with the
placement service/scheduler stuff, and capabilities in the API. With so
much random custom use case code, it's really hard to know what changes
will break something else you thought was unrelated.
Also, we have consistently had operators give feedback at summits that
basically say, 'nova is the most boring project in my deployment, and
that's a good thing' - that's about the best compliment we can get:
stability matters.
And for that matter, the OpenStack community has said that
interoperability matters, and as most have seen we've been working on
removing plugin and extension points over the last several releases to
get to that end. Having said that, it's an entirely whole other topic
unto itself and I have mixed feelings about it personally, i.e. if I'm
building a private devops cloud and don't care about
defcore/refstack/interoperability needed for marketing a product, then
what's wrong with me having configuration and extension point freedom
coming out of my ears? That's a valid point, I understand that. I also
understand that we don't test any of that stuff upstream and we break
it, and then get people shouting at us for breaking things that broke
their snowflake configuration.
At the moment both of these cruxes seem to be treated as things that
are too hard to change, or at least too hard discuss.
Chris, I keep hearing this, especially from you. I don't know what needs
to happen to move past this. We talked about this at length at the BCN
summit. We talked about new contributors and ways to help there at the
ATX summit. So it's not like no one is willing to discuss these things -
we obviously have already. But saying things like 'no one is willing to
even discuss this' is really frustrating to continue hearing, and what's
even more frustrating is I get the feeling that if I even mention that,
it's going to somehow justify the point. "See, he said he's tired of
talking about this, which means he doesn't want to talk about it."
On the tech crux some people claim that the decomposition (and the
related boundary hardening) that would lead to safe contracts between
subsystems have already gone as far as they can or taking them
further requires such a huge amount of effort that given resources
and other commitments it can't happen. That smacks of a lack of
imagination and hope.
I also don't agree with this. I think the sentiment at the summit, if
we're talking about the same thing, was that we've done a lot of
boundary hardening already. There is obviously room for improvement -
but to get there, you're going to have to wade through a LOT of custom
code added by vendors in years past and make sure not to break that.
That's totally fine as a goal, but are you personally going to take up
that challenge and start working on it? If not, then I care less about
hearing about it over and over and continuing to talk about this. We
have priorities each release, and we have several cores working on those
priorities, and with the HPE and Mirantis layoffs we have even fewer
dedicated people able to work on some of these bigger sweeping changes -
and those people had the background on what dragons lay in wait when you
take on some of these big systematic changes. So the point is we have to
prioritize, and I put out of tree enablement less of a priority compared
to the other things we need to get done that operators actually care about.
My main point here is, if you see the opportunity and justification for
the work to completely change something, then by all means, work on that
and keep everyone involved. But I'm tired of hearing how unwilling
people in nova, and by people of course I mean the core team, are to
listening to ideas for positive change and then being thrown under the
bus when we don't have all of the time or priority to nurture all of
those changes.
On the social crux, the impression I got is that people are either
too exhausted, too busy, too angry, or too scared to actually talk
things out and reach a conclusion and instead bitch behind each other's
backs about the lack of agreement and some other core who will either
reject or accept code they shouldn't be. We need to be honest about
this lack of trust if we want to make it better. And we need resolve
the vision of the project, in a concrete fashion.
What's really baffling to me is when I'm in the hallway with you and
Matt Booth and others talking about this stuff face to face in person,
and explaining my side of the issue, and I'm listening to yours, we seem
to all be nodding our heads and agreeing, or at least understanding a
bit better. But then when we get back to the world it's all us vs them
and 'a lot of people are saying nova sucks and they are ready to give up'.
First, the 'speaker of the people' thing gets tiring. We're all working
on the same thing. If people have issues I welcome them to air it out,
but they need to listen to and understand the reply, and try to
understand why we have to say no to things at times, or that we can't
prioritize something.
Second, at least from my view, I get the feeling that there are a few
people beating this drum and complaining about not getting enough
attention and we need to stop everything we're doing whenever they say
something and I just have a hard time with that. What I value most in
nova contributors are people that are doing the dirty work, like
triaging and fixing bugs, fixing technical debt, trying to take on the
dirtier issues to help us move forward, and probably most of all - lots
of quality code reviews. At the end of the day quality code reviews are
going to make you core, and that's what's going to move code through the
system. But people would rather complain about not being core, or the
lack of cores, than actually bust their ass to help out and make core.
Until at least one of those cruxes is resolved it's going to be
really hard to make substantial progress and many of the strategies
we try are going to be tactical band-aids.
mdbooth's proposal is somewhat different from some of the other
options because implementing it requires a leap of faith over
both cruxes into what could be a self-reinforcing environment: we will
trust one another because we've said we will, and we will build the
stronger contracts because we need them in order to be able to trust
people, and we will talk about it all (better than we do now) because it
won't work if we don't.
That might work. It could be a mess, but it is better than an
alternative which is feeling increasingly likely: People go elsewhere.
And finally, the hostage taking approach of everyone going elsewhere is
really wearing on me. If developers have a choice of projects to work on
in OpenStack and for whatever reason find it impossible to work in Nova,
then I frankly think they should move on, for the benefit of both parties.
You know other reasons that people are going to go elsewhere? Because
their company took them off OpenStack, or fired them, because they were
working on OpenStack and they decided as a technology it didn't work
well enough for them and they are going to invest resources elsewhere.
That's why I find it a major distraction to be talking about all of the
ideals of how things would work when instead we have a ton of work to
already get done and we're understaffed, and the future is unsure
regardless - so while I'm here I want to get the major changes in that
we've been working on for years as a project, like cells v2, placement,
and moving off nova-network.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
There, now I feel equally terrible and relieved. I really don't want to
be anyone's barrier, and I feel great when I'm able to help someone out,
new contributor or not. I don't really know what else to say. I know all
of this probably doesn't mean anything and there will be a massive reply
thread discounting each point and exquisite detail about how I'm wrong -
that's fine, it's what I get for taking the bait to respond in detail to
something like this. But I felt the need to do so.
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
