On 08/20/2014 12:37 PM, Zane Bitter wrote: > On 11/08/14 05:24, Thierry Carrez wrote: >> So the idea that being (and remaining) in the integrated release should >> also be judged on technical merit is a slightly different effort. It's >> always been a factor in our choices, but like Devananda says, it's more >> difficult than just checking a number of QA/integration checkboxes. In >> some cases, blessing one project in a problem space stifles competition, >> innovation and alternate approaches. In some other cases, we reinvent >> domain-specific solutions rather than standing on the shoulders of >> domain-specific giants in neighboring open source projects. > > I totally agree that these are the things we need to be vigilant about. > > Stifling competition is a big worry, but it appears to me that a lot of > the stifling is happening even before incubation. Everyone's time is > limited, so if you happen to notice a new project on the incubation > trajectory doing things in what you think is the Wrong Way, you're most > likely to either leave some drive-by feedback or to just ignore it and > carry on with your life. What you're most likely *not* to do is to start > a competing project to prove them wrong, or to jump in full time to the > existing project and show them the light. It's really hard to argue > against the domain experts too - when you're acutely aware of how > shallow your knowledge is in a particular area it's very hard to know > how hard to push. (Perhaps ironically, since becoming a PTL I feel I > have to be much more cautious in what I say too, because people are > inclined to read too much into my opinion - I wonder if TC members feel > the same pressure.) I speak from first-hand instances of guilt here - > for example, I gave some feedback to the Mistral folks just before the > last design summit, but I haven't had time to follow it up at all. I > wouldn't be a bit surprised if they showed up with an incubation > request, a largely-unchanged user interface and an expectation that I > would support it. > > The result is that projects often don't hear the feedback they need > until far too late - often when they get to the incubation review (maybe > not even their first incubation review). In the particularly unfortunate > case of Marconi, it wasn't until the graduation review. (More about that > in a second.) My best advice to new projects here is that you must be > like a ferret up the pant-leg of any negative feedback. Grab hold of any > criticism and don't let go until you have either converted the person > giving it into your biggest supporter, been converted by them, or > provoked them to start a competing project. (Any of those is a win as > far as the community is concerned.) > > Perhaps we could consider a space like a separate mailing list > (openstack-future?) reserved just for announcements of Related projects, > their architectural principles, and discussions of the same? They > certainly tend to get drowned out amidst the noise of openstack-dev. > (Project management, meeting announcements, and internal project > discussion would all be out of scope for this list.) > > As for reinventing domain-specific solutions, I'm not sure that happens > as often as is being made out. IMO the defining feature of IaaS that > makes the cloud the cloud is on-demand (i.e. real-time) self-service. > Everything else more or less falls out of that requirement, but the very > first thing to fall out is multi-tenancy and there just aren't that many > multi-tenant services floating around out there. There are a couple of > obvious strategies to deal with that: one is to run existing software > within a tenant-local resource provisioned by OpenStack (Trove and > Sahara are examples of this), and the other is to wrap a multi-tenancy > framework around an existing piece of software (Nova and Cinder are > examples of this). (BTW the former is usually inherently less > satisfying, because it scales at a much coarser granularity.) The answer > to a question of the form: > > "Why do we need OpenStack project $X, when open source project $Y > already exists?" > > is almost always: > > "Because $Y is not multi-tenant aware; we need to wrap it with a > multi-tenancy layer with OpenStack-native authentication, metering and > quota management. That even allows us to set up an abstraction layer so > that you can substitute $Z as the back end too." > > This is completely uncontroversial when you substitute X, Y, Z = Nova, > libvirt, Xen. However, when you instead substitute X, Y, Z = > Zaqar/Marconi, Qpid, MongoDB it suddenly becomes *highly* controversial. > I'm all in favour of a healthy scepticism, but I think we've passed that > point now. (How would *you* make an AMQP bus multi-tenant?) > > To be clear, Marconi did made a mistake. The Marconi API presented > semantics to the user that excluded many otherwise-obvious choices of > back-end plugin (i.e. Qpid/RabbitMQ). It seems to be a common thing (see > also: Mistral) to want to design for every feature an existing > Enterprisey application might use, which IMHO kind of ignores the fact > that applications need to be rewritten to use these new APIs anyway, and > we would be better off presenting _simple_ interfaces that attract > developers, lead to good design for new applications and provide > flexibility on the back-end side. I digress though, because that wasn't > the mistake. The mistake was failing to educate the entire community > (but in particular the TC) on the relative merits of this feature enough > get either buy-in or rejection on this critical detail much earlier in > the process. > > By the way, I claimed without justification earlier that "there just > aren't that many multi-tenant services floating around out there", but > there is actually a very good reason for that. Before there were open > source IaaS platforms out there, there wasn't any reason to build such a > service. And now that there are, _the obvious place to build such a > service in in OpenStack_. We have the users, we have the authentication > API that already ties in to their existing directories (Keystone is > well-named) and we have the community.
I think this is a very crisp view of what OpenStack is trying to do. I like it. :) >> This all has created a world where you need to be*in* OpenStack to >> matter, or to justify the investment. This has created a world where >> everything and everyone wants to be in the "OpenStack" integrated >> release. This has created more pressure to add new projects, and less >> pressure to fix and make the existing projects perfect. 4 years in, we >> might want to inflect that trajectory and take steps to fix this world. > > We should certainly consider this possibility, that we've set up > perverse incentives leading to failure. But what if it's just because we > haven't yet come even close to satisfying all of our users' needs? I > mean, AWS has more than 30 services that could be considered equivalent > in scope to an OpenStack project... if anything our scope is increasing > more _slowly_ than the industry at large. I'm slightly shocked that > nobody in this thread appears to have even entertained the idea that > *this is what success looks like*. That's a fair point of view. However I'm sure the AWS team is staffed a bit differently where the feature implementers don't outnumber the QA/Docs/Infra folks by a factor for 10 to 1. I'm also sure there are a lot more dedicated folks to cross project effort than we see today. If we weren't so feature biased in our effort, 30 services might be a sensible target. But the lack of folks working on some of the holistic / cross project efforts means I don't think it's really viable. > The world is not going to stop because we want to get off, take a > breather, do a "consolidation cycle". Most deployers I've talked to would actually love for OpenStack to slow down a bit. They are at least a year behind on their deployments, and a stabalization cycle is something they'd lvove to see. > I think we can all agree that any cross-project function program that > aims to _be_ that function for all of OpenStack, as opposed to enabling > that function in the individual projects, will not scale. It won't scale > even if we never accept another new project. It won't scale even if we > de-integrate some projects pour encourager les autres. I think that the > work that Doug has been doing with Oslo points the way here. Every > cross-project function (including release management - get rid of PTLs) > should have a designated co-ordinator/liason in each project. This becomes an N^2 communication problem and eventually consistency problem. > The move of functional tests out of Tempest and into the individual > projects' repositories is a great step in the right direction. Hopefully > that will also address the growth of CI testing capacity requirements - > while projects like Nova, Cinder and Neutron all depend on and benefit > from being gated against each other, that is not at all true for many > projects, and in particular the kinds of new projects that are now > appearing. Let's take actual dependencies into account when we choose > what to gate on - the resulting growth might not be sublinear, but it > needn't be n^2 either. > > Simply put, I don't think we're anywhere near so close to running out of > ideas that we have to stop and freeze the world. We have scaling > challenges, and we'll have to make changes, and we'll have to do it - > sometimes under pressure - even as the world changes around us because > that's just how life works. > > The question that is not being asked enough IMO is how exactly does it > benefit our *users* (it is still all about the users, right?) to be > telling developers "you arrived too late, so you're not real OpenStack > developers, with free conference tickets and ATC badges and stuff". > Because if it's just that the support functions within OpenStack can't > handle them yet, that's on us not them. If it's that we don't have a > clear way of communicating to users the information they need to make > the right decision (for them) on whether to deploy some code, then we > need to communicate that information more clearly. And if we believe > that somehow companies will magically divert all of the resources they > would otherwise have committed to new projects into bug fixing on > existing projects, I think we're being incredibly naive. If companies > are not seeing the value in strategically contributing to the core of > projects then it's because we have failed to demonstrate the value, and > simply blocking off other avenues for them to create value (or blocking > recognition of those avenues) will absolutely not solve it. I also think it's completely fair to think about this as small tent, large ecosystem. The Linux Kernel stops at a certain function point for a reason. That didn't prevent a ton of exceptionally useful function to exist above it. But it also mean that things like c library / init system / packaging / compilers were things where innovation could play out. And where things could evolve over time. I don't think small test / big ecosystem prevents anyone from working on what they are passionate about. It does bring the value of n in n^2 relationships to something tractable. Maybe there are better people to handle the cross project interactions that can handle the value of n we currently have, or the value of n we might have in the future. But I'm watching the strain on the existing teams doing this and I think under current course and speed we mostly will lose key leadership (under burn out), and not grow more of it. And that's a really important thing to consider in keeping something like OpenStack a functioning base. If the replacement rate of key leadership does not exceed the fall off rate we won't end up in a happy place in a few more cycles. -Sean -- Sean Dague http://dague.net _______________________________________________ OpenStack-dev mailing list OpenStackemail@example.com http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev