On 06/17/2015 03:08 PM, Doug Hellmann wrote: > Excerpts from Sean Dague's message of 2015-06-17 14:07:35 -0400: >> On 06/17/2015 01:29 PM, Clint Byrum wrote: >>> Excerpts from Sean Dague's message of 2015-06-16 10:16:34 -0700: >>>> On 06/16/2015 12:49 PM, Clint Byrum wrote: >>>>> Excerpts from Sean Dague's message of 2015-06-16 06:22:23 -0700: >>>>>> FYI, >>>>>> >>>>>> One of the things that came out of the summit for Devstack plans going >>>>>> forward is to trim it back to something more opinionated and remove a >>>>>> bunch of low use optionality in the process. >>>>>> >>>>>> One of those branches to be trimmed is all the support for things beyond >>>>>> RabbitMQ in the rpc layer. RabbitMQ is what's used by 95%+ of our >>>>>> community, that's what the development environment should focus on. >>>>>> >>>>>> The patch to remove all of this is here - >>>>>> https://review.openstack.org/#/c/192154/. Expect this to merge by the >>>>>> end of the month. If people are interested in non RabbitMQ external >>>>>> plugins, now is the time to start writing them. The oslo.messaging team >>>>>> already moved their functional test installation for alternative >>>>>> platforms off of devstack, so this should impact a very small number of >>>>>> people. >>>>>> >>>>> >>>>> The recent spec we added to define a policy for oslo.messaging drivers is >>>>> intended as a way to encourage that 5% who feels a different messaging >>>>> layer is critical to participate upstream by adding devstack-gate jobs >>>>> and committing developers to keep them stable. This change basically >>>>> slams the door in their face and says "good luck, we don't actually care >>>>> about accomodating you." This will drive them more into the shadows, >>>>> and push their forks even further away from the core of the project. If >>>>> that's your intention, then we need to have a longer conversation where >>>>> you explain to me why you feel that's a good thing. >>>> >>>> I believe it is not the responsibility of the devstack team to support >>>> every possible backend one could imagine and carry that technical debt >>>> in tree, confusing new users in the process that any of these things >>>> might actually work. I believe that if you feel that your spec assumed >>>> that was going to be the case, you made a large incorrect externalities >>>> assumption. >>>> >>> >>> I agree with you, and support your desire to move things into plugins. >>> >>> However, your timing is problematic and the lack of coordination with >>> the ongoing effort to deprecate untested messaging drivers gracefully >>> is really frustrating. We've been asking (on this list) zmq interested >>> parties to add devstack-gate jobs and identify themselves as contacts >>> to support these drivers. Meanwhile this change and the wording around >>> it suggest that they're not welcome in devstack. >> >> So there has clearly been some disconnect here. This patch was >> originally going to come later in the cycle, but some back and forth on >> proton fixes with Flavio made me realize we really needed to get this >> direction out in front of more people (which is why it wasn't just a >> patch, it was also an email heads up). So there wasn't surprise when it >> was merged. >> >> We built the external plugin mechanism in devstack to make it very easy >> to extend out of tree, and make it easy to let people consume your out >> of tree stuff. It's the only way that devstack works in the big tent >> world, because there just is too much stuff for the team to support. > > Every change like this makes it harder for newcomers to participate. > Frankly, it makes it harder for everyone because it means there are > more moving parts, but in this specific case many of the people > involved in these messaging drivers are relatively new, so I point > that out. The already difficult task of setting up sufficient > functional tests has now turned into "figure out devstack", too. > The long-term Oslo team members can't do all of this work, any more > than the devstack team can, but things were at least working in > what we thought was a stable way so we could try to provide guidance. > >> >>>>> Also, I take issue with the value assigned to dropping it. If that 95% >>>>> is calculated as orgs_running_on_rabbit/orgs then it's telling a really >>>>> lop-sided story. I'd rather see compute_nodes_on_rabbit/compute_nodes. >>>>> >>>>> I'd like to propose that we leave all of this in tree to match what is >>>>> in oslo.messaging. I think devstack should follow oslo.messaging and >>>>> deprecate the ones that oslo.messaging deprecates. Otherwise I feel like >>>>> we're Vizzini cutting the rope just as The Dread Pirate 0mq is about to >>>>> climb the last 10 meters to the top of the cliffs of insanity and battle >>>>> RabbitMQ left handed. I know, "inconceivable" right? >>>> >>>> We have an external plugin mechanism for devstack. That's a viable >>>> option here. People will have to own and do that work, instead of >>>> expecting the small devstack team to do that for them. I believe I left >>>> enough of a hook in place that it's possible. >>>> >>> >>> So lets do some communication, and ask for the qpid and zmq people to >>> step up, and help them move their code into an external plugin, and add >>> documentation to help their users find it. The burden should shift, but >>> it still rests with devstack until it _does_ shift. >> >> We still need to set a clock, because in the past when we haven't, the >> burden never shifts. > > In the past when we've made big changes like this, we've treated > them as deprecations with long lead times. So, how about the beginning > of the N cycle (or end of M)? That gives the teams time to ramp up > and understand what's going on within Oslo, before having to figure > out the whole world in order to meet the publicly discussed testing > requirements they already knew about. > >> >>>> That would also let them control the code relevant to their plugin, >>>> because there is no way that devstack was going to gate against other >>>> backends here, so we'd end up breaking them pretty often, and it taking >>>> a while to fix them in tree. >>> >>> I love that idea. That is not what the change does though. It deletes >>> with nary a word about what users of this code should do until new >>> external plugins appear. >> >> Sure, lets get folks engaged now. I'm happy to help people debug this >> code to get it working. The burden of the effort does need to be on the >> folks with the feature they want to easily enable in devstack, but we've >> spent a lot of time getting this stuff working for a lot of different >> use cases. It should only be a couple of days of poking, even for >> someone pretty new at this. > > Does the existing code in devstack not work? Is retaining it > preventing other work the devstack team wants to undertake? Does > this have to be done *right now*? Because, as Clint pointed out, the > timing is really bad for the people your change affects. > >> If we have a "pretty close" attempt by a team, I'll happily work out any >> last mile problems. That actually worked out really great with the >> modular grenade / heat effort, as all the last mile bits got sorted in 2 >> hrs time because there was mostly functional bits on both sides of the >> fence. > > Is there documentation for the new plugin system? Are there examples > available?
Yes - http://docs.openstack.org/developer/devstack/plugins.html And yes: * https://github.com/stackforge/ec2-api/tree/master/devstack * https://github.com/openstack/trove/tree/master/devstack * https://github.com/openstack/zaqar/tree/master/devstack * https://github.com/openstack/gnocchi/tree/master/devstack * https://git.openstack.org/openstack/dragonflow * https://github.com/openstack/networking-odl/tree/master/devstack * https://github.com/openstack/magnum/tree/master/devstack And a few more I could dig up. Projects have been running this way as gating jobs since Feb (I think ec2-api was the first, that commit is Feb 7th). We should bring links to all the examples back into the devstack tree, just haven't gotten around to it yet. -Sean -- Sean Dague http://dague.net __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev