Re: [openstack-dev] [neutron][devstack] State of the refactor
More updates: https://review.openstack.org/#/q/status:open+project:openstack-dev/devstack+branch:master+topic:neutron-refactor Has the fixes for things I screwed up. Basically it looks like just the _configure_neutron_l3_agent got clobbered which could have broken people using neutron-legacy. I've been watching Zuul all afternoon, but oddly it didn't trigger any breakage in the gate so far. So, hopefully we can clean up my boo-boos quickly and pretend this never happened. -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
I'm seeing some mistakes I made when moving over the contents of neutron-legacy that were l3 related into the new file. It didn't trigger any failures in the gate, but it might impact some developer machines or third party systems. I'm putting together patches, but also considering reverting https://review.openstack.org/168438. Ugh. -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
On 05/11/2016 02:48 PM, Sean M. Collins wrote: > A quick update, https://review.openstack.org/168438 has landed. > > There is a lot of work that needs to be done, Armando and I spoke > yesterday about some of the hooks that are present in neutron-legacy > that both in-tree and out-of-tree consumers rely on being called. > > Doing a quick git grep - we have the following: I'd like to rethink these a bit if we can. > * neutron_plugin_install_agent_packages Is this not just the pre-install or install phase of a plugin? i.e. is there any reason that it can't happen at the normal point. > * neutron_plugin_configure_dhcp_agent neutron_plugin_configure_l3_agent > * neutron_plugin_configure_plugin_agent neutron_plugin_configure_service > * neutron_plugin_setup_interface_driver How are these different from 'post-config' (other than in the neutron case they got micro split) > * neutron_plugin_create_initial_networks I do see a very specific place for 'create_initial_networks' (and it getting called directly in stack.sh is indicative of that fact). So before we go and double the size of the contract can you map out where relative to neutron - pre-install, install, post-config, extra each of these calls currently happen? And why they couldn't just be done by a plugin in said phase. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
A quick update, https://review.openstack.org/168438 has landed. There is a lot of work that needs to be done, Armando and I spoke yesterday about some of the hooks that are present in neutron-legacy that both in-tree and out-of-tree consumers rely on being called. Doing a quick git grep - we have the following: * neutron_plugin_install_agent_packages * neutron_plugin_configure_dhcp_agent neutron_plugin_configure_l3_agent * neutron_plugin_configure_plugin_agent neutron_plugin_configure_service * neutron_plugin_setup_interface_driver * neutron_plugin_create_initial_networks I think that many of these entrypoints most likely need to be formalized in our plugin.sh[1] contract. Perhaps not exactly as they are today, but we do need a phase in the plugin contract where these kinds of things should exist. [1]: http://docs.openstack.org/developer/devstack/plugins.html#plugin-sh-contract -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
On 5 May 2016 at 11:31, Sean M. Collins wrote: > Sean M. Collins wrote: > > Here is the patch I'm using to test the refactor against the Neutron > > CI jobs. > > > > https://review.openstack.org/#/c/278417/ > > > > Here's the test patch to make sure anything that is using the > neutron-legacy file isn't disrupted: > > https://review.openstack.org/#/c/313049/ > > Good news everyone! I didn't break neutron-legacy! Hooray! > I filed [1], to validate that something else besides Neutron doesn't break either. Good job, and kudos to your perseverance...that always pays off in the end (which is in sight)! [1] https://review.openstack.org/#/c/313157/ > -- > Sean M. Collins > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
Where do I propose my new option NEUTRON_MAYBE_L3_MTU_VXLAN_AGENTMODE_FLOATINGIPS ? On Thu, May 5, 2016 at 11:31 AM, Sean M. Collins wrote: > Sean M. Collins wrote: > > Here is the patch I'm using to test the refactor against the Neutron > > CI jobs. > > > > https://review.openstack.org/#/c/278417/ > > > > Here's the test patch to make sure anything that is using the > neutron-legacy file isn't disrupted: > > https://review.openstack.org/#/c/313049/ > > Good news everyone! I didn't break neutron-legacy! Hooray! > > -- > Sean M. Collins > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
On 2016-05-05 15:57:22 + (+), Sean M. Collins wrote: [...] > So we did. > > https://review.openstack.org/168438 [...] Remind me to buy you and Dean beer the next time I see you. It's like you flipped the Etch A Sketch upside down and shook vigorously. <> -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
Sean M. Collins wrote: > Here is the patch I'm using to test the refactor against the Neutron > CI jobs. > > https://review.openstack.org/#/c/278417/ > Here's the test patch to make sure anything that is using the neutron-legacy file isn't disrupted: https://review.openstack.org/#/c/313049/ Good news everyone! I didn't break neutron-legacy! Hooray! -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
Terrific decision! Thank a lot for all this great work. Edgar On 5/5/16, 9:12 AM, "Monty Taylor" wrote: >On 05/05/2016 10:57 AM, Sean M. Collins wrote: >> During the Austin summit, there was a discussion in the QA meetings >> about the future of Neutron's support in DevStack. It's been an ongoing >> effort, and I'd like to step back and give everyone a view of the Big >> Picture. >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__etherpad.openstack.org_p_newton-2Dqa-2Ddevstack-2Droadmap&d=CwICAg&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ&m=rQU0CNTLs59JVaD2xnR4lfhgqi7Bp567V_rp3eglXBU&s=fRKjc7Z8RpVFg2mRBUCannAWvfPenic4UPOIk9Qe3Z8&e= >> >> >> A year ago at the NYC QA midcycle, consensus was reached that in order >> to get Neutron to become the default deployment model for Devstack and >> finally deprecate Nova-Network, some grunt work would need to be done to >> clean up the DevStack code. >> >> Dean wrote about the experience in his blog: >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__hackstack.org_x_blog_2015_03_30_qa-2Dcode-2Dsprint&d=CwICAg&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ&m=rQU0CNTLs59JVaD2xnR4lfhgqi7Bp567V_rp3eglXBU&s=ZlxFE_krg9WPuNtByMvCpfRYDpNrU2H6nr-7bBnNDrY&e= >> >> >>> DevStack's Neutron code is in bad shape. It has been continually >>> updated by many people who do not understand the Big Picture. I blame >>> myself for not staying on top of this code and allowing it to diverge >>>from the rest of DevStack's style and implementation. Other Sean found >>> a number of inconsistencies in variable names and uses as he tried to >>> get the single interface work done and we came to the inevitable >>> conclusion that it was time to start over. >> >> So we did. >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_168438&d=CwICAg&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ&m=rQU0CNTLs59JVaD2xnR4lfhgqi7Bp567V_rp3eglXBU&s=M0LvdXPXlRTXCdEr3sCq4VCoo7pQr7OhD_gv0tHHJf8&e= >> >> >> The new lib/neutron is an attempt to only have the *bare minimum* >> configured for either an OVS or Linux Bridge deployment of Neutron. My >> strategy was - start from scratch and build up enough Neutron >> configuration to get everything to launch, and have the network plumbed >> correctly. Everything else was left on the cutting room floor. >> >> There are still some things that I did for the sake of expediency, that >> I am sure will need to be improved in the future. However, the code is >> very close to completion - there's still some work that needs to be >> done, but I see the light at the end of the tunnel. >> >> For anything that isn't ML2 based, or the LB or OVS agent, Dean and I >> share this view: >> >>> But what about the plugins? What about the advanced services? What >>> about the vendors? I can't do it all at once. The first priority is to >>> get a 'minimum viable Neutron' configuration to use as the default. >>> The old code was renamed not removed, and exactly zero of the >>> configuration variables and service names have changed. Nothing should >>> be directly importing lib/neutron* so that change is handled in >>> DevStack and Grenade already. After we have working linuxbridge and >>> ovs configurations we can look at what else needs to be included. >> >> Sean Dague and I have also been kicking around some ideas about adding >> another hook to the DevStack plugin contract so that DevStack plugins >> that do network things have a chance to create networks and wire >> everything during installation and configuration, as part of a DevStack >> plugin call. >> >> The basic reasoning for this is, the current lib/neutron-legacy has tons >> of knobs for plugging veths into things, creating provider networks, >> etc.. - those things are very specific to a deployment or networking >> type, so they should be moved into a plugin so they don't gunk up the >> main code path and also avoid trampling one another (like they do >> today). >> >> The current lib/neutron weighs in around 500 lines of code, and the l3 >> setup code (which was the nastiest) is 300 lines, split out into a >> separate file. Partially to allow other plugins to call this code, and >> partially because of a divide-and-conquer strategy I am employing for >> the refactor. >> >> If you haven't read my post about eliminating the DevStack layer, this >> is also part of my thought process for the refactor, and how to support >> other configurations in DevStack. >> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_pipermail_openstack-2Ddev_2016-2DApril_091764.html&d=CwICAg&c=DS6PUFBBr_KiLo7Sjt3ljp5jaW5k2i9ijVXllEdOozc&r=G0XRJfDQsuBvqa_wpWyDAUlSpeMV4W1qfWqBfctlWwQ&m=rQU0CNTLs59JVaD2xnR4lfhgqi7Bp567V_rp3eglXBU&s=WDOcyjDzV45GQClN_e5wsY2QKl5Re8wo0iZSEZ_uHsQ&e= >> >> >> So, take a look at what I've done so far, take it for a spin, an
Re: [openstack-dev] [neutron][devstack] State of the refactor
On Thu, May 5, 2016 at 11:12 AM, Monty Taylor wrote: > On 05/05/2016 10:57 AM, Sean M. Collins wrote: >> >> During the Austin summit, there was a discussion in the QA meetings >> about the future of Neutron's support in DevStack. It's been an ongoing >> effort, and I'd like to step back and give everyone a view of the Big >> Picture. >> >> https://etherpad.openstack.org/p/newton-qa-devstack-roadmap >> >> A year ago at the NYC QA midcycle, consensus was reached that in order >> to get Neutron to become the default deployment model for Devstack and >> finally deprecate Nova-Network, some grunt work would need to be done to >> clean up the DevStack code. >> >> Dean wrote about the experience in his blog: >> >> http://hackstack.org/x/blog/2015/03/30/qa-code-sprint >> >>> DevStack's Neutron code is in bad shape. It has been continually >>> updated by many people who do not understand the Big Picture. I blame >>> myself for not staying on top of this code and allowing it to diverge >>> from the rest of DevStack's style and implementation. Other Sean found >>> a number of inconsistencies in variable names and uses as he tried to >>> get the single interface work done and we came to the inevitable >>> conclusion that it was time to start over. >> >> >> So we did. >> >> https://review.openstack.org/168438 >> >> The new lib/neutron is an attempt to only have the *bare minimum* >> configured for either an OVS or Linux Bridge deployment of Neutron. My >> strategy was - start from scratch and build up enough Neutron >> configuration to get everything to launch, and have the network plumbed >> correctly. Everything else was left on the cutting room floor. >> >> There are still some things that I did for the sake of expediency, that >> I am sure will need to be improved in the future. However, the code is >> very close to completion - there's still some work that needs to be >> done, but I see the light at the end of the tunnel. >> >> For anything that isn't ML2 based, or the LB or OVS agent, Dean and I >> share this view: >> >>> But what about the plugins? What about the advanced services? What >>> about the vendors? I can't do it all at once. The first priority is to >>> get a 'minimum viable Neutron' configuration to use as the default. >>> The old code was renamed not removed, and exactly zero of the >>> configuration variables and service names have changed. Nothing should >>> be directly importing lib/neutron* so that change is handled in >>> DevStack and Grenade already. After we have working linuxbridge and >>> ovs configurations we can look at what else needs to be included. >> >> >> Sean Dague and I have also been kicking around some ideas about adding >> another hook to the DevStack plugin contract so that DevStack plugins >> that do network things have a chance to create networks and wire >> everything during installation and configuration, as part of a DevStack >> plugin call. >> >> The basic reasoning for this is, the current lib/neutron-legacy has tons >> of knobs for plugging veths into things, creating provider networks, >> etc.. - those things are very specific to a deployment or networking >> type, so they should be moved into a plugin so they don't gunk up the >> main code path and also avoid trampling one another (like they do >> today). >> >> The current lib/neutron weighs in around 500 lines of code, and the l3 >> setup code (which was the nastiest) is 300 lines, split out into a >> separate file. Partially to allow other plugins to call this code, and >> partially because of a divide-and-conquer strategy I am employing for >> the refactor. >> >> If you haven't read my post about eliminating the DevStack layer, this >> is also part of my thought process for the refactor, and how to support >> other configurations in DevStack. >> >> http://lists.openstack.org/pipermail/openstack-dev/2016-April/091764.html >> >> So, take a look at what I've done so far, take it for a spin, and if you >> have any thoughts or ideas please share them! > > > Everything about this is the best thing that has ever happened since the > dawn of time. > <3 > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
> On May 5, 2016, at 8:57 AM, Sean M. Collins wrote: > > During the Austin summit, there was a discussion in the QA meetings > about the future of Neutron's support in DevStack. It's been an ongoing > effort, and I'd like to step back and give everyone a view of the Big > Picture. > > https://etherpad.openstack.org/p/newton-qa-devstack-roadmap > > A year ago at the NYC QA midcycle, consensus was reached that in order > to get Neutron to become the default deployment model for Devstack and > finally deprecate Nova-Network, some grunt work would need to be done to > clean up the DevStack code. > > Dean wrote about the experience in his blog: > > http://hackstack.org/x/blog/2015/03/30/qa-code-sprint > >> DevStack's Neutron code is in bad shape. It has been continually >> updated by many people who do not understand the Big Picture. I blame >> myself for not staying on top of this code and allowing it to diverge >> from the rest of DevStack's style and implementation. Other Sean found >> a number of inconsistencies in variable names and uses as he tried to >> get the single interface work done and we came to the inevitable >> conclusion that it was time to start over. > > So we did. > > https://review.openstack.org/168438 > > The new lib/neutron is an attempt to only have the *bare minimum* > configured for either an OVS or Linux Bridge deployment of Neutron. My I’d actually suggest just linuxbridge (the install guide default), and make the OVS jobs use whatever mechanism the other plugins will have to use. Thanks, doug > strategy was - start from scratch and build up enough Neutron > configuration to get everything to launch, and have the network plumbed > correctly. Everything else was left on the cutting room floor. > > There are still some things that I did for the sake of expediency, that > I am sure will need to be improved in the future. However, the code is > very close to completion - there's still some work that needs to be > done, but I see the light at the end of the tunnel. > > For anything that isn't ML2 based, or the LB or OVS agent, Dean and I > share this view: > >> But what about the plugins? What about the advanced services? What >> about the vendors? I can't do it all at once. The first priority is to >> get a 'minimum viable Neutron' configuration to use as the default. >> The old code was renamed not removed, and exactly zero of the >> configuration variables and service names have changed. Nothing should >> be directly importing lib/neutron* so that change is handled in >> DevStack and Grenade already. After we have working linuxbridge and >> ovs configurations we can look at what else needs to be included. > > Sean Dague and I have also been kicking around some ideas about adding > another hook to the DevStack plugin contract so that DevStack plugins > that do network things have a chance to create networks and wire > everything during installation and configuration, as part of a DevStack > plugin call. > > The basic reasoning for this is, the current lib/neutron-legacy has tons > of knobs for plugging veths into things, creating provider networks, > etc.. - those things are very specific to a deployment or networking > type, so they should be moved into a plugin so they don't gunk up the > main code path and also avoid trampling one another (like they do > today). > > The current lib/neutron weighs in around 500 lines of code, and the l3 > setup code (which was the nastiest) is 300 lines, split out into a > separate file. Partially to allow other plugins to call this code, and > partially because of a divide-and-conquer strategy I am employing for > the refactor. > > If you haven't read my post about eliminating the DevStack layer, this > is also part of my thought process for the refactor, and how to support > other configurations in DevStack. > > http://lists.openstack.org/pipermail/openstack-dev/2016-April/091764.html > > So, take a look at what I've done so far, take it for a spin, and if you > have any thoughts or ideas please share them! > > -- > Sean M. Collins > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
Yeah, seriously, thanks a million for this. You are making developers lives wa better! 2016-05-05 18:12 GMT+02:00 Monty Taylor : > On 05/05/2016 10:57 AM, Sean M. Collins wrote: > >> During the Austin summit, there was a discussion in the QA meetings >> about the future of Neutron's support in DevStack. It's been an ongoing >> effort, and I'd like to step back and give everyone a view of the Big >> Picture. >> >> https://etherpad.openstack.org/p/newton-qa-devstack-roadmap >> >> A year ago at the NYC QA midcycle, consensus was reached that in order >> to get Neutron to become the default deployment model for Devstack and >> finally deprecate Nova-Network, some grunt work would need to be done to >> clean up the DevStack code. >> >> Dean wrote about the experience in his blog: >> >> http://hackstack.org/x/blog/2015/03/30/qa-code-sprint >> >> DevStack's Neutron code is in bad shape. It has been continually >>> updated by many people who do not understand the Big Picture. I blame >>> myself for not staying on top of this code and allowing it to diverge >>> from the rest of DevStack's style and implementation. Other Sean found >>> a number of inconsistencies in variable names and uses as he tried to >>> get the single interface work done and we came to the inevitable >>> conclusion that it was time to start over. >>> >> >> So we did. >> >> https://review.openstack.org/168438 >> >> The new lib/neutron is an attempt to only have the *bare minimum* >> configured for either an OVS or Linux Bridge deployment of Neutron. My >> strategy was - start from scratch and build up enough Neutron >> configuration to get everything to launch, and have the network plumbed >> correctly. Everything else was left on the cutting room floor. >> >> There are still some things that I did for the sake of expediency, that >> I am sure will need to be improved in the future. However, the code is >> very close to completion - there's still some work that needs to be >> done, but I see the light at the end of the tunnel. >> >> For anything that isn't ML2 based, or the LB or OVS agent, Dean and I >> share this view: >> >> But what about the plugins? What about the advanced services? What >>> about the vendors? I can't do it all at once. The first priority is to >>> get a 'minimum viable Neutron' configuration to use as the default. >>> The old code was renamed not removed, and exactly zero of the >>> configuration variables and service names have changed. Nothing should >>> be directly importing lib/neutron* so that change is handled in >>> DevStack and Grenade already. After we have working linuxbridge and >>> ovs configurations we can look at what else needs to be included. >>> >> >> Sean Dague and I have also been kicking around some ideas about adding >> another hook to the DevStack plugin contract so that DevStack plugins >> that do network things have a chance to create networks and wire >> everything during installation and configuration, as part of a DevStack >> plugin call. >> >> The basic reasoning for this is, the current lib/neutron-legacy has tons >> of knobs for plugging veths into things, creating provider networks, >> etc.. - those things are very specific to a deployment or networking >> type, so they should be moved into a plugin so they don't gunk up the >> main code path and also avoid trampling one another (like they do >> today). >> >> The current lib/neutron weighs in around 500 lines of code, and the l3 >> setup code (which was the nastiest) is 300 lines, split out into a >> separate file. Partially to allow other plugins to call this code, and >> partially because of a divide-and-conquer strategy I am employing for >> the refactor. >> >> If you haven't read my post about eliminating the DevStack layer, this >> is also part of my thought process for the refactor, and how to support >> other configurations in DevStack. >> >> http://lists.openstack.org/pipermail/openstack-dev/2016-April/091764.html >> >> So, take a look at what I've done so far, take it for a spin, and if you >> have any thoughts or ideas please share them! >> > > Everything about this is the best thing that has ever happened since the > dawn of time. > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
On 05/05/2016 10:57 AM, Sean M. Collins wrote: During the Austin summit, there was a discussion in the QA meetings about the future of Neutron's support in DevStack. It's been an ongoing effort, and I'd like to step back and give everyone a view of the Big Picture. https://etherpad.openstack.org/p/newton-qa-devstack-roadmap A year ago at the NYC QA midcycle, consensus was reached that in order to get Neutron to become the default deployment model for Devstack and finally deprecate Nova-Network, some grunt work would need to be done to clean up the DevStack code. Dean wrote about the experience in his blog: http://hackstack.org/x/blog/2015/03/30/qa-code-sprint DevStack's Neutron code is in bad shape. It has been continually updated by many people who do not understand the Big Picture. I blame myself for not staying on top of this code and allowing it to diverge from the rest of DevStack's style and implementation. Other Sean found a number of inconsistencies in variable names and uses as he tried to get the single interface work done and we came to the inevitable conclusion that it was time to start over. So we did. https://review.openstack.org/168438 The new lib/neutron is an attempt to only have the *bare minimum* configured for either an OVS or Linux Bridge deployment of Neutron. My strategy was - start from scratch and build up enough Neutron configuration to get everything to launch, and have the network plumbed correctly. Everything else was left on the cutting room floor. There are still some things that I did for the sake of expediency, that I am sure will need to be improved in the future. However, the code is very close to completion - there's still some work that needs to be done, but I see the light at the end of the tunnel. For anything that isn't ML2 based, or the LB or OVS agent, Dean and I share this view: But what about the plugins? What about the advanced services? What about the vendors? I can't do it all at once. The first priority is to get a 'minimum viable Neutron' configuration to use as the default. The old code was renamed not removed, and exactly zero of the configuration variables and service names have changed. Nothing should be directly importing lib/neutron* so that change is handled in DevStack and Grenade already. After we have working linuxbridge and ovs configurations we can look at what else needs to be included. Sean Dague and I have also been kicking around some ideas about adding another hook to the DevStack plugin contract so that DevStack plugins that do network things have a chance to create networks and wire everything during installation and configuration, as part of a DevStack plugin call. The basic reasoning for this is, the current lib/neutron-legacy has tons of knobs for plugging veths into things, creating provider networks, etc.. - those things are very specific to a deployment or networking type, so they should be moved into a plugin so they don't gunk up the main code path and also avoid trampling one another (like they do today). The current lib/neutron weighs in around 500 lines of code, and the l3 setup code (which was the nastiest) is 300 lines, split out into a separate file. Partially to allow other plugins to call this code, and partially because of a divide-and-conquer strategy I am employing for the refactor. If you haven't read my post about eliminating the DevStack layer, this is also part of my thought process for the refactor, and how to support other configurations in DevStack. http://lists.openstack.org/pipermail/openstack-dev/2016-April/091764.html So, take a look at what I've done so far, take it for a spin, and if you have any thoughts or ideas please share them! Everything about this is the best thing that has ever happened since the dawn of time. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [neutron][devstack] State of the refactor
During the Austin summit, there was a discussion in the QA meetings about the future of Neutron's support in DevStack. It's been an ongoing effort, and I'd like to step back and give everyone a view of the Big Picture. https://etherpad.openstack.org/p/newton-qa-devstack-roadmap A year ago at the NYC QA midcycle, consensus was reached that in order to get Neutron to become the default deployment model for Devstack and finally deprecate Nova-Network, some grunt work would need to be done to clean up the DevStack code. Dean wrote about the experience in his blog: http://hackstack.org/x/blog/2015/03/30/qa-code-sprint >DevStack's Neutron code is in bad shape. It has been continually >updated by many people who do not understand the Big Picture. I blame >myself for not staying on top of this code and allowing it to diverge >from the rest of DevStack's style and implementation. Other Sean found >a number of inconsistencies in variable names and uses as he tried to >get the single interface work done and we came to the inevitable >conclusion that it was time to start over. So we did. https://review.openstack.org/168438 The new lib/neutron is an attempt to only have the *bare minimum* configured for either an OVS or Linux Bridge deployment of Neutron. My strategy was - start from scratch and build up enough Neutron configuration to get everything to launch, and have the network plumbed correctly. Everything else was left on the cutting room floor. There are still some things that I did for the sake of expediency, that I am sure will need to be improved in the future. However, the code is very close to completion - there's still some work that needs to be done, but I see the light at the end of the tunnel. For anything that isn't ML2 based, or the LB or OVS agent, Dean and I share this view: > But what about the plugins? What about the advanced services? What > about the vendors? I can't do it all at once. The first priority is to > get a 'minimum viable Neutron' configuration to use as the default. > The old code was renamed not removed, and exactly zero of the > configuration variables and service names have changed. Nothing should > be directly importing lib/neutron* so that change is handled in > DevStack and Grenade already. After we have working linuxbridge and > ovs configurations we can look at what else needs to be included. Sean Dague and I have also been kicking around some ideas about adding another hook to the DevStack plugin contract so that DevStack plugins that do network things have a chance to create networks and wire everything during installation and configuration, as part of a DevStack plugin call. The basic reasoning for this is, the current lib/neutron-legacy has tons of knobs for plugging veths into things, creating provider networks, etc.. - those things are very specific to a deployment or networking type, so they should be moved into a plugin so they don't gunk up the main code path and also avoid trampling one another (like they do today). The current lib/neutron weighs in around 500 lines of code, and the l3 setup code (which was the nastiest) is 300 lines, split out into a separate file. Partially to allow other plugins to call this code, and partially because of a divide-and-conquer strategy I am employing for the refactor. If you haven't read my post about eliminating the DevStack layer, this is also part of my thought process for the refactor, and how to support other configurations in DevStack. http://lists.openstack.org/pipermail/openstack-dev/2016-April/091764.html So, take a look at what I've done so far, take it for a spin, and if you have any thoughts or ideas please share them! -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron][devstack] State of the refactor
Here is the patch I'm using to test the refactor against the Neutron CI jobs. https://review.openstack.org/#/c/278417/ There is still some work to be done, and it shows. -- Sean M. Collins __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev