Re: [openstack-dev] [Nova] Things to tackle in Liberty
True. For example, the infiniband passthrough blueprint might need port type info from neutron-nova? Thanks, Kevin From: Neil Jerram [neil.jer...@metaswitch.com] Sent: Friday, April 17, 2015 9:44 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [Nova] Things to tackle in Liberty On 17/04/15 17:24, Daniel P. Berrange wrote: On Fri, Apr 17, 2015 at 12:16:25PM -0400, Jay Pipes wrote: On 04/10/2015 11:48 AM, Neil Jerram wrote: What I imagine, though, is that the _source_ of the plugging information could move from Nova to Neutron, so that future plugging-related code changes are a matter for Neutron rather than for Nova. The plugging would still _happen_ from within Nova, as directed by that information. -1. One of the biggest problems I have with the current implementation for Nova VIF types is that stuff leaks improperly out of the Neutron API. Take a look at all the Contrail implementation specifics in here: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L551-L589 That belongs in the Neutron plugin/agent side of things. Nova should not need to know how to call the vrouter-port-control Contrail CLI command. That should be Neutron's domain -- and not the Neutron API, but instead the underlying L2 agent/plugin. Most of the port binding profile stuff that is returned by the Neutron API's primitives should never have been exposed by the Neutron API, in my opinion. There's just too much driver-implementation specifics in there. Yes, that's exactly the set of tasks that is going to be hidden from Nova in the work Brent is doing to enable scripts. Ultimately all the plug/unplug methods in vif.py should go away, and Neutron will just pass over the name of a script to execute at plug unplug stages. So all the vif.py file in libvirt will need todo is invoke the nominated script at the right time, and build the libvirt XML config. All the business logic will be entirely under the control of the Neutron maintainer. Yes; but, as I commented on the spec earlier today, I don't think the vif-plugin-script work as it stands quite gets us there. We also still need either a set of base VIF types in Nova - e.g., in the libvirt case, to control whether the generated XML is interface type='ethernet' ... or interface type='bridge' ... - or some equivalently generic information that is passed from Neutron to allow Nova to generate the required XML (or equivalent for other hypervisors). Regards, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On 17/04/15 17:16, Jay Pipes wrote: On 04/10/2015 11:48 AM, Neil Jerram wrote: What I imagine, though, is that the _source_ of the plugging information could move from Nova to Neutron, so that future plugging-related code changes are a matter for Neutron rather than for Nova. The plugging would still _happen_ from within Nova, as directed by that information. -1. One of the biggest problems I have with the current implementation for Nova VIF types is that stuff leaks improperly out of the Neutron API. Take a look at all the Contrail implementation specifics in here: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L551-L589 That belongs in the Neutron plugin/agent side of things. Nova should not need to know how to call the vrouter-port-control Contrail CLI command. That should be Neutron's domain -- and not the Neutron API, but instead the underlying L2 agent/plugin. I entirely agree on this big picture. I think the disagreement (i.e. your -1) is just with some detail of my wording above; perhaps the from within Nova? Most of the port binding profile stuff that is returned by the Neutron API's primitives should never have been exposed by the Neutron API, in my opinion. There's just too much driver-implementation specifics in there. (For libvirt, I believe the required information would be the interface type and parameters to go into the libvirt XML, and then any further processing to do after that XML has been launched. I believe the latter is covered by the vif-plug-script spec, but not the former.) Agreed. However, on thinking this through, I see that this really is bound up with the wider question of nova-net / Neutron migration - because it obviously can't make sense for plugging information to come from Neutron if some people are not using Neutron at all for their networking. Stepping right back, though, is it agreed in principle that detailed networking code should move from Nova to Neutron, if that is possible while preserving all existing behaviours? If it is agreed, that's something that I'd like to help with. What precisely do you mean by preserving all existing behaviours? :) Just that we always need to respect back-compatibility. Isn't that a normal assumption within OpenStack? (Perhaps I didn't need to say it, then... :-)) Regards, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On 17/04/15 17:24, Daniel P. Berrange wrote: On Fri, Apr 17, 2015 at 12:16:25PM -0400, Jay Pipes wrote: On 04/10/2015 11:48 AM, Neil Jerram wrote: What I imagine, though, is that the _source_ of the plugging information could move from Nova to Neutron, so that future plugging-related code changes are a matter for Neutron rather than for Nova. The plugging would still _happen_ from within Nova, as directed by that information. -1. One of the biggest problems I have with the current implementation for Nova VIF types is that stuff leaks improperly out of the Neutron API. Take a look at all the Contrail implementation specifics in here: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L551-L589 That belongs in the Neutron plugin/agent side of things. Nova should not need to know how to call the vrouter-port-control Contrail CLI command. That should be Neutron's domain -- and not the Neutron API, but instead the underlying L2 agent/plugin. Most of the port binding profile stuff that is returned by the Neutron API's primitives should never have been exposed by the Neutron API, in my opinion. There's just too much driver-implementation specifics in there. Yes, that's exactly the set of tasks that is going to be hidden from Nova in the work Brent is doing to enable scripts. Ultimately all the plug/unplug methods in vif.py should go away, and Neutron will just pass over the name of a script to execute at plug unplug stages. So all the vif.py file in libvirt will need todo is invoke the nominated script at the right time, and build the libvirt XML config. All the business logic will be entirely under the control of the Neutron maintainer. Yes; but, as I commented on the spec earlier today, I don't think the vif-plugin-script work as it stands quite gets us there. We also still need either a set of base VIF types in Nova - e.g., in the libvirt case, to control whether the generated XML is interface type='ethernet' ... or interface type='bridge' ... - or some equivalently generic information that is passed from Neutron to allow Nova to generate the required XML (or equivalent for other hypervisors). Regards, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On 04/10/2015 11:48 AM, Neil Jerram wrote: What I imagine, though, is that the _source_ of the plugging information could move from Nova to Neutron, so that future plugging-related code changes are a matter for Neutron rather than for Nova. The plugging would still _happen_ from within Nova, as directed by that information. -1. One of the biggest problems I have with the current implementation for Nova VIF types is that stuff leaks improperly out of the Neutron API. Take a look at all the Contrail implementation specifics in here: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L551-L589 That belongs in the Neutron plugin/agent side of things. Nova should not need to know how to call the vrouter-port-control Contrail CLI command. That should be Neutron's domain -- and not the Neutron API, but instead the underlying L2 agent/plugin. Most of the port binding profile stuff that is returned by the Neutron API's primitives should never have been exposed by the Neutron API, in my opinion. There's just too much driver-implementation specifics in there. (For libvirt, I believe the required information would be the interface type and parameters to go into the libvirt XML, and then any further processing to do after that XML has been launched. I believe the latter is covered by the vif-plug-script spec, but not the former.) Agreed. However, on thinking this through, I see that this really is bound up with the wider question of nova-net / Neutron migration - because it obviously can't make sense for plugging information to come from Neutron if some people are not using Neutron at all for their networking. Stepping right back, though, is it agreed in principle that detailed networking code should move from Nova to Neutron, if that is possible while preserving all existing behaviours? If it is agreed, that's something that I'd like to help with. What precisely do you mean by preserving all existing behaviours? :) -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On Fri, Apr 17, 2015 at 12:16:25PM -0400, Jay Pipes wrote: On 04/10/2015 11:48 AM, Neil Jerram wrote: What I imagine, though, is that the _source_ of the plugging information could move from Nova to Neutron, so that future plugging-related code changes are a matter for Neutron rather than for Nova. The plugging would still _happen_ from within Nova, as directed by that information. -1. One of the biggest problems I have with the current implementation for Nova VIF types is that stuff leaks improperly out of the Neutron API. Take a look at all the Contrail implementation specifics in here: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L551-L589 That belongs in the Neutron plugin/agent side of things. Nova should not need to know how to call the vrouter-port-control Contrail CLI command. That should be Neutron's domain -- and not the Neutron API, but instead the underlying L2 agent/plugin. Most of the port binding profile stuff that is returned by the Neutron API's primitives should never have been exposed by the Neutron API, in my opinion. There's just too much driver-implementation specifics in there. Yes, that's exactly the set of tasks that is going to be hidden from Nova in the work Brent is doing to enable scripts. Ultimately all the plug/unplug methods in vif.py should go away, and Neutron will just pass over the name of a script to execute at plug unplug stages. So all the vif.py file in libvirt will need todo is invoke the nominated script at the right time, and build the libvirt XML config. All the business logic will be entirely under the control of the Neutron maintainer. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On 4/10/2015 10:29 AM, Matt Riedemann wrote: On 4/8/2015 4:07 PM, Michael Still wrote: I just wanted to send a note about John running in the PTL election for Nova. I want to make it clear that I think having more than one candidate is a good thing -- its a healthy part of a functional democracy, and it also means regardless of the outcome we have at least one succession planning option should a PTL need to step down at some point in the future. That said, I think there are a few things we need to do in Liberty, regardless of who is PTL. I started this as a Google doc to share with John if he won so that we didn’t drop the ball, but then I realised that nothing here is secret. So, here is my brain dump of things we need to do in Liberty, in no particular order: nova-coresec reboot The nova-coresec team has been struggling recently to keep up with their workload. We need to drop people off this team who haven’t had time recently to work on security bugs, and we need to find new people to volunteer for this team, noting that the team is kept deliberately small because of embargoed security vulnerabilities. If I am not re-elected as PTL, I will probably volunteer for this team. priorities and specs === I think the current spec process is starting to work well for us, and that priorities was a success. We should continue with specs, but with an attempt to analyse why so many approved specs don’t land (we have had about 50% of our approved specs not land in Juno and Kilo). Is that as simple as code review bandwidth? Or is the problem more complicated than that? We just don’t know until someone goes digging. Priorities worked well. We need to start talking about what should be a priority in Liberty now, and the first step is to decide as a team what we think the big problems we’re trying to solve in Liberty are. nova-core I think there are a couple of things to be done here. There are still a few idle cores, particularly people who haven’t done less than ten reviews in the last 90 days. We should drop those people from core and thank them for their work in the past noting once again that this is a natural part of the Open Source process -- those people are off working on other problems now and that’s cool. We also need to come up with a way to grow more cores. Passive approaches like asking existing cores to keep an eye out for talent they trust haven’t worked, so I think its time to actively start mentoring core candidates. I am not convinced that just adding cores will solve our review bandwidth problems though. We have these conversations about why people’s reviews sit around without a lot of data to back them up, and I feel like we often jump to conclusions that feel intuitive but that aren’t supported by the data. nova-net === OMG, this is still a thing. We need to actually work out what we’re doing here, and then do it. The path isn’t particularly clear to me any more, I thought I understood what we needed to do in Kilo, but it turns out that operators don’t feel that plan meets their needs. Somehow we need to get this work done. This is an obvious candidate for a summit session, if we can come up with a concrete proposal to discuss. bugs Trivial bug monkey’ing has worked well for us in Kilo, but one of our monkeys is off running as a PTL. We need to ensure we have this staffed with people who understand the constraints on the bugs we’re willing to let through this process. It would be sad to see this die on the vine. We also need to fix more bugs. I know we always say this, but we don’t have enough senior developers just kicking around looking at bugs to fix in a systematic way. This is something I used to do when I had more time before PTL’ing became a thing. If I am not elected this is the other thing I’ll probably go back to spending time on. conclusion I make no claim that my list is exhaustive. What else do you think we should be tackling in Liberty? Michael For better test coverage, we have a few things to do: 1. Make the ceph job voting: https://review.openstack.org/#/c/170913/ 2. Get the aiocpu job (which tests live block migrate) stable and voting, it has already caught some live migrate regressions recently. 3. Get the cells job stable and voting (we're pretty close). 4. We can get the Fedora 21 job in Nova's experimental queue: https://review.openstack.org/#/c/171795/ That allows us to test libvirt features (like live snapshot) on a newer version than what's in the gate on ubuntu 14.04. An update: 1. The ceph job is voting on master (and eventually stable/kilo) for nova/cinder/glance. 2. The aiopcpu job is in the process of being stabilized and should soon be voting. 3. WIP 4. The fc21 job is in the nova experimental queue now. -- Adding 5: We don't have any evacuate testing in Tempest today. We have rebuild, but not evacuate. I'll start working on that as a per-requisite for Dan's
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On 04/08/2015 05:07 PM, Michael Still wrote: There are still a few idle cores, particularly people who haven’t done less than ten reviews in the last 90 days. We should drop those people from core and thank them for their work in the past noting once again that this is a natural part of the Open Source process -- those people are off working on other problems now and that’s cool. I fit this description. :-) I started a bit of a networking kick in Kilo and I see that continuing for a while. I started helping with Neutron a bit, and now I've started helping build OVN and its OpenStack integration [1]. So, I'd like to go ahead and drop myself from nova*-core. However, anyone should feel free to ping me about any specific things that I could provide input on based on previous involvement. Thanks, everyone! nova-net === OMG, this is still a thing. We need to actually work out what we’re doing here, and then do it. The path isn’t particularly clear to me any more, I thought I understood what we needed to do in Kilo, but it turns out that operators don’t feel that plan meets their needs. Somehow we need to get this work done. This is an obvious candidate for a summit session, if we can come up with a concrete proposal to discuss. I totally agree with your sentiment here. I'm very interested in the future of networking for OpenStack with an open source backend. :-) [1] http://blog.russellbryant.net/2015/04/08/ovn-and-openstack-integration-development-update/ -- Russell Bryant __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Things to tackle in Liberty
Le 08/04/2015 23:07, Michael Still a écrit : I just wanted to send a note about John running in the PTL election for Nova. I want to make it clear that I think having more than one candidate is a good thing -- its a healthy part of a functional democracy, and it also means regardless of the outcome we have at least one succession planning option should a PTL need to step down at some point in the future. That said, I think there are a few things we need to do in Liberty, regardless of who is PTL. I started this as a Google doc to share with John if he won so that we didn’t drop the ball, but then I realised that nothing here is secret. So, here is my brain dump of things we need to do in Liberty, in no particular order: nova-coresec reboot The nova-coresec team has been struggling recently to keep up with their workload. We need to drop people off this team who haven’t had time recently to work on security bugs, and we need to find new people to volunteer for this team, noting that the team is kept deliberately small because of embargoed security vulnerabilities. If I am not re-elected as PTL, I will probably volunteer for this team. priorities and specs === I think the current spec process is starting to work well for us, and that priorities was a success. We should continue with specs, but with an attempt to analyse why so many approved specs don’t land (we have had about 50% of our approved specs not land in Juno and Kilo). Is that as simple as code review bandwidth? Or is the problem more complicated than that? We just don’t know until someone goes digging. As a reviewer, I think it's sometimes hard to figure out which specs can be looked first as we have more than 100 changes. For Kilo, I tried to query Gerrit using keywords but I found it was not good because I missed some important specs that I discovered once merged. On the other hand, some specs can be missed while there is a consensus. Could we maybe imagine to triage those specs using like we do in Launchpad ? I don't think amending the commit msgs is good, I'm more thinking about a dynamic etherpad that we can use for finding those. Now, as a proposer having had 4 specs approved by Kilo but only 2 of them landed (all of them part of a priority), I don't even think I can give a rule for that. I had a spec which was approved very early in Kilo but took most of my engineering effort for Kilo, I had one spec which was approved very late with a high number of iterations but whose implementation was implemented and merged in less than one week (!) and two specs which were kinda rational but failed at the implementation stage, mainly because some corner cases were not identified at the spec stage. Based on that experience, I would be tempted to consider that we underestimate how long it is necessary to provide a good spec by considering the design issues and the implementation details. If a spec is really easy to be approved and straightforward to implement, I would therefore consider if it would even be worth submitting a spec for it. I think the Kilo initiative to reduce the number of specs by easing what can be merged with only a blueprint move towards the right direction. We maybe need to further identify how a spec is really a design document which is not only a declaration of intent, but rather a very technical document which presents the steps and the change quite precisely. Priorities worked well. We need to start talking about what should be a priority in Liberty now, and the first step is to decide as a team what we think the big problems we’re trying to solve in Liberty are. ++ nova-core I think there are a couple of things to be done here. There are still a few idle cores, particularly people who haven’t done less than ten reviews in the last 90 days. We should drop those people from core and thank them for their work in the past noting once again that this is a natural part of the Open Source process -- those people are off working on other problems now and that’s cool. We also need to come up with a way to grow more cores. Passive approaches like asking existing cores to keep an eye out for talent they trust haven’t worked, so I think its time to actively start mentoring core candidates. I am not convinced that just adding cores will solve our review bandwidth problems though. We have these conversations about why people’s reviews sit around without a lot of data to back them up, and I feel like we often jump to conclusions that feel intuitive but that aren’t supported by the data. nova-net === OMG, this is still a thing. We need to actually work out what we’re doing here, and then do it. The path isn’t particularly clear to me any more, I thought I understood what we needed to do in Kilo, but it turns out that operators don’t feel that plan meets their needs. Somehow we need to get this work done. This is an obvious candidate for a summit
Re: [openstack-dev] [Nova] Things to tackle in Liberty
On 08/04/15 22:07, Michael Still wrote: priorities and specs === I think the current spec process is starting to work well for us, and that priorities was a success. We should continue with specs, but with an attempt to analyse why so many approved specs don’t land [...] nova-net === OMG, this is still a thing. We need to actually work out what we’re doing here, and then do it. [...] conclusion I make no claim that my list is exhaustive. What else do you think we should be tackling in Liberty? Something kind of related to two of the strands above, from the point of view of someone who had an approved networking-related Nova spec that failed to land for Kilo... Basically, should Nova try harder to get out of the networking business? Currently the situation is that OpenStack networking experimentation is mostly in Neutron (as I assume it should be) but also often requires changes to the VIF type code in Nova. Should we try to close off that situation, I would guess through some structural solution that puts all the required code changes firmly into Neutron's domain? I don't want to prejudge what the solution might be. My point here is to suggest discussing and deciding whether this could be a worthwhile priority. If it sounds of interest, I could add something to the etherpad for Nova design session ideas. (I appreciate that the nova-net question is way bigger and more practically important overall than my specific point about the VIF type code. However it is possible that the VIF type code contributes to a continuing lack of clarity about where networking function lies in OpenStack.) Regards, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova] Things to tackle in Liberty
I just wanted to send a note about John running in the PTL election for Nova. I want to make it clear that I think having more than one candidate is a good thing -- its a healthy part of a functional democracy, and it also means regardless of the outcome we have at least one succession planning option should a PTL need to step down at some point in the future. That said, I think there are a few things we need to do in Liberty, regardless of who is PTL. I started this as a Google doc to share with John if he won so that we didn’t drop the ball, but then I realised that nothing here is secret. So, here is my brain dump of things we need to do in Liberty, in no particular order: nova-coresec reboot The nova-coresec team has been struggling recently to keep up with their workload. We need to drop people off this team who haven’t had time recently to work on security bugs, and we need to find new people to volunteer for this team, noting that the team is kept deliberately small because of embargoed security vulnerabilities. If I am not re-elected as PTL, I will probably volunteer for this team. priorities and specs === I think the current spec process is starting to work well for us, and that priorities was a success. We should continue with specs, but with an attempt to analyse why so many approved specs don’t land (we have had about 50% of our approved specs not land in Juno and Kilo). Is that as simple as code review bandwidth? Or is the problem more complicated than that? We just don’t know until someone goes digging. Priorities worked well. We need to start talking about what should be a priority in Liberty now, and the first step is to decide as a team what we think the big problems we’re trying to solve in Liberty are. nova-core I think there are a couple of things to be done here. There are still a few idle cores, particularly people who haven’t done less than ten reviews in the last 90 days. We should drop those people from core and thank them for their work in the past noting once again that this is a natural part of the Open Source process -- those people are off working on other problems now and that’s cool. We also need to come up with a way to grow more cores. Passive approaches like asking existing cores to keep an eye out for talent they trust haven’t worked, so I think its time to actively start mentoring core candidates. I am not convinced that just adding cores will solve our review bandwidth problems though. We have these conversations about why people’s reviews sit around without a lot of data to back them up, and I feel like we often jump to conclusions that feel intuitive but that aren’t supported by the data. nova-net === OMG, this is still a thing. We need to actually work out what we’re doing here, and then do it. The path isn’t particularly clear to me any more, I thought I understood what we needed to do in Kilo, but it turns out that operators don’t feel that plan meets their needs. Somehow we need to get this work done. This is an obvious candidate for a summit session, if we can come up with a concrete proposal to discuss. bugs Trivial bug monkey’ing has worked well for us in Kilo, but one of our monkeys is off running as a PTL. We need to ensure we have this staffed with people who understand the constraints on the bugs we’re willing to let through this process. It would be sad to see this die on the vine. We also need to fix more bugs. I know we always say this, but we don’t have enough senior developers just kicking around looking at bugs to fix in a systematic way. This is something I used to do when I had more time before PTL’ing became a thing. If I am not elected this is the other thing I’ll probably go back to spending time on. conclusion I make no claim that my list is exhaustive. What else do you think we should be tackling in Liberty? Michael -- Rackspace Australia __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
