Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Hi all, On Wed, Mar 04, 2015 at 10:52:10AM -0330, Brent Eagles wrote: Hi all, snip/ Thanks Maxime. I've made some updates to the etherpad. (https://etherpad.openstack.org/p/nova_vif_plug_script_spec) I'm going to start some proof of concept work these evening. If I get anything worth reading, I'll put it up as a WIP/Draft review. Whatever state it is in I will be pushing up bits and pieces to github. https://github.com/beagles/neutron_hacking vif-plug-script https://github.com/beagles/nova vif-plug-script Cheers, Brent snip/ The proof-of-concept hacking progressed to the point where I was able to use the hacked up version of the ML2 OVS driver to trigger a test plug/unplug script, achieving connectiving with a test VM. With the exception of some boneheaded assumptions, things went reasonably well. I will squash the commits and post WIP patches on gerrit tomorrow. In my opinion, the big question now is what to do about rootwrap. The current proof-of-concept does *not* use it because of the sudo-stripping-environment variable issue. Environment variables can be passed in on the command line and the 'Env' rootwrap filter employed, but I don't know what is more workable from a deployment/3rd party integration point of view: use rootwrap and require rootwrap filters be added at deploy time; or don't use rootwrap from within nova and leave it up to the plug script/executable. If rootwrap isn't used when executing the plug script, the plug script itself could still use rootwrap. Thoughts? Cheers, Brent pgpQ3D6Ly7_2H.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Hi all, On Wed, Feb 18, 2015 at 04:12:11PM -0330, Brent Eagles wrote: Hi, snip/ Thanks Maxime. I've made some updates to the etherpad. (https://etherpad.openstack.org/p/nova_vif_plug_script_spec) I'm going to start some proof of concept work these evening. If I get anything worth reading, I'll put it up as a WIP/Draft review. Whatever state it is in I will be pushing up bits and pieces to github. https://github.com/beagles/neutron_hacking vif-plug-script https://github.com/beagles/nova vif-plug-script Cheers, Brent I had made several updates to the nova vif-plug-script branch over the past few days, but at some point I goofed and did a rebase when I didn't want to and messed up the branch. To keep things from getting messier, I deleted and reconstructed the branch based on the current master. This will only impact you if you had actually pulled the branch down (which I suspect is a very small to zero-length list of people). My apologies if this causes you inconvenience. Also, the Nova spec repo is open for L now. Unless anyone objects, I would like to cleanup the etherpad contents and submit the spec for review feedback. Cheers, Brent pgp9JKJ7OBYES.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
So are we talking about using script to eliminate unnecessary new vif types? Then, a little confusion that why this BP[1] is postponed to L, and this BP[2] is merged in K. [1] https://review.openstack.org/#/c/146914/ [2] https://review.openstack.org/#/c/148805/ In fact [2] can be replaced by [1] with a customized vouter script, no need for a totally new vif types introduced by K cycle. On Thu, Feb 19, 2015 at 3:42 AM, Brent Eagles beag...@redhat.com wrote: Hi, On 18/02/2015 1:53 PM, Maxime Leroy wrote: Hi Brent, snip/ Thanks for your help on this feature. I have just created a channel irc: #vif-plug-script-support to speak about it. I think it will help to synchronize effort on vif_plug_script development. Anyone is welcome on this channel! Cheers, Maxime Thanks Maxime. I've made some updates to the etherpad. (https://etherpad.openstack.org/p/nova_vif_plug_script_spec) I'm going to start some proof of concept work these evening. If I get anything worth reading, I'll put it up as a WIP/Draft review. Whatever state it is in I will be pushing up bits and pieces to github. https://github.com/beagles/neutron_hacking vif-plug-script https://github.com/beagles/nova vif-plug-script Cheers, Brent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Hi Brent, On Wed, Feb 18, 2015 at 3:26 PM, Brent Eagles beag...@redhat.com wrote: [..] I want to get the ball rolling on this ASAP so, I've started on this as well and will be updating the etherpad accordingly. I'm also keen to get W.I.P./P.O.C. patches to go along with it. I'll notify on the mailing list (and direct so you don't miss it ;)) as soon as I've completed a reasonable first swipe through the spec (which should be in the next day or so). Cheers, Brent Thanks for your help on this feature. I have just created a channel irc: #vif-plug-script-support to speak about it. I think it will help to synchronize effort on vif_plug_script development. Anyone is welcome on this channel! Cheers, Maxime __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Hi, On 18/02/2015 1:53 PM, Maxime Leroy wrote: Hi Brent, snip/ Thanks for your help on this feature. I have just created a channel irc: #vif-plug-script-support to speak about it. I think it will help to synchronize effort on vif_plug_script development. Anyone is welcome on this channel! Cheers, Maxime Thanks Maxime. I've made some updates to the etherpad. (https://etherpad.openstack.org/p/nova_vif_plug_script_spec) I'm going to start some proof of concept work these evening. If I get anything worth reading, I'll put it up as a WIP/Draft review. Whatever state it is in I will be pushing up bits and pieces to github. https://github.com/beagles/neutron_hacking vif-plug-script https://github.com/beagles/nova vif-plug-script Cheers, Brent signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Hi Maxime, Neil, On 16/01/2015 1:39 PM, Maxime Leroy wrote: On Wed, Jan 14, 2015 at 6:16 PM, Neil Jerram neil.jer...@metaswitch.com wrote: Maxime Leroy maxime.le...@6wind.com writes: Ok, thank you for the details. I will look how to implement this feature. Hi Maxime, Did you have time yet to start looking at this? My team now has a use case that could be met by using vif_plug_script, so I would be happy to help with writing the spec for that. Would that be of interest to you? Thanks, Neil Hi Neil, I have planned to look later how to implement this new feature. As we are in feature freeze for Nova and Neutron, there is no hurry right now. I think we need to have these 2 news specs ready before the next summit. Of course, any help is welcome ! ;) I have just created an etherpad to write the spec for Nova: https://etherpad.openstack.org/p/nova_vif_plug_script_spec Feel free to modify it. Thanks, Maxime I want to get the ball rolling on this ASAP so, I've started on this as well and will be updating the etherpad accordingly. I'm also keen to get W.I.P./P.O.C. patches to go along with it. I'll notify on the mailing list (and direct so you don't miss it ;)) as soon as I've completed a reasonable first swipe through the spec (which should be in the next day or so). Cheers, Brent signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Wed, Jan 14, 2015 at 6:16 PM, Neil Jerram neil.jer...@metaswitch.com wrote: Maxime Leroy maxime.le...@6wind.com writes: Ok, thank you for the details. I will look how to implement this feature. Hi Maxime, Did you have time yet to start looking at this? My team now has a use case that could be met by using vif_plug_script, so I would be happy to help with writing the spec for that. Would that be of interest to you? Thanks, Neil Hi Neil, I have planned to look later how to implement this new feature. As we are in feature freeze for Nova and Neutron, there is no hurry right now. I think we need to have these 2 news specs ready before the next summit. Of course, any help is welcome ! ;) I have just created an etherpad to write the spec for Nova: https://etherpad.openstack.org/p/nova_vif_plug_script_spec Feel free to modify it. Thanks, Maxime __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Maxime Leroy maxime.le...@6wind.com writes: Ok, thank you for the details. I will look how to implement this feature. Hi Maxime, Did you have time yet to start looking at this? My team now has a use case that could be met by using vif_plug_script, so I would be happy to help with writing the spec for that. Would that be of interest to you? Thanks, Neil __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Daniel P. Berrange berra...@redhat.com writes: Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug And so you see implementing a new Neutron mechanism in this way would not require *any* changes in Nova whatsoever. The work would be entirely self-contained within the scope of Neutron. It is simply a packaging task to get the vif script installed on the compute hosts, so that Nova can execute it. This is essentially providing a flexible VIF plugin system for Nova, without having to have it plug directly into the Nova codebase with the API RPC stability constraints that implies. I agree that this is a very promising idea. But... what about the problem that it is libvirt-specific? Does that matter? Regards, Neil ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Mon, Dec 15, 2014 at 10:36:59AM +, Neil Jerram wrote: Daniel P. Berrange berra...@redhat.com writes: Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug And so you see implementing a new Neutron mechanism in this way would not require *any* changes in Nova whatsoever. The work would be entirely self-contained within the scope of Neutron. It is simply a packaging task to get the vif script installed on the compute hosts, so that Nova can execute it. This is essentially providing a flexible VIF plugin system for Nova, without having to have it plug directly into the Nova codebase with the API RPC stability constraints that implies. I agree that this is a very promising idea. But... what about the problem that it is libvirt-specific? Does that matter? Libvirt defines terminology that is generally applicable to different hypervisors. Of course not all hypevisors will be capable of supporting all VIF types, but that's true no matter what terminology you choose, so I don't see any problem here. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Fri, Dec 12, 2014 at 3:16 PM, Daniel P. Berrange berra...@redhat.com wrote:
On Fri, Dec 12, 2014 at 03:05:28PM +0100, Maxime Leroy wrote:
On Fri, Dec 12, 2014 at 10:46 AM, Daniel P. Berrange
berra...@redhat.com wrote:
On Fri, Dec 12, 2014 at 01:21:36PM +0900, Ryu Ishimoto wrote:
On Thu, Dec 11, 2014 at 7:41 PM, Daniel P. Berrange berra...@redhat.com
wrote:
[..]
Port binding mechanism could vary among different networking technologies,
which is not nova's concern, so this proposal makes sense. Note that some
vendors already provide port binding scripts that are currently executed
directly from nova's vif.py ('mm-ctl' of midonet and 'ifc_ctl' for iovisor
are two such examples), and this proposal makes it unnecessary to have
these hard-coded in nova. The only question I have is, how would nova
figure out the arguments for these scripts? Should nova dictate what they
are?
We could define some standard set of arguments environment variables
to pass the information from the VIF to the script in a standard way.
Many information are used by the plug/unplug method: vif_id,
vif_address, ovs_interfaceid, firewall, net_id, tenant_id, vnic_type,
instance_uuid...
Not sure we can define a set of standard arguments.
That's really not a problem. There will be some set of common info
needed for all. Then for any particular vif type we know what extra
specific fields are define in the port binding metadata. We'll just
set env variables for each of those.
Maybe instead to use a script we should load some plug/unplug
functions from a python module with importlib. So a vif_plug_module
option instead to have a vif_plug_script ?
No, we explicitly do *not* want any usage of the Nova python modules.
That is all private internal Nova implementation detail that nothing
is permitted to rely on - this is why the VIF plugin feature was
removed in the first place.
There are several other problems to solve if we are going to use this
vif_plug_script:
- How to have the authorization to run this script (i.e. rootwrap)?
Yes, rootwrap.
- How to test plug/unplug function from these scripts?
Now, we have unity tests in nova/tests/unit/virt/libvirt/test_vif.py
for plug/unplug method.
Integration and/or functional tests run for the VIF impl would
exercise this code still.
- How this script will be installed?
- should it be including in the L2 agent package? Some L2 switch
doesn't have a L2 agent.
That's just a normal downstream packaging task which is easily
handled by people doing that work. If there's no L2 agent package
they can trivially just create a new package for the script(s)
that need installing on the comput node. They would have to be
doing exactly this anyway if you had the VIF plugin as a python
module instead.
Ok, thank you for the details. I will look how to implement this feature.
Regards,
Maxime
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Fri, Dec 12, 2014 at 01:21:36PM +0900, Ryu Ishimoto wrote:
On Thu, Dec 11, 2014 at 7:41 PM, Daniel P. Berrange berra...@redhat.com
wrote:
Yes, I really think this is a key point. When we introduced the VIF type
mechanism we never intended for there to be soo many different VIF types
created. There is a very small, finite number of possible ways to configure
the libvirt guest XML and it was intended that the VIF types pretty much
mirror that. This would have given us about 8 distinct VIF type maximum.
I think the reason for the larger than expected number of VIF types, is
that the drivers are being written to require some arbitrary tools to
be invoked in the plug unplug methods. It would really be better if
those could be accomplished in the Neutron code than the Nova code, via
a host agent run provided by the Neutron mechanism. This would let
us have a very small number of VIF types and so avoid the entire problem
that this thread is bringing up.
Failing that though, I could see a way to accomplish a similar thing
without a Neutron launched agent. If one of the VIF type binding
parameters were the name of a script, we could run that script on
plug unplug. So we'd have a finite number of VIF types, and each
new Neutron mechanism would merely have to provide a script to invoke
eg consider the existing midonet iovisor VIF types as an example.
Both of them use the libvirt ethernet config, but have different
things running in their plug methods. If we had a mechanism for
associating a plug script with a vif type, we could use a single
VIF type for both.
eg iovisor port binding info would contain
vif_type=ethernet
vif_plug_script=/usr/bin/neutron-iovisor-vif-plug
while midonet would contain
vif_type=ethernet
vif_plug_script=/usr/bin/neutron-midonet-vif-plug
And so you see implementing a new Neutron mechanism in this way would
not require *any* changes in Nova whatsoever. The work would be entirely
self-contained within the scope of Neutron. It is simply a packaging
task to get the vif script installed on the compute hosts, so that Nova
can execute it.
This is essentially providing a flexible VIF plugin system for Nova,
without having to have it plug directly into the Nova codebase with
the API RPC stability constraints that implies.
+1
Port binding mechanism could vary among different networking technologies,
which is not nova's concern, so this proposal makes sense. Note that some
vendors already provide port binding scripts that are currently executed
directly from nova's vif.py ('mm-ctl' of midonet and 'ifc_ctl' for iovisor
are two such examples), and this proposal makes it unnecessary to have
these hard-coded in nova. The only question I have is, how would nova
figure out the arguments for these scripts? Should nova dictate what they
are?
We could define some standard set of arguments environment variables
to pass the information from the VIF to the script in a standard way.
Regards,
Daniel
--
|: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Fri, Dec 12, 2014 at 10:46 AM, Daniel P. Berrange
berra...@redhat.com wrote:
On Fri, Dec 12, 2014 at 01:21:36PM +0900, Ryu Ishimoto wrote:
On Thu, Dec 11, 2014 at 7:41 PM, Daniel P. Berrange berra...@redhat.com
wrote:
[..]
Port binding mechanism could vary among different networking technologies,
which is not nova's concern, so this proposal makes sense. Note that some
vendors already provide port binding scripts that are currently executed
directly from nova's vif.py ('mm-ctl' of midonet and 'ifc_ctl' for iovisor
are two such examples), and this proposal makes it unnecessary to have
these hard-coded in nova. The only question I have is, how would nova
figure out the arguments for these scripts? Should nova dictate what they
are?
We could define some standard set of arguments environment variables
to pass the information from the VIF to the script in a standard way.
Many information are used by the plug/unplug method: vif_id,
vif_address, ovs_interfaceid, firewall, net_id, tenant_id, vnic_type,
instance_uuid...
Not sure we can define a set of standard arguments.
Maybe instead to use a script we should load some plug/unplug
functions from a python module with importlib. So a vif_plug_module
option instead to have a vif_plug_script ?
There are several other problems to solve if we are going to use this
vif_plug_script:
- How to have the authorization to run this script (i.e. rootwrap)?
- How to test plug/unplug function from these scripts?
Now, we have unity tests in nova/tests/unit/virt/libvirt/test_vif.py
for plug/unplug method.
- How this script will be installed?
- should it be including in the L2 agent package? Some L2 switch
doesn't have a L2 agent.
Regards,
Maxime
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Fri, Dec 12, 2014 at 03:05:28PM +0100, Maxime Leroy wrote:
On Fri, Dec 12, 2014 at 10:46 AM, Daniel P. Berrange
berra...@redhat.com wrote:
On Fri, Dec 12, 2014 at 01:21:36PM +0900, Ryu Ishimoto wrote:
On Thu, Dec 11, 2014 at 7:41 PM, Daniel P. Berrange berra...@redhat.com
wrote:
[..]
Port binding mechanism could vary among different networking technologies,
which is not nova's concern, so this proposal makes sense. Note that some
vendors already provide port binding scripts that are currently executed
directly from nova's vif.py ('mm-ctl' of midonet and 'ifc_ctl' for iovisor
are two such examples), and this proposal makes it unnecessary to have
these hard-coded in nova. The only question I have is, how would nova
figure out the arguments for these scripts? Should nova dictate what they
are?
We could define some standard set of arguments environment variables
to pass the information from the VIF to the script in a standard way.
Many information are used by the plug/unplug method: vif_id,
vif_address, ovs_interfaceid, firewall, net_id, tenant_id, vnic_type,
instance_uuid...
Not sure we can define a set of standard arguments.
That's really not a problem. There will be some set of common info
needed for all. Then for any particular vif type we know what extra
specific fields are define in the port binding metadata. We'll just
set env variables for each of those.
Maybe instead to use a script we should load some plug/unplug
functions from a python module with importlib. So a vif_plug_module
option instead to have a vif_plug_script ?
No, we explicitly do *not* want any usage of the Nova python modules.
That is all private internal Nova implementation detail that nothing
is permitted to rely on - this is why the VIF plugin feature was
removed in the first place.
There are several other problems to solve if we are going to use this
vif_plug_script:
- How to have the authorization to run this script (i.e. rootwrap)?
Yes, rootwrap.
- How to test plug/unplug function from these scripts?
Now, we have unity tests in nova/tests/unit/virt/libvirt/test_vif.py
for plug/unplug method.
Integration and/or functional tests run for the VIF impl would
exercise this code still.
- How this script will be installed?
- should it be including in the L2 agent package? Some L2 switch
doesn't have a L2 agent.
That's just a normal downstream packaging task which is easily
handled by people doing that work. If there's no L2 agent package
they can trivially just create a new package for the script(s)
that need installing on the comput node. They would have to be
doing exactly this anyway if you had the VIF plugin as a python
module instead.
Regards,
Daniel
--
|: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Thu, Dec 11, 2014 at 2:37 AM, henry hly henry4...@gmail.com wrote: On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: [..] The problem is that we effectively prevent running an out of tree Neutron driver (which *is* perfectly legitimate) if it uses a VIF plugging mechanism that isn't in Nova, as we can't use out of tree code and we won't accept in code ones for out of tree drivers. +1 well said ! The question is, do we really need such flexibility for so many nova vif types? Are we going to accept a new VIF_TYPE in nova if it's only used by an external ml2/l2 plugin ? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is nice if your out-of-tree l2/ml2 plugin needs a tap interface or a vhostuser socket. But if your external l2/ml2 plugin needs a specific type of nic (i.e. a new method get_config to provide specific parameters to libvirt for the nic) that not supported in the nova tree, you still need to have a plugin mechanism. [..] Your issue is one of testing. Is there any way we could set up a better testing framework for VIF drivers where Nova interacts with something to test the plugging mechanism actually passes traffic? I don't believe there's any specific limitation on it being *Neutron* that uses the plugging interaction. My spec proposes to use the same plugin mechanism for the vif drivers in the tree and for the external vif drivers. Please see my RFC patch: https://review.openstack.org/#/c/136857/ Maxime ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Thu, Dec 11, 2014 at 09:37:31AM +0800, henry hly wrote: On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: So the problem of Nova review bandwidth is a constant problem across all areas of the code. We need to solve this problem for the team as a whole in a much broader fashion than just for people writing VIF drivers. The VIF drivers are really small pieces of code that should be straightforward to review get merged in any release cycle in which they are proposed. I think we need to make sure that we focus our energy on doing this and not ignoring the problem by breaking stuff off out of tree. The problem is that we effectively prevent running an out of tree Neutron driver (which *is* perfectly legitimate) if it uses a VIF plugging mechanism that isn't in Nova, as we can't use out of tree code and we won't accept in code ones for out of tree drivers. The question is, do we really need such flexibility for so many nova vif types? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. However I'm not saying to move existing vif driver out, those open backend have been used widely. But from now on the tap and vhostuser mode should be encouraged: one common vif driver to many long-tail backend. Yes, I really think this is a key point. When we introduced the VIF type mechanism we never intended for there to be soo many different VIF types created. There is a very small, finite number of possible ways to configure the libvirt guest XML and it was intended that the VIF types pretty much mirror that. This would have given us about 8 distinct VIF type maximum. I think the reason for the larger than expected number of VIF types, is that the drivers are being written to require some arbitrary tools to be invoked in the plug unplug methods. It would really be better if those could be accomplished in the Neutron code than the Nova code, via a host agent run provided by the Neutron mechanism. This would let us have a very small number of VIF types and so avoid the entire problem that this thread is bringing up. Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug And so you see implementing a new Neutron mechanism in this way would not require *any* changes in Nova whatsoever. The work would be entirely self-contained within the scope of Neutron. It is simply a packaging task to get the vif script installed on the compute hosts, so that Nova can execute it. This is essentially providing a flexible VIF plugin system for Nova, without having to have it plug directly into the Nova codebase with the API RPC stability constraints that implies. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Thu, Dec 11, 2014 at 11:41 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Dec 11, 2014 at 09:37:31AM +0800, henry hly wrote: On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: [..] The question is, do we really need such flexibility for so many nova vif types? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. However I'm not saying to move existing vif driver out, those open backend have been used widely. But from now on the tap and vhostuser mode should be encouraged: one common vif driver to many long-tail backend. Yes, I really think this is a key point. When we introduced the VIF type mechanism we never intended for there to be soo many different VIF types created. There is a very small, finite number of possible ways to configure the libvirt guest XML and it was intended that the VIF types pretty much mirror that. This would have given us about 8 distinct VIF type maximum. I think the reason for the larger than expected number of VIF types, is that the drivers are being written to require some arbitrary tools to be invoked in the plug unplug methods. It would really be better if those could be accomplished in the Neutron code than the Nova code, via a host agent run provided by the Neutron mechanism. This would let us have a very small number of VIF types and so avoid the entire problem that this thread is bringing up. Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug Having less VIF types, then using scripts to plug/unplug the vif in nova is a good idea. So, +1 for the idea. If you want, I can propose a new spec for this. Do you think we have enough time to approve this new spec before the 18th December? Anyway I think we still need to have a vif_driver plugin mechanism: For example, if your external l2/ml2 plugin needs a specific type of nic (i.e. a new method get_config to provide specific parameters to libvirt for the nic) that is not supported in the nova tree. Maybe we can find an other way to support it? Or, are we going to accept new VIF_TYPE in nova if it's only used by an external ml2/l2 plugin? Maxime ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Thu, Dec 11, 2014 at 04:15:00PM +0100, Maxime Leroy wrote: On Thu, Dec 11, 2014 at 11:41 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Dec 11, 2014 at 09:37:31AM +0800, henry hly wrote: On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: [..] The question is, do we really need such flexibility for so many nova vif types? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. However I'm not saying to move existing vif driver out, those open backend have been used widely. But from now on the tap and vhostuser mode should be encouraged: one common vif driver to many long-tail backend. Yes, I really think this is a key point. When we introduced the VIF type mechanism we never intended for there to be soo many different VIF types created. There is a very small, finite number of possible ways to configure the libvirt guest XML and it was intended that the VIF types pretty much mirror that. This would have given us about 8 distinct VIF type maximum. I think the reason for the larger than expected number of VIF types, is that the drivers are being written to require some arbitrary tools to be invoked in the plug unplug methods. It would really be better if those could be accomplished in the Neutron code than the Nova code, via a host agent run provided by the Neutron mechanism. This would let us have a very small number of VIF types and so avoid the entire problem that this thread is bringing up. Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug Having less VIF types, then using scripts to plug/unplug the vif in nova is a good idea. So, +1 for the idea. If you want, I can propose a new spec for this. Do you think we have enough time to approve this new spec before the 18th December? Anyway I think we still need to have a vif_driver plugin mechanism: For example, if your external l2/ml2 plugin needs a specific type of nic (i.e. a new method get_config to provide specific parameters to libvirt for the nic) that is not supported in the nova tree. As I said above, there's a really small finite set of libvirt configs we need to care about. We don't need to have a plugin system for that. It is no real burden to support them in tree Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Dec 11, 2014, at 2:41 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Dec 11, 2014 at 09:37:31AM +0800, henry hly wrote: On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: So the problem of Nova review bandwidth is a constant problem across all areas of the code. We need to solve this problem for the team as a whole in a much broader fashion than just for people writing VIF drivers. The VIF drivers are really small pieces of code that should be straightforward to review get merged in any release cycle in which they are proposed. I think we need to make sure that we focus our energy on doing this and not ignoring the problem by breaking stuff off out of tree. The problem is that we effectively prevent running an out of tree Neutron driver (which *is* perfectly legitimate) if it uses a VIF plugging mechanism that isn't in Nova, as we can't use out of tree code and we won't accept in code ones for out of tree drivers. The question is, do we really need such flexibility for so many nova vif types? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. However I'm not saying to move existing vif driver out, those open backend have been used widely. But from now on the tap and vhostuser mode should be encouraged: one common vif driver to many long-tail backend. Yes, I really think this is a key point. When we introduced the VIF type mechanism we never intended for there to be soo many different VIF types created. There is a very small, finite number of possible ways to configure the libvirt guest XML and it was intended that the VIF types pretty much mirror that. This would have given us about 8 distinct VIF type maximum. I think the reason for the larger than expected number of VIF types, is that the drivers are being written to require some arbitrary tools to be invoked in the plug unplug methods. It would really be better if those could be accomplished in the Neutron code than the Nova code, via a host agent run provided by the Neutron mechanism. This would let us have a very small number of VIF types and so avoid the entire problem that this thread is bringing up. Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug +1 This is a great suggestion. Vish And so you see implementing a new Neutron mechanism in this way would not require *any* changes in Nova whatsoever. The work would be entirely self-contained within the scope of Neutron. It is simply a packaging task to get the vif script installed on the compute hosts, so that Nova can execute it. This is essentially providing a flexible VIF plugin system for Nova, without having to have it plug directly into the Nova codebase with the API RPC stability constraints that implies. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: Message signed with OpenPGP using GPGMail ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
+100! So, for the vif-type-vhostuser, a general script path name replace the vif-detail vhost_user_ovs_plug, because it's not the responsibility of nova to understand it. On Thu, Dec 11, 2014 at 11:24 PM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Dec 11, 2014 at 04:15:00PM +0100, Maxime Leroy wrote: On Thu, Dec 11, 2014 at 11:41 AM, Daniel P. Berrange berra...@redhat.com wrote: On Thu, Dec 11, 2014 at 09:37:31AM +0800, henry hly wrote: On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: [..] The question is, do we really need such flexibility for so many nova vif types? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. However I'm not saying to move existing vif driver out, those open backend have been used widely. But from now on the tap and vhostuser mode should be encouraged: one common vif driver to many long-tail backend. Yes, I really think this is a key point. When we introduced the VIF type mechanism we never intended for there to be soo many different VIF types created. There is a very small, finite number of possible ways to configure the libvirt guest XML and it was intended that the VIF types pretty much mirror that. This would have given us about 8 distinct VIF type maximum. I think the reason for the larger than expected number of VIF types, is that the drivers are being written to require some arbitrary tools to be invoked in the plug unplug methods. It would really be better if those could be accomplished in the Neutron code than the Nova code, via a host agent run provided by the Neutron mechanism. This would let us have a very small number of VIF types and so avoid the entire problem that this thread is bringing up. Failing that though, I could see a way to accomplish a similar thing without a Neutron launched agent. If one of the VIF type binding parameters were the name of a script, we could run that script on plug unplug. So we'd have a finite number of VIF types, and each new Neutron mechanism would merely have to provide a script to invoke eg consider the existing midonet iovisor VIF types as an example. Both of them use the libvirt ethernet config, but have different things running in their plug methods. If we had a mechanism for associating a plug script with a vif type, we could use a single VIF type for both. eg iovisor port binding info would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-iovisor-vif-plug while midonet would contain vif_type=ethernet vif_plug_script=/usr/bin/neutron-midonet-vif-plug Having less VIF types, then using scripts to plug/unplug the vif in nova is a good idea. So, +1 for the idea. If you want, I can propose a new spec for this. Do you think we have enough time to approve this new spec before the 18th December? Anyway I think we still need to have a vif_driver plugin mechanism: For example, if your external l2/ml2 plugin needs a specific type of nic (i.e. a new method get_config to provide specific parameters to libvirt for the nic) that is not supported in the nova tree. As I said above, there's a really small finite set of libvirt configs we need to care about. We don't need to have a plugin system for that. It is no real burden to support them in tree Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Thu, Dec 11, 2014 at 7:41 PM, Daniel P. Berrange berra...@redhat.com
wrote:
Yes, I really think this is a key point. When we introduced the VIF type
mechanism we never intended for there to be soo many different VIF types
created. There is a very small, finite number of possible ways to configure
the libvirt guest XML and it was intended that the VIF types pretty much
mirror that. This would have given us about 8 distinct VIF type maximum.
I think the reason for the larger than expected number of VIF types, is
that the drivers are being written to require some arbitrary tools to
be invoked in the plug unplug methods. It would really be better if
those could be accomplished in the Neutron code than the Nova code, via
a host agent run provided by the Neutron mechanism. This would let
us have a very small number of VIF types and so avoid the entire problem
that this thread is bringing up.
Failing that though, I could see a way to accomplish a similar thing
without a Neutron launched agent. If one of the VIF type binding
parameters were the name of a script, we could run that script on
plug unplug. So we'd have a finite number of VIF types, and each
new Neutron mechanism would merely have to provide a script to invoke
eg consider the existing midonet iovisor VIF types as an example.
Both of them use the libvirt ethernet config, but have different
things running in their plug methods. If we had a mechanism for
associating a plug script with a vif type, we could use a single
VIF type for both.
eg iovisor port binding info would contain
vif_type=ethernet
vif_plug_script=/usr/bin/neutron-iovisor-vif-plug
while midonet would contain
vif_type=ethernet
vif_plug_script=/usr/bin/neutron-midonet-vif-plug
And so you see implementing a new Neutron mechanism in this way would
not require *any* changes in Nova whatsoever. The work would be entirely
self-contained within the scope of Neutron. It is simply a packaging
task to get the vif script installed on the compute hosts, so that Nova
can execute it.
This is essentially providing a flexible VIF plugin system for Nova,
without having to have it plug directly into the Nova codebase with
the API RPC stability constraints that implies.
+1
Port binding mechanism could vary among different networking technologies,
which is not nova's concern, so this proposal makes sense. Note that some
vendors already provide port binding scripts that are currently executed
directly from nova's vif.py ('mm-ctl' of midonet and 'ifc_ctl' for iovisor
are two such examples), and this proposal makes it unnecessary to have
these hard-coded in nova. The only question I have is, how would nova
figure out the arguments for these scripts? Should nova dictate what they
are?
Ryu
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Wed, Dec 10, 2014 at 07:41:55AM +, Irena Berezovsky wrote: Hi Daniel, Please see inline -Original Message- From: Daniel P. Berrange [mailto:berra...@redhat.com] Sent: Tuesday, December 09, 2014 4:04 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver The VIF parameters are mapped into the nova.network.model.VIF class which is doing some crude validation. I would anticipate that this validation will be increasing over time, because any functional data flowing over the API and so needs to be carefully managed for upgrade reasons. Even if the Neutron impl is out of tree, I would still expect both Nova and Neutron core to sign off on any new VIF type name and its associated details (if any). [IB] This maybe the reasonable integration point. But it requires nova team review and approval. From my experience nova team is extremely overloaded, therefor getting this code reviewed become very difficult mission. What other reasons am I missing to not have VIF driver classes as a public extension point ? Having to find install VIF driver classes from countless different vendors, each hiding their code away in their own obsecure website, will lead to awful end user experiance when deploying Nova. Users are better served by having it all provided when they deploy Nova IMHO If every vendor goes off works in their own isolated world we also loose the scope to align the implementations, so that common concepts work the same way in all cases and allow us to minimize the number of new VIF types required. The proposed vhostuser VIF type is a good example of this - it allows a single Nova VIF driver to be capable of potentially supporting multiple different impls on the Neutron side. If every vendor worked in their own world, we would have ended up with multiple VIF drivers doing the same thing in Nova, each with their own set of bugs quirks. [IB] I think that most of the vendors that maintain vif_driver out of nova, do not do it on purpose and would prefer to see it upstream. Sometimes host side binding is not fully integrated with libvirt and requires some temporary additional code, till libvirt provides complete support. Sometimes, it is just lack of nova team attention to the proposed spec/code to be reviewed and accepted on time, which ends up with fully supported neutron part and missing small but critical vif_driver piece. So the problem of Nova review bandwidth is a constant problem across all areas of the code. We need to solve this problem for the team as a whole in a much broader fashion than just for people writing VIF drivers. The VIF drivers are really small pieces of code that should be straightforward to review get merged in any release cycle in which they are proposed. I think we need to make sure that we focus our energy on doing this and not ignoring the problem by breaking stuff off out of tree. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On 12/10/2014 04:31 AM, Daniel P. Berrange wrote: So the problem of Nova review bandwidth is a constant problem across all areas of the code. We need to solve this problem for the team as a whole in a much broader fashion than just for people writing VIF drivers. The VIF drivers are really small pieces of code that should be straightforward to review get merged in any release cycle in which they are proposed. I think we need to make sure that we focus our energy on doing this and not ignoring the problem by breaking stuff off out of tree. +1 well said. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: So the problem of Nova review bandwidth is a constant problem across all areas of the code. We need to solve this problem for the team as a whole in a much broader fashion than just for people writing VIF drivers. The VIF drivers are really small pieces of code that should be straightforward to review get merged in any release cycle in which they are proposed. I think we need to make sure that we focus our energy on doing this and not ignoring the problem by breaking stuff off out of tree. The problem is that we effectively prevent running an out of tree Neutron driver (which *is* perfectly legitimate) if it uses a VIF plugging mechanism that isn't in Nova, as we can't use out of tree code and we won't accept in code ones for out of tree drivers. This will get more confusing as *all* of the Neutron drivers and plugins move out of the tree, as that constraint becomes essentially arbitrary. Your issue is one of testing. Is there any way we could set up a better testing framework for VIF drivers where Nova interacts with something to test the plugging mechanism actually passes traffic? I don't believe there's any specific limitation on it being *Neutron* that uses the plugging interaction. -- Ian. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Thu, Dec 11, 2014 at 3:48 AM, Ian Wells ijw.ubu...@cack.org.uk wrote: On 10 December 2014 at 01:31, Daniel P. Berrange berra...@redhat.com wrote: So the problem of Nova review bandwidth is a constant problem across all areas of the code. We need to solve this problem for the team as a whole in a much broader fashion than just for people writing VIF drivers. The VIF drivers are really small pieces of code that should be straightforward to review get merged in any release cycle in which they are proposed. I think we need to make sure that we focus our energy on doing this and not ignoring the problem by breaking stuff off out of tree. The problem is that we effectively prevent running an out of tree Neutron driver (which *is* perfectly legitimate) if it uses a VIF plugging mechanism that isn't in Nova, as we can't use out of tree code and we won't accept in code ones for out of tree drivers. The question is, do we really need such flexibility for so many nova vif types? I also think that VIF_TYPE_TAP and VIF_TYPE_VHOSTUSER is good example, nova shouldn't known too much details about switch backend, it should only care about the VIF itself, how the VIF is plugged to switch belongs to Neutron half. However I'm not saying to move existing vif driver out, those open backend have been used widely. But from now on the tap and vhostuser mode should be encouraged: one common vif driver to many long-tail backend. Best Regards, Henry This will get more confusing as *all* of the Neutron drivers and plugins move out of the tree, as that constraint becomes essentially arbitrary. Your issue is one of testing. Is there any way we could set up a better testing framework for VIF drivers where Nova interacts with something to test the plugging mechanism actually passes traffic? I don't believe there's any specific limitation on it being *Neutron* that uses the plugging interaction. -- Ian. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Tue, Dec 09, 2014 at 10:53:19AM +0100, Maxime Leroy wrote: I have also proposed a blueprint to have a new plugin mechanism in nova to load external vif driver. (nova-specs: https://review.openstack.org/#/c/136827/ and nova (rfc patch): https://review.openstack.org/#/c/136857/) From my point-of-view of a developer having a plugin framework for internal/external vif driver seems to be a good thing. It makes the code more modular and introduce a clear api for vif driver classes. So far, it raises legitimate questions concerning API stability and public API that request a wider discussion on the ML (as asking by John Garbut). I think having a plugin mechanism and a clear api for vif driver is not going against this policy: http://docs.openstack.org/developer/nova/devref/policies.html#out-of-tree-support. There is no needs to have a stable API. It is up to the owner of the external VIF driver to ensure that its driver is supported by the latest API. And not the nova community to manage a stable API for this external VIF driver. Does it make senses ? Experiance has shown that even if it is documented as unsupported, once the extension point exists, vendors users will ignore the small print about support status. There will be complaints raised every time it gets broken until we end up being forced to maintain it as stable API whether we want to or not. That's not a route we want to go down. Considering the network V2 API, L2/ML2 mechanism driver and VIF driver need to exchange information such as: binding:vif_type and binding:vif_details. From my understanding, 'binding:vif_type' and 'binding:vif_details' as a field part of the public network api. There is no validation constraints for these fields (see http://docs.openstack.org/api/openstack-network/2.0/content/binding_ext_ports.html), it means that any value is accepted by the API. So, the values set in 'binding:vif_type' and 'binding:vif_details' are not part of the public API. Is my understanding correct ? The VIF parameters are mapped into the nova.network.model.VIF class which is doing some crude validation. I would anticipate that this validation will be increasing over time, because any functional data flowing over the API and so needs to be carefully managed for upgrade reasons. Even if the Neutron impl is out of tree, I would still expect both Nova and Neutron core to sign off on any new VIF type name and its associated details (if any). What other reasons am I missing to not have VIF driver classes as a public extension point ? Having to find install VIF driver classes from countless different vendors, each hiding their code away in their own obsecure website, will lead to awful end user experiance when deploying Nova. Users are better served by having it all provided when they deploy Nova IMHO If every vendor goes off works in their own isolated world we also loose the scope to align the implementations, so that common concepts work the same way in all cases and allow us to minimize the number of new VIF types required. The proposed vhostuser VIF type is a good example of this - it allows a single Nova VIF driver to be capable of potentially supporting multiple different impls on the Neutron side. If every vendor worked in their own world, we would have ended up with multiple VIF drivers doing the same thing in Nova, each with their own set of bugs quirks. I expect the quality of the code the operator receives will be lower if it is never reviewed by anyone except the vendor who writes it in the first place. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
On Dec 9, 2014, at 7:04 AM, Daniel P. Berrange berra...@redhat.com wrote: On Tue, Dec 09, 2014 at 10:53:19AM +0100, Maxime Leroy wrote: I have also proposed a blueprint to have a new plugin mechanism in nova to load external vif driver. (nova-specs: https://review.openstack.org/#/c/136827/ and nova (rfc patch): https://review.openstack.org/#/c/136857/) From my point-of-view of a developer having a plugin framework for internal/external vif driver seems to be a good thing. It makes the code more modular and introduce a clear api for vif driver classes. So far, it raises legitimate questions concerning API stability and public API that request a wider discussion on the ML (as asking by John Garbut). I think having a plugin mechanism and a clear api for vif driver is not going against this policy: http://docs.openstack.org/developer/nova/devref/policies.html#out-of-tree-support. There is no needs to have a stable API. It is up to the owner of the external VIF driver to ensure that its driver is supported by the latest API. And not the nova community to manage a stable API for this external VIF driver. Does it make senses ? Experiance has shown that even if it is documented as unsupported, once the extension point exists, vendors users will ignore the small print about support status. There will be complaints raised every time it gets broken until we end up being forced to maintain it as stable API whether we want to or not. That's not a route we want to go down. Is the support contract for a given API have to be binary - ‘supported’ vs ‘unsupported’? The stability requirements for REST API’s that end users and all kinds of tooling consume are arguably different from those of an internal API, and recognizing this difference could be useful. Considering the network V2 API, L2/ML2 mechanism driver and VIF driver need to exchange information such as: binding:vif_type and binding:vif_details. From my understanding, 'binding:vif_type' and 'binding:vif_details' as a field part of the public network api. There is no validation constraints for these fields (see http://docs.openstack.org/api/openstack-network/2.0/content/binding_ext_ports.html), it means that any value is accepted by the API. So, the values set in 'binding:vif_type' and 'binding:vif_details' are not part of the public API. Is my understanding correct ? The VIF parameters are mapped into the nova.network.model.VIF class which is doing some crude validation. I would anticipate that this validation will be increasing over time, because any functional data flowing over the API and so needs to be carefully managed for upgrade reasons. Even if the Neutron impl is out of tree, I would still expect both Nova and Neutron core to sign off on any new VIF type name and its associated details (if any). What other reasons am I missing to not have VIF driver classes as a public extension point ? Having to find install VIF driver classes from countless different vendors, each hiding their code away in their own obsecure website, will lead to awful end user experiance when deploying Nova. Users are better served by having it all provided when they deploy Nova IMHO Shipping drivers in-tree makes sense for a purely open source solution for the reasons you mention. The logic doesn’t necessarily extend to deployment of a proprietary solution, though. If a given OpenStack deployment is intended to integrate with such a solution, it is likely that the distro/operator/deployer will have a direct relationship with the solution provider and the required software (including VIF driver(s), if necessary) is likely to have a well-defined distribution channel. If every vendor goes off works in their own isolated world we also loose the scope to align the implementations, so that common concepts work the same way in all cases and allow us to minimize the number of new VIF types required. The proposed vhostuser VIF type is a good example of this - it allows a single Nova VIF driver to be capable of potentially supporting multiple different impls on the Neutron side. If every vendor worked in their own world, we would have ended up with multiple VIF drivers doing the same thing in Nova, each with their own set of bugs quirks. I’m not sure the suggestion is that every vendor go off and do their own thing. Rather, the option for out-of-tree drivers could be made available to those that are pursuing initiatives that aren’t found to be in-keeping with Nova’s current priorities. I believe that allowing out-of-tree extensions is essential to ensuring the long-term viability of OpenStack. There is only so much experimental work that is going to be acceptable in core OpenStack projects, if only to ensure stability. Yes, there is the potential for duplicative effort with results of varying quality, but that’s the price of competitive innovation whether in the field of ideas or
Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver
Hi Daniel, Please see inline -Original Message- From: Daniel P. Berrange [mailto:berra...@redhat.com] Sent: Tuesday, December 09, 2014 4:04 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova][Neutron] out-of-tree plugin for Mech driver/L2 and vif_driver On Tue, Dec 09, 2014 at 10:53:19AM +0100, Maxime Leroy wrote: I have also proposed a blueprint to have a new plugin mechanism in nova to load external vif driver. (nova-specs: https://review.openstack.org/#/c/136827/ and nova (rfc patch): https://review.openstack.org/#/c/136857/) From my point-of-view of a developer having a plugin framework for internal/external vif driver seems to be a good thing. It makes the code more modular and introduce a clear api for vif driver classes. So far, it raises legitimate questions concerning API stability and public API that request a wider discussion on the ML (as asking by John Garbut). I think having a plugin mechanism and a clear api for vif driver is not going against this policy: http://docs.openstack.org/developer/nova/devref/policies.html#out-of-tree-support. There is no needs to have a stable API. It is up to the owner of the external VIF driver to ensure that its driver is supported by the latest API. And not the nova community to manage a stable API for this external VIF driver. Does it make senses ? Experiance has shown that even if it is documented as unsupported, once the extension point exists, vendors users will ignore the small print about support status. There will be complaints raised every time it gets broken until we end up being forced to maintain it as stable API whether we want to or not. That's not a route we want to go down. [IB] It should be up to the vendor to maintain it and make sure it's not broken. Having proper 3rd party CI in place should catch API contract changes. Considering the network V2 API, L2/ML2 mechanism driver and VIF driver need to exchange information such as: binding:vif_type and binding:vif_details. From my understanding, 'binding:vif_type' and 'binding:vif_details' as a field part of the public network api. There is no validation constraints for these fields (see http://docs.openstack.org/api/openstack-network/2.0/content/binding_ex t_ports.html), it means that any value is accepted by the API. So, the values set in 'binding:vif_type' and 'binding:vif_details' are not part of the public API. Is my understanding correct ? The VIF parameters are mapped into the nova.network.model.VIF class which is doing some crude validation. I would anticipate that this validation will be increasing over time, because any functional data flowing over the API and so needs to be carefully managed for upgrade reasons. Even if the Neutron impl is out of tree, I would still expect both Nova and Neutron core to sign off on any new VIF type name and its associated details (if any). [IB] This maybe the reasonable integration point. But it requires nova team review and approval. From my experience nova team is extremely overloaded, therefor getting this code reviewed become very difficult mission. What other reasons am I missing to not have VIF driver classes as a public extension point ? Having to find install VIF driver classes from countless different vendors, each hiding their code away in their own obsecure website, will lead to awful end user experiance when deploying Nova. Users are better served by having it all provided when they deploy Nova IMHO If every vendor goes off works in their own isolated world we also loose the scope to align the implementations, so that common concepts work the same way in all cases and allow us to minimize the number of new VIF types required. The proposed vhostuser VIF type is a good example of this - it allows a single Nova VIF driver to be capable of potentially supporting multiple different impls on the Neutron side. If every vendor worked in their own world, we would have ended up with multiple VIF drivers doing the same thing in Nova, each with their own set of bugs quirks. [IB] I think that most of the vendors that maintain vif_driver out of nova, do not do it on purpose and would prefer to see it upstream. Sometimes host side binding is not fully integrated with libvirt and requires some temporary additional code, till libvirt provides complete support. Sometimes, it is just lack of nova team attention to the proposed spec/code to be reviewed and accepted on time, which ends up with fully supported neutron part and missing small but critical vif_driver piece. I expect the quality of the code the operator receives will be lower if it is never reviewed by anyone except the vendor who writes it in the first place. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :|
