On 2013?10?29? 03:22, Robert Li (baoli) wrote:
Hi Irena,
Thank you very much for your comments. See inline.
--Robert
On 10/27/13 3:48 AM, "Irena Berezovsky" <[email protected]
<mailto:[email protected]>> wrote:
Hi Robert,
Thank you very much for sharing the information regarding your
efforts. Can you please share your idea of the end to end flow?
How do you suggest to bind Nova and Neutron?
The end to end flow is actually encompassed in the blueprints in a
nutshell. I will reiterate it in below. The binding between Nova and
Neutron occurs with the neutron v2 API that nova invokes in order to
provision the neutron services. The vif driver is responsible for
plugging in an instance onto the networking setup that neutron has
created on the host.
Normally, one will invoke "nova boot" api with the —nic options to
specify the nic with which the instance will be connected to the
network. It currently allows net-id, fixed ip and/or port-id to be
specified for the option. However, it doesn't allow one to specify
special networking requirements for the instance. Thanks to the nova
pci-passthrough work, one can specify PCI passthrough device(s) in the
nova flavor. But it doesn't provide means to tie up these PCI devices
in the case of ethernet adpators with networking services. Therefore
the idea is actually simple as indicated by the blueprint titles, to
provide means to tie up SRIOV devices with neutron services. A work
flow would roughly look like this for 'nova boot':
-- Specifies networking requirements in the —nic option.
Specifically for SRIOV, allow the following to be specified in
addition to the existing required information:
. PCI alias
. direct pci-passthrough/macvtap
. port profileid that is compliant with 802.1Qbh
The above information is optional. In the absence of them, the
existing behavior remains.
-- if special networking requirements exist, Nova api creates PCI
requests in the nova instance type for scheduling purpose
-- Nova scheduler schedules the instance based on the requested
flavor plus the PCI requests that are created for networking.
-- Nova compute invokes neutron services with PCI passthrough
information if any
-- Neutron performs its normal operations based on the request,
such as allocating a port, assigning ip addresses, etc. Specific to
SRIOV, it should validate the information such as profileid, and
stores them in its db. It's also possible to associate a port
profileid with a neutron network so that port profileid becomes
optional in the —nic option. Neutron returns nova the port
information, especially for PCI passthrough related information in the
port binding object. Currently, the port binding object contains the
following information:
binding:vif_type
binding:host_id
binding:profile
binding:capabilities
(openstack bonce stop me to sent to so many people at one time, so i
remove cc & to, hope every one can see this)
i heard of some nic passthrough solution in summary, and you metioned
this, in high level of implement the NIC passthrough there is:
hardware VEB (Virtual ethernet Switches )
the nic need external switch like 802.1qbg
so question is:
where is the diffrent type infomation?
does 802.1qbg need know which port the PF connected to?
-- nova constructs the domain xml and plug in the instance by
calling the vif driver. The vif driver can build up the interface xml
based on the port binding information.
The blueprints you registered make sense. On Nova side, there is a
need to bind between requested virtual network and PCI
device/interface to be allocated as vNIC.
On the Neutron side, there is a need to support networking
configuration of the vNIC. Neutron should be able to identify the
PCI device/macvtap interface in order to apply configuration. I
think it makes sense to provide neutron integration via dedicated
Modular Layer 2 Mechanism Driver to allow PCI pass-through vNIC
support along with other networking technologies.
I haven't sorted through this yet. A neutron port could be associated
with a PCI device or not, which is a common feature, IMHO. However, a
ML2 driver may be needed specific to a particular SRIOV technology.
During the Havana Release, we introduced Mellanox Neutron plugin
that enables networking via SRIOV pass-through devices or macvtap
interfaces.
We want to integrate our solution with PCI pass-through Nova
support. I will be glad to share more details if you are interested.
Good to know that you already have a SRIOV implementation. I found out
some information online about the mlnx plugin, but need more time to
get to know it better. And certainly I'm interested in knowing its
details.
The PCI pass-through networking support is planned to be discussed
during the summit: http://summit.openstack.org/cfp/details/129. I
think it’s worth to drill down into more detailed proposal and
present it during the summit, especially since it impacts both
nova and neutron projects.
I agree. Maybe we can steal some time in that discussion.
Would you be interested in collaboration on this effort? Would you
be interested to exchange more emails or set an IRC/WebEx meeting
during this week before the summit?
Sure. If folks want to discuss it before the summit, we can schedule a
webex later this week. Or otherwise, we can continue the discussion
with email.
Regards,
Irena
*From:*Robert Li (baoli) [mailto:[email protected]]
*Sent:* Friday, October 25, 2013 11:16 PM
*To:* [email protected]
<mailto:[email protected]>; Irena Berezovsky;
[email protected] <mailto:[email protected]>;
[email protected] <mailto:[email protected]>;
[email protected] <mailto:[email protected]>
*Cc:* OpenStack Development Mailing List; Brian Bowen (brbowen);
Kyle Mestery (kmestery); Sandhya Dasu (sadasu)
*Subject:* Re: [openstack-dev] [nova] [neutron] PCI pass-through
network support
Hi Irena,
This is Robert Li from Cisco Systems. Recently, I was tasked to
investigate such support for Cisco's systems that support VM-FEX,
which is a SRIOV technology supporting 802-1Qbh. I was able to
bring up nova instances with SRIOV interfaces, and establish
networking in between the instances that employes the SRIOV
interfaces. Certainly, this was accomplished with hacking and some
manual intervention. Based on this experience and my study with
the two existing nova pci-passthrough blueprints that have been
implemented and committed into Havana
(https://blueprints.launchpad.net/nova/+spec/pci-passthrough-base and
https://blueprints.launchpad.net/nova/+spec/pci-passthrough-libvirt),
I registered a couple of blueprints (one on Nova side, the other
on the Neutron side):
https://blueprints.launchpad.net/nova/+spec/pci-passthrough-sriov
https://blueprints.launchpad.net/neutron/+spec/pci-passthrough-sriov
in order to address SRIOV support in openstack.
Please take a look at them and see if they make sense, and let me
know any comments and questions. We can also discuss this in the
summit, I suppose.
I noticed that there is another thread on this topic, so copy
those folks from that thread as well.
thanks,
Robert
On 10/16/13 4:32 PM, "Irena Berezovsky" <[email protected]
<mailto:[email protected]>> wrote:
Hi,
As one of the next steps for PCI pass-through I would like to
discuss is the support for PCI pass-through vNIC.
While nova takes care of PCI pass-through device resources
management and VIF settings, neutron should manage their
networking configuration.
I would like to register asummit proposal to discuss the
support for PCI pass-through networking.
I am not sure what would be the right topic to discuss the PCI
pass-through networking, since it involve both nova and neutron.
There is already a session registered by Yongli on nova topic
to discuss the PCI pass-through next steps.
I think PCI pass-through networking is quite a big topic and
it worth to have a separate discussion.
Is there any other people who are interested to discuss it and
share their thoughts and experience?
Regards,
Irena
_______________________________________________
OpenStack-dev mailing list
[email protected]
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
