Re: [openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support

2017-09-07 Thread Eric Fried 
Stephen-

FYI, we're teeing up (anti-)affinity and neutron-nova interaction
topics for the "generic device management" discussions at the PTG.
Please jump on the etherpad [1] and expand/expound in the relevant
spots, which, as of this writing, you can find by searching for
"aggregate", "vf selection policy", and "interplay".

Thanks,
Eric

[1]
https://etherpad.openstack.org/p/nova-ptg-queens-generic-device-management

On 09/07/2017 01:07 PM, Mooney, Sean K wrote:
> 
> 
>> -Original Message-
>> From: Stephen Finucane [mailto:sfinu...@redhat.com]
>> Sent: Thursday, September 7, 2017 5:42 PM
>> To: OpenStack Development Mailing List (not for usage questions)
>> 
>> Cc: Jakub Libosvar ; Karthik Sundaravel
>> ; Mooney, Sean K 
>> Subject: [nova] [neutron] Adding neutron VIF NUMA locality support
>>
>> Hey,
>>
>> NUMA locality matters as much for NICs used e.g for Open vSwitch as for
>> SR-IOV devices. At the moment, nova support NUMA affinity for PCI
>> passthrough devices and SR-IOV devices, but it makes no attempt to do
>> the same for other NICs. In the name of NFV enablement, we should
>> probably close this gap.
> [Mooney, Sean K] I like this idea in general, that said in ovs-dpdk we 
> modified
> ovs to schedule the vhost-user port to be processed on a pmd that is on the 
> same
> Numa node as the vm and reallocate the vhsot user port memory where possible
> To also have the same affinity. 
>>
>> I have some ideas around how this could work, but they're fuzzy enough
>> and involve exchanging os-vif objects between nova and neutron. This is
>> probably the most difficult path as we've been trying to get os-vif
>> objects over the nova-neutron wire for a while now, to no success.
> [Mooney, Sean K] actually we have so poc code you should proably review
> This topic.
> https://blueprints.launchpad.net/os-vif/+spec/vif-port-profile
> https://review.openstack.org/#/c/490829/ 
> https://review.openstack.org/#/c/490819/ 
> https://review.openstack.org/#/c/441590/
> the first patch of the neutron side poc should be up before the ptg.
> 
>>
>> Anyone else keen on such a feature? Given that there are a significant
>> amount of people from nova, neutron, and general NFV backgrounds at the
>> PTG next week, we have a very good opportunity to talk about this
>> (either in the nova- neutron sync, if that's not already full, or in
>> some hallway somewhere).
> [Mooney, Sean K] in terms of basic numa affinity this is not as important
> With ovs-dpdk because we make best effort to fix it in ovs this is less 
> pressing
> Then it used to be. It is still important for other backbends but we need
> Also have a mechanism to control numa affinity policy like 
>  https://review.openstack.org/#/c/361140/ to not break existing deployments.
> 
> I have some taught about modeling network backbends
> in placement and also passing traits requests for neutron that this would dove
> tail with so would love to talk to anyone who is interested in this.
> By modeling ovs and other network backend in placement and combining that
> With traits and the nova-neutron negotiation protocol we support several
> Advance usescase.
> 
> By the way  ovs-dpdk allow you to specify vhost-port rx/tx queue mapping 
> to pmd which could give a nice performance boost if done correctly. It
> might be worth extending os-vif to do that in the future though this could
> equally be handeled by the neutron ovs l2 agent.
>>
>> At this point in the day, this is probably very much a Rocky feature,
>> but we could definitely put in whatever groundwork is necessary this
>> cycle to make the work in Rocky as easy possible.
> [Mooney, Sean K] I'm hoping we can get the nova neutron negotiation done in 
> queens.
>>
>> Cheers,
>> Stephen
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support

2017-09-07 Thread Mooney, Sean K 


> -Original Message-
> From: Stephen Finucane [mailto:sfinu...@redhat.com]
> Sent: Thursday, September 7, 2017 5:42 PM
> To: OpenStack Development Mailing List (not for usage questions)
> 
> Cc: Jakub Libosvar ; Karthik Sundaravel
> ; Mooney, Sean K 
> Subject: [nova] [neutron] Adding neutron VIF NUMA locality support
> 
> Hey,
> 
> NUMA locality matters as much for NICs used e.g for Open vSwitch as for
> SR-IOV devices. At the moment, nova support NUMA affinity for PCI
> passthrough devices and SR-IOV devices, but it makes no attempt to do
> the same for other NICs. In the name of NFV enablement, we should
> probably close this gap.
[Mooney, Sean K] I like this idea in general, that said in ovs-dpdk we modified
ovs to schedule the vhost-user port to be processed on a pmd that is on the same
Numa node as the vm and reallocate the vhsot user port memory where possible
To also have the same affinity. 
> 
> I have some ideas around how this could work, but they're fuzzy enough
> and involve exchanging os-vif objects between nova and neutron. This is
> probably the most difficult path as we've been trying to get os-vif
> objects over the nova-neutron wire for a while now, to no success.
[Mooney, Sean K] actually we have so poc code you should proably review
This topic.
https://blueprints.launchpad.net/os-vif/+spec/vif-port-profile
https://review.openstack.org/#/c/490829/ 
https://review.openstack.org/#/c/490819/ 
https://review.openstack.org/#/c/441590/
the first patch of the neutron side poc should be up before the ptg.

> 
> Anyone else keen on such a feature? Given that there are a significant
> amount of people from nova, neutron, and general NFV backgrounds at the
> PTG next week, we have a very good opportunity to talk about this
> (either in the nova- neutron sync, if that's not already full, or in
> some hallway somewhere).
[Mooney, Sean K] in terms of basic numa affinity this is not as important
With ovs-dpdk because we make best effort to fix it in ovs this is less pressing
Then it used to be. It is still important for other backbends but we need
Also have a mechanism to control numa affinity policy like 
 https://review.openstack.org/#/c/361140/ to not break existing deployments.

I have some taught about modeling network backbends
in placement and also passing traits requests for neutron that this would dove
tail with so would love to talk to anyone who is interested in this.
By modeling ovs and other network backend in placement and combining that
With traits and the nova-neutron negotiation protocol we support several
Advance usescase.

By the way  ovs-dpdk allow you to specify vhost-port rx/tx queue mapping 
to pmd which could give a nice performance boost if done correctly. It
might be worth extending os-vif to do that in the future though this could
equally be handeled by the neutron ovs l2 agent.
> 
> At this point in the day, this is probably very much a Rocky feature,
> but we could definitely put in whatever groundwork is necessary this
> cycle to make the work in Rocky as easy possible.
[Mooney, Sean K] I'm hoping we can get the nova neutron negotiation done in 
queens.
> 
> Cheers,
> Stephen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] [neutron] Adding neutron VIF NUMA locality support

2017-09-07 Thread Stephen Finucane 
Hey,

NUMA locality matters as much for NICs used e.g for Open vSwitch as for SR-IOV
devices. At the moment, nova support NUMA affinity for PCI passthrough devices
and SR-IOV devices, but it makes no attempt to do the same for other NICs. In
the name of NFV enablement, we should probably close this gap.

I have some ideas around how this could work, but they're fuzzy enough and
involve exchanging os-vif objects between nova and neutron. This is probably
the most difficult path as we've been trying to get os-vif objects over the
nova-neutron wire for a while now, to no success.

Anyone else keen on such a feature? Given that there are a significant amount
of people from nova, neutron, and general NFV backgrounds at the PTG next
week, we have a very good opportunity to talk about this (either in the nova-
neutron sync, if that's not already full, or in some hallway somewhere).

At this point in the day, this is probably very much a Rocky feature, but we
could definitely put in whatever groundwork is necessary this cycle to make the
work in Rocky as easy possible.

Cheers,
Stephen

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev