FYI: I created blueprint https://blueprints.launchpad.net/nova/+spec/share-pci-device-between-numa-nodes and want to discuss it in Austin.
2016-03-25 9:49 GMT+03:00 Sergey Nikitin <sniki...@mirantis.com>: > Guys, thank you for fast response. I'm glad that I'm not a single one who > face this problem. > > 2016-03-24 19:54 GMT+03:00 Czesnowicz, Przemyslaw < > przemyslaw.czesnow...@intel.com>: > >> >> >> > -----Original Message----- >> > From: Nikola Đipanov [mailto:ndipa...@redhat.com] >> > Sent: Thursday, March 24, 2016 4:34 PM >> > To: Sergey Nikitin <sniki...@mirantis.com>; OpenStack Development >> Mailing >> > List (not for usage questions) <openstack-dev@lists.openstack.org> >> > Cc: Czesnowicz, Przemyslaw <przemyslaw.czesnow...@intel.com> >> > Subject: Re: [openstack-dev] [nova] NUMA + SR-IOV >> > >> > On 03/24/2016 04:18 PM, Sergey Nikitin wrote: >> > > >> > > Hi, folks. >> > > >> > > I want to start a discussion about NUMA + SR-IOV environment. I have a >> > > two-sockets server. It has two NUMA nodes and only one SR-IOV PCI >> > > device. This device is associated with the first NUMA node. I booted a >> > > set of VMs with SR-IOV support. Each of these VMs was booted on the >> > > first NUMA node. As I understand it happened for better performance >> > > (VM should be booted in NUMA node which has PCI device for this VM) >> > [1]. >> > > >> > > But this behavior leaves my 2-sockets machines half-populated. What if >> > > I don't care about SR-IOV performance? I just want every VM from *any* >> > > of NUMA nodes to use this single SR-IOV PCI device. >> > > >> > > But I can't do it because of behavior of numa_topology_filter. In this >> > > filter we want to know if current host has required PCI device [2]. >> > > But we want to have this device *only* in some numa cell on this host. >> > > It is hardcoded here [3]. If we do *not* pass variable "cells" to the >> > > method >> > > support_requests() [4] we will boot VM on the current host, if it has >> > > required PCI device *on host* (maybe not in the same NUMA node). >> > > >> > > So my question is: >> > > Is it correct that we *always* want to boot VM in NUMA node associated >> > > with requested PCI device and user has no choice? >> > > Or should we give a choice to the user and let him boot a VM with PCI >> > > device, associated with another NUMA node? >> > > >> >> The rationale for choosing this behavior was that if you are requiring a >> NUMA topology for your VM >> and you request an SRIOV device as well then this is an high performance >> application and it should be configured appropriately. >> >> Similarly if you request hugepages your VM will be confined to one NUMA >> (unless specified otherwise) >> node and if there is no single NUMA node with enough resources it won't >> be created. >> >> >> > >> > This has come up before, and the fact that it keeps coming up tells me >> that >> > we should probably do something about it. >> > >> > Potentially it makes sense to be lax by default unless user specifies >> that they >> > want to make sure that the device is on the same NUMA node, but that is >> > not backwards compatible. >> > >> > It does not make sense to ask user to specify that they don't care >> IMHO, as >> > unless you know there is a problem (and users have nowhere near enough >> > information to tell), there is no reason for you to specify it - it's >> just not >> > sensible UI IMHO. >> > >> >> Yes this did come up few times, having a way to specify a requirement is >> probably a good idea. >> If it would be done the way you propose that would change the behavior >> for existing users, not sure how big problem this is. >> >> Przemek >> >> > My 0.02 cents. >> >> >> >> >> >> > >> > N. >> > >> > > >> > > [1] >> > > https://specs.openstack.org/openstack/nova- >> > specs/specs/kilo/implemente >> > > d/input-output-based-numa-scheduling.html >> > > [2] >> > > >> > https://github.com/openstack/nova/blob/master/nova/scheduler/filters/n >> > > uma_topology_filter.py#L85 >> > > [3] >> > > >> > https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L >> > 1 >> > > 246-L1247 [4] >> > > https://github.com/openstack/nova/blob/master/nova/pci/stats.py#L277 >> >> >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev