I can confirm that using other_config:dpdk-extra is indeed already effective to change the hugepage file prefix (admittedly without inserting a ref to the pid) and specify a pci whitelist.
Regards, Billy. > -----Original Message----- > From: Mooney, Sean K > Sent: Thursday, December 7, 2017 5:53 PM > To: Chandran, Sugesh <sugesh.chand...@intel.com>; O Mahony, Billy > <billy.o.mah...@intel.com>; d...@openvswitch.org; b...@ovn.org > Cc: Mooney, Sean K <sean.k.moo...@intel.com> > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to whitelist > DPDK physical ports. > > > > > -----Original Message----- > > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev- > > boun...@openvswitch.org] On Behalf Of Chandran, Sugesh > > Sent: Thursday, December 7, 2017 5:07 PM > > To: O Mahony, Billy <billy.o.mah...@intel.com>; d...@openvswitch.org; > > b...@ovn.org > > Subject: Re: [ovs-dev] [PATCH 2/2] Adding configuration option to > > whitelist DPDK physical ports. > > > > > > > > Regards > > _Sugesh > > > > > -----Original Message----- > > > From: O Mahony, Billy > > > Sent: Thursday, December 7, 2017 11:47 AM > > > To: Chandran, Sugesh <sugesh.chand...@intel.com>; > > d...@openvswitch.org; > > > b...@ovn.org > > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to > > > whitelist DPDK physical ports. > > > > > > Hi Sugesh, > > > > > > > -----Original Message----- > > > > From: Chandran, Sugesh > > > > Sent: Wednesday, December 6, 2017 6:23 PM > > > > To: O Mahony, Billy <billy.o.mah...@intel.com>; > > d...@openvswitch.org; > > > > b...@ovn.org > > > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to > > > > whitelist DPDK physical ports. > > > > > > > > Thank you Billy for the review. > > > > Please find below my reply. > > > > > > > > Regards > > > > _Sugesh > > > > > > > > > > > > > -----Original Message----- > > > > > From: O Mahony, Billy > > > > > Sent: Wednesday, December 6, 2017 5:31 PM > > > > > To: Chandran, Sugesh <sugesh.chand...@intel.com>; > > > > > d...@openvswitch.org; b...@ovn.org > > > > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option > > > > > to whitelist DPDK physical ports. > > > > > > > > > > Hi Sugesh, > > > > > > > > > > This is definitely a very useful feature. I'm looking forward to > > > > > running trex on the same DUT as my ovs-dpdk. > [Mooney, Sean K] you can all ready to this you just need to set the > whitelist In > other_config:dpdk-extra just repeat "-w $address" for each device. > To have two dpdk primary processes on the same system you will also need to > change The hugepage prfix used be dpdk which you can also do via the dpdk- > extra option. > > After this patch we will still be able to specify the whitelist using > other_config:dpdk-extra correct? If not this may break ovs-dpdk support in > openstack installers. I ported our whitelist code in networking-ovs-dpdk to > use > dpdk-extra when when we moved the dpdk params to the db and I also added it > to kolla. > im pretty sure tripple0 and fule also do the same. > > > > > > > > > > > However I'd suggest adding an sscanf or some such to verify that > > > > > the domain is also specified for each whitelist member. And > > either > > > > > add the default of '0000' or complain loudly if the domain is > > absent. > > > > [Sugesh] Will throw an error in that case then . > > > > > > > > > > > > > > Currently (without this patch) you must specify the domain when > > > > > adding > > > ports: > > > > > Vsctl add-port ... options:dpdk-devargs=0000:05:00.0 Or else > > an > > > > > error such as 'Cannot find unplugged device (05:00.0)' is > > reported. > > > > > > > > > > And with the patch if you include the domain in the other_config > > (e.g. > > > > > other_config:dpdk-whitelist-pci-ids="0000:05:00.0") everything > > > > > works just as before. > > > > > > > > > > However with the patch if you add the whitelist *without* a > > domain e.g. > > > > > ovs-vsctl --no-wait set Open_vSwitch . > > > > > other_config:dpdk-whitelist-pci- ids="05:00.0" > > > > > > > > > > There is no immediate error. However later when doing add-port > > > > > if you include the domain (current required practice) you will > > > > > get > > an error. > > > > > If you omit the domain all is well. > > > > [Sugesh] It looks to me, the dpdk-devargs need the PCI id with the > > '0000'. > > > > But to bind and PCI scan its not necessary. > > > > So to keep it consistent, I would add check for PCI-ID in > > > > whitelist config too, and throw error incase pci-id are mentioned > > > > wrong(means without > > > '0000'. > [Mooney, Sean K] don't assume it is '0000' it is only '0000' if the pci > device is A > child of the first pci root usally connected to socket 0 unless you server is > old > Enough to still use a FSB instead of qpi/dmi. > > > > > Does it looks OK to you? > > > > > > [[BO'M]] I think the error is the right thing to do. It would be > > > tempting to insert the default '0000' if the domain is omitted but > > > then you would have a confusing inconsistency in that it would be ok > > > to omit the domain in one place (whitelist) but not in the other > > (add-port). > > > > > [Sugesh] Thank you Billy,Will add the check and error out accordingly. > > Will release the next patch version by incorporating the comments, > > once you have complete the review of second part of the patch too. > [Mooney, Sean K] it should be noted that the dpdk devbing script will only > work > without The domain if its unambiguious. On a multi socket system with the > right > or wrong depending on your perspective placement of physical pci devise it > possible to get colltions in which case dpdk-devbind will not be able to > correctly > determing what device to manage so you should always use the full pci address. > > > > > > > > > > > It's a little bit strange as regardless of domain or no domain > > > > > in the other_config the PCI probe always reports the NIC as > > expected: > > > > > 2017-12-06T16:55:27Z|00013|dpdk|INFO|EAL: PCI device > > > > > 0000:05:00.0 on NUMA socket -1 > > > > > 2017-12-06T16:55:27Z|00014|dpdk|WARN|EAL: Invalid NUMA > > socket, > > > > > default to 0 > > > > > 2017-12-06T16:55:27Z|00015|dpdk|INFO|EAL: probe driver: > > 8086:1572 > > > > > net_i40e > > > > > > > > > > I'll be using the other patch in this series "isolate > > > > > rte-mempool allocation" over the next few days so I'll review > > > > > that in due > > course. > > > > > > > > > > Thanks, > > > > > Billy. > > > > > > > > > > > -----Original Message----- > > > > > > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev- > > > > > > boun...@openvswitch.org] On Behalf Of Sugesh Chandran > > > > > > Sent: Friday, November 10, 2017 1:29 AM > > > > > > To: d...@openvswitch.org; b...@ovn.org > > > > > > Subject: [ovs-dev] [PATCH 2/2] Adding configuration option to > > > > > > whitelist DPDK physical ports. > > > > > > > > > > > > Adding a OVS configuration option to whitelist DPDK physical > > ports. > > > > > > By default running multiple instances of DPDK on a single > > > > > > platform cannot use physical ports at the same time even > > > > > > though > > they are distinct. > > > > > > > > > > > > The eal init scans all the ports that are bound to DPDK and > > > > > > initialize the drivers accordingly. This happens for every > > > > > > DPDK > > process init. > > > > > > On a multi instance deployment usecase, it causes issues for > > > > > > using physical NIC ports. > > > > > > Consider a two DPDK process that are running on a single > > > > > > platform, the second DPDK primary process will try to > > initialize > > > > > > the drivers for all the physical ports even though it may be > > used in first DPDK process. > > > > > > > > > > > > To avoid this situation user can whitelist the ports for each > > > > > > DPDK > > > application. > > > > > > Whitelisting of ports/PCI-ID in a DPDK process will limit the > > > > > > eal-init only on those ports. > > > > > > > > > > > > To whitelist two physical ports "0000:06:00.0" and > > > > > > "0000:06:00.1", the configuration option in OVS would be > > > > > > ovs-vsctl set Open_vSwitch . other_config:dpdk-whitelist- > > pci- > > > > > > ids="0000:06:00.0,0000:06:00.1" > > > > > > > > > > > > To update the whitelist ports, OVS daemon has to be restarted. > > > > > > > > > > > > Signed-off-by: Sugesh Chandran <sugesh.chand...@intel.com> > > > > > > --- > > > > > > lib/dpdk.c | 29 +++++++++++++++++++++++++++++ > > [snip] > > > > > > > > > > > > _______________________________________________ > > > > > > dev mailing list > > > > > > d...@openvswitch.org > > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > > _______________________________________________ > > dev mailing list > > d...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev