> -----Original Message----- > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev- > boun...@openvswitch.org] On Behalf Of Chandran, Sugesh > Sent: Thursday, December 7, 2017 5:07 PM > To: O Mahony, Billy <billy.o.mah...@intel.com>; d...@openvswitch.org; > b...@ovn.org > Subject: Re: [ovs-dev] [PATCH 2/2] Adding configuration option to > whitelist DPDK physical ports. > > > > Regards > _Sugesh > > > -----Original Message----- > > From: O Mahony, Billy > > Sent: Thursday, December 7, 2017 11:47 AM > > To: Chandran, Sugesh <sugesh.chand...@intel.com>; > d...@openvswitch.org; > > b...@ovn.org > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to > > whitelist DPDK physical ports. > > > > Hi Sugesh, > > > > > -----Original Message----- > > > From: Chandran, Sugesh > > > Sent: Wednesday, December 6, 2017 6:23 PM > > > To: O Mahony, Billy <billy.o.mah...@intel.com>; > d...@openvswitch.org; > > > b...@ovn.org > > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to > > > whitelist DPDK physical ports. > > > > > > Thank you Billy for the review. > > > Please find below my reply. > > > > > > Regards > > > _Sugesh > > > > > > > > > > -----Original Message----- > > > > From: O Mahony, Billy > > > > Sent: Wednesday, December 6, 2017 5:31 PM > > > > To: Chandran, Sugesh <sugesh.chand...@intel.com>; > > > > d...@openvswitch.org; b...@ovn.org > > > > Subject: RE: [ovs-dev] [PATCH 2/2] Adding configuration option to > > > > whitelist DPDK physical ports. > > > > > > > > Hi Sugesh, > > > > > > > > This is definitely a very useful feature. I'm looking forward to > > > > running trex on the same DUT as my ovs-dpdk. [Mooney, Sean K] you can all ready to this you just need to set the whitelist In other_config:dpdk-extra just repeat "-w $address" for each device. To have two dpdk primary processes on the same system you will also need to change The hugepage prfix used be dpdk which you can also do via the dpdk-extra option.
After this patch we will still be able to specify the whitelist using other_config:dpdk-extra correct? If not this may break ovs-dpdk support in openstack installers. I ported our whitelist code in networking-ovs-dpdk to use dpdk-extra when when we moved the dpdk params to the db and I also added it to kolla. im pretty sure tripple0 and fule also do the same. > > > > > > > > However I'd suggest adding an sscanf or some such to verify that > > > > the domain is also specified for each whitelist member. And > either > > > > add the default of '0000' or complain loudly if the domain is > absent. > > > [Sugesh] Will throw an error in that case then . > > > > > > > > > > > Currently (without this patch) you must specify the domain when > > > > adding > > ports: > > > > Vsctl add-port ... options:dpdk-devargs=0000:05:00.0 Or else > an > > > > error such as 'Cannot find unplugged device (05:00.0)' is > reported. > > > > > > > > And with the patch if you include the domain in the other_config > (e.g. > > > > other_config:dpdk-whitelist-pci-ids="0000:05:00.0") everything > > > > works just as before. > > > > > > > > However with the patch if you add the whitelist *without* a > domain e.g. > > > > ovs-vsctl --no-wait set Open_vSwitch . > > > > other_config:dpdk-whitelist-pci- ids="05:00.0" > > > > > > > > There is no immediate error. However later when doing add-port if > > > > you include the domain (current required practice) you will get > an error. > > > > If you omit the domain all is well. > > > [Sugesh] It looks to me, the dpdk-devargs need the PCI id with the > '0000'. > > > But to bind and PCI scan its not necessary. > > > So to keep it consistent, I would add check for PCI-ID in whitelist > > > config too, and throw error incase pci-id are mentioned wrong(means > > > without > > '0000'. [Mooney, Sean K] don't assume it is '0000' it is only '0000' if the pci device is A child of the first pci root usally connected to socket 0 unless you server is old Enough to still use a FSB instead of qpi/dmi. > > > Does it looks OK to you? > > > > [[BO'M]] I think the error is the right thing to do. It would be > > tempting to insert the default '0000' if the domain is omitted but > > then you would have a confusing inconsistency in that it would be ok > > to omit the domain in one place (whitelist) but not in the other > (add-port). > > > [Sugesh] Thank you Billy,Will add the check and error out accordingly. > Will release the next patch version by incorporating the comments, once > you have complete the review of second part of the patch too. [Mooney, Sean K] it should be noted that the dpdk devbing script will only work without The domain if its unambiguious. On a multi socket system with the right or wrong depending on your perspective placement of physical pci devise it possible to get colltions in which case dpdk-devbind will not be able to correctly determing what device to manage so you should always use the full pci address. > > > > > > > > It's a little bit strange as regardless of domain or no domain in > > > > the other_config the PCI probe always reports the NIC as > expected: > > > > 2017-12-06T16:55:27Z|00013|dpdk|INFO|EAL: PCI device > > > > 0000:05:00.0 on NUMA socket -1 > > > > 2017-12-06T16:55:27Z|00014|dpdk|WARN|EAL: Invalid NUMA > socket, > > > > default to 0 > > > > 2017-12-06T16:55:27Z|00015|dpdk|INFO|EAL: probe driver: > 8086:1572 > > > > net_i40e > > > > > > > > I'll be using the other patch in this series "isolate rte-mempool > > > > allocation" over the next few days so I'll review that in due > course. > > > > > > > > Thanks, > > > > Billy. > > > > > > > > > -----Original Message----- > > > > > From: ovs-dev-boun...@openvswitch.org [mailto:ovs-dev- > > > > > boun...@openvswitch.org] On Behalf Of Sugesh Chandran > > > > > Sent: Friday, November 10, 2017 1:29 AM > > > > > To: d...@openvswitch.org; b...@ovn.org > > > > > Subject: [ovs-dev] [PATCH 2/2] Adding configuration option to > > > > > whitelist DPDK physical ports. > > > > > > > > > > Adding a OVS configuration option to whitelist DPDK physical > ports. > > > > > By default running multiple instances of DPDK on a single > > > > > platform cannot use physical ports at the same time even though > they are distinct. > > > > > > > > > > The eal init scans all the ports that are bound to DPDK and > > > > > initialize the drivers accordingly. This happens for every DPDK > process init. > > > > > On a multi instance deployment usecase, it causes issues for > > > > > using physical NIC ports. > > > > > Consider a two DPDK process that are running on a single > > > > > platform, the second DPDK primary process will try to > initialize > > > > > the drivers for all the physical ports even though it may be > used in first DPDK process. > > > > > > > > > > To avoid this situation user can whitelist the ports for each > > > > > DPDK > > application. > > > > > Whitelisting of ports/PCI-ID in a DPDK process will limit the > > > > > eal-init only on those ports. > > > > > > > > > > To whitelist two physical ports "0000:06:00.0" and > > > > > "0000:06:00.1", the configuration option in OVS would be > > > > > ovs-vsctl set Open_vSwitch . other_config:dpdk-whitelist- > pci- > > > > > ids="0000:06:00.0,0000:06:00.1" > > > > > > > > > > To update the whitelist ports, OVS daemon has to be restarted. > > > > > > > > > > Signed-off-by: Sugesh Chandran <sugesh.chand...@intel.com> > > > > > --- > > > > > lib/dpdk.c | 29 +++++++++++++++++++++++++++++ > [snip] > > > > > > > > > > _______________________________________________ > > > > > dev mailing list > > > > > d...@openvswitch.org > > > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
