I support Billy's point here. There are a number of constraints/trade-offs that can lead to an OVS deployments in OpenStack with e.g. a single PMD. Without this change, a dual socket system is effectively rendered use-less. At least one socket is lost for running VMs.
BR, Jan > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of O Mahony, Billy > Sent: Tuesday, 28 February, 2017 14:59 > To: Ilya Maximets <[email protected]>; [email protected] > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local numa > node. > > Hi Ilya, > > Comments below. > > BR, > Billy. > > > -----Original Message----- > > From: Ilya Maximets [mailto:[email protected]] > > Sent: Tuesday, February 28, 2017 12:49 PM > > To: O Mahony, Billy <[email protected]>; [email protected] > > Cc: Daniele Di Proietto <[email protected]> > > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local numa > > node. > > > > On 28.02.2017 14:43, O Mahony, Billy wrote: > > > Hi Ilya, > > > > > > Thanks for the quick response. You make some good points. > > > > > > It's important to point out that local pmds are still chosen when > > > available. > > > The change only operates to avoid totally non-operational/ non-polled > > ports. > > > > > > This is something we came across when deploying DPDK-enabled OVS in > > > OpenStack environments (OPNFV project). Where we had remote (both > > > physically and administratively) multi-node labs already wired up and > > > would have much preferred to have sub-optimal operation that a non- > > operational OpenStack environment. > > > > > > Some further comments below. > > > > > > Best Regards, > > > Billy > > > > > >> -----Original Message----- > > >> From: Ilya Maximets [mailto:[email protected]] > > >> Sent: Tuesday, February 28, 2017 11:21 AM > > >> To: O Mahony, Billy <[email protected]>; [email protected] > > >> Cc: Daniele Di Proietto <[email protected]> > > >> Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local > > >> numa node. > > >> > > >> Hello. > > >> > > >> On 28.02.2017 13:12, Billy O'Mahony wrote: > > >>> From: billyom <[email protected]> > > >>> > > >>> Previously if there is no available (non-isolated) pmd on the numa > > >>> node for a port then the port is not polled at all. This can result > > >>> in a non-operational system until such time as nics are physically > > >>> repositioned. > > >> > > >> Why you can't just reconfigure your pmd-cpu-mask after NICs' > > repositioning? > > > [[BO'M]] The idea is to avoid having to repositioning NICs as they may > > > be in remote data-centers administered by other organisations. Also > > > this can be related to multi-node clusters where it won't be just one > > > NIC on one system that needs to be moved but several NICS/nodes > > affected. > > > > > > > > >> Maybe you can use pmd-rxq-affinity to assign port on another NUMA > > node? > > > [[BO'M]] The low-performance assignment will only occur if there is no > > > available PMD on the local NUMA node. Ie if possible the normal highly > > > performant assignment is made. It is only when it is a choice between > > > lower performance and total non-performance that the lesser of two evils > > is chosen. > > > > Maybe you can include at least one core from each node in pmd-cpu-mask? > > [[BO'M]] Ideally everyone using and developing the many OpenStack deployment > tools would be aware of the optimal settings but we can > expect that this will not the case in many situations. Poor configurations > will occur. A warning message is included in the VLOG in this case > to give users a hint that performance will not be optimal but without > imposing the penalty of a broken system - which is a heavy penalty if > you don’t even have direct access to the pmd-cpu-mask as you are driving the > ovs configuration via an OpenStack installation tool such as > Fuel, RDO, etc. > > > > > >> The main concern here is that this 'remote' port will degrade > > >> performance of other ports served by chosen PMD thread significantly. > > > [[BO'M]] That is a good point that there are second-order consequences on > > other ports. > > > But certainly with a many cloud systems even if one port is > > > non-operational it means the entire node is effectively down - for > > > instance an OpenStack compute node with a non-working provider > > network > > > for it's tenant VMs is useless even though the administration and control > > networks are still working. > > >> > > >> Best regards, Ilya Maximets. > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
