Hi Darrell, > -----Original Message----- > From: Darrell Ball [mailto:[email protected]] > Sent: Wednesday, March 29, 2017 6:35 PM > To: O Mahony, Billy <[email protected]>; Jan Scheurich > <[email protected]>; Ilya Maximets <[email protected]>; > [email protected] > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local numa > node. > > I see there is warning log added for the non-local case. > I wonder if during setup, there will be no issues with basic connectivity > seen. > > Then some long time later at high PPS usage, unexpected low performance > will be seen. > I am not sure whether the log will be immediately consulted to check for this, > as the reasons for low PPS performance would likely be elsewhere; this is > pretty subtle. > > I think there needs to be documentation updates associated with this change > explaining this. > Ideally the warning would be referenced in the documentation as clear cross > reference. [[BO'M]] That's a good idea. I'll submit a v2 shortly. > > Thanks > Darrell > > > On 3/9/17, 7:16 AM, "[email protected] on behalf of O > Mahony, Billy" <[email protected] on behalf of > [email protected]> wrote: > > Hi Jan, > > > > Thanks. That is another valid point if you don't - currently if you have > dual- > socket (numa) system then you have to spend at least 2 cores on your ovs- > dpdk vswitch. With this patch it will allow such systems to operate on just > one core (with reduced switching capacity of course). > > > > I'd be interested to hear any other comments from the community on this > patch. > > > > Regards, > > /Billy > > > > > -----Original Message----- > > > From: Jan Scheurich [mailto:[email protected]] > > > Sent: Monday, March 6, 2017 8:25 AM > > > To: O Mahony, Billy <[email protected]>; Ilya Maximets > > > <[email protected]>; [email protected] > > > Subject: RE: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local > numa > > > node. > > > > > > I support Billy's point here. There are a number of > constraints/trade-offs > that > > > can lead to an OVS deployments in OpenStack with e.g. a single PMD. > > > Without this change, a dual socket system is effectively rendered use- > less. At > > > least one socket is lost for running VMs. > > > > > > BR, Jan > > > > > > > -----Original Message----- > > > > From: [email protected] > > > > [mailto:[email protected]] On Behalf Of O Mahony, > Billy > > > > Sent: Tuesday, 28 February, 2017 14:59 > > > > To: Ilya Maximets <[email protected]>; [email protected] > > > > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on non-local > > > numa node. > > > > > > > > Hi Ilya, > > > > > > > > Comments below. > > > > > > > > BR, > > > > Billy. > > > > > > > > > -----Original Message----- > > > > > From: Ilya Maximets [mailto:[email protected]] > > > > > Sent: Tuesday, February 28, 2017 12:49 PM > > > > > To: O Mahony, Billy <[email protected]>; > [email protected] > > > > > Cc: Daniele Di Proietto <[email protected]> > > > > > Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on > > > > > non-local numa node. > > > > > > > > > > On 28.02.2017 14:43, O Mahony, Billy wrote: > > > > > > Hi Ilya, > > > > > > > > > > > > Thanks for the quick response. You make some good points. > > > > > > > > > > > > It's important to point out that local pmds are still chosen when > > > available. > > > > > > The change only operates to avoid totally non-operational/ > > > > > > non-polled > > > > > ports. > > > > > > > > > > > > This is something we came across when deploying DPDK-enabled > OVS > > > > > > in OpenStack environments (OPNFV project). Where we had > remote > > > > > > (both physically and administratively) multi-node labs already > > > > > > wired up and would have much preferred to have sub-optimal > > > > > > operation that a non- > > > > > operational OpenStack environment. > > > > > > > > > > > > Some further comments below. > > > > > > > > > > > > Best Regards, > > > > > > Billy > > > > > > > > > > > >> -----Original Message----- > > > > > >> From: Ilya Maximets [mailto:[email protected]] > > > > > >> Sent: Tuesday, February 28, 2017 11:21 AM > > > > > >> To: O Mahony, Billy <[email protected]>; > > > > > >> [email protected] > > > > > >> Cc: Daniele Di Proietto <[email protected]> > > > > > >> Subject: Re: [ovs-dev] dpif-netdev: Assign ports to pmds on > > > > > >> non-local numa node. > > > > > >> > > > > > >> Hello. > > > > > >> > > > > > >> On 28.02.2017 13:12, Billy O'Mahony wrote: > > > > > >>> From: billyom <[email protected]> > > > > > >>> > > > > > >>> Previously if there is no available (non-isolated) pmd on the > > > > > >>> numa node for a port then the port is not polled at all. This > > > > > >>> can result in a non-operational system until such time as nics > > > > > >>> are physically repositioned. > > > > > >> > > > > > >> Why you can't just reconfigure your pmd-cpu-mask after NICs' > > > > > repositioning? > > > > > > [[BO'M]] The idea is to avoid having to repositioning NICs as they > > > > > > may be in remote data-centers administered by other > organisations. > > > > > > Also this can be related to multi-node clusters where it won't be > > > > > > just one NIC on one system that needs to be moved but several > > > > > > NICS/nodes > > > > > affected. > > > > > > > > > > > > > > > > > >> Maybe you can use pmd-rxq-affinity to assign port on another > NUMA > > > > > node? > > > > > > [[BO'M]] The low-performance assignment will only occur if there > > > > > > is no available PMD on the local NUMA node. Ie if possible the > > > > > > normal highly performant assignment is made. It is only when it is > > > > > > a choice between lower performance and total non-performance > that > > > > > > the lesser of two evils > > > > > is chosen. > > > > > > > > > > Maybe you can include at least one core from each node in pmd-cpu- > > > mask? > > > > > > > > [[BO'M]] Ideally everyone using and developing the many OpenStack > > > > deployment tools would be aware of the optimal settings but we can > > > > expect that this will not the case in many situations. Poor > > > > configurations will occur. A warning message is included in the VLOG > > > > in this case to give users a hint that performance will not be > optimal but > > > without imposing the penalty of a broken system - which is a heavy > penalty if > > > you don’t even have direct access to the pmd-cpu-mask as you are > driving > > > the ovs configuration via an OpenStack installation tool such as Fuel, > RDO, > > > etc. > > > > > > > > > > > > > > >> The main concern here is that this 'remote' port will degrade > > > > > >> performance of other ports served by chosen PMD thread > > > significantly. > > > > > > [[BO'M]] That is a good point that there are second-order > > > > > > consequences on > > > > > other ports. > > > > > > But certainly with a many cloud systems even if one port is > > > > > > non-operational it means the entire node is effectively down - for > > > > > > instance an OpenStack compute node with a non-working provider > > > > > network > > > > > > for it's tenant VMs is useless even though the administration and > > > > > > control > > > > > networks are still working. > > > > > >> > > > > > >> Best regards, Ilya Maximets. > > > > _______________________________________________ > > > > dev mailing list > > > > [email protected] > > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__mail.openvswitch.org_mailman_listinfo_ovs- > 2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=GNRRfh6vBdD0g81qjkYyoE4hWv80KC2GRbQro6HR_go&s=0nEw3l > SrDPhqOGsgc7cUQhvvyh90VzJgqRbMXaEG-B8&e= > > _______________________________________________ > dev mailing list > [email protected] > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__mail.openvswitch.org_mailman_listinfo_ovs- > 2Ddev&d=DwIGaQ&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih- > uZnsw&m=GNRRfh6vBdD0g81qjkYyoE4hWv80KC2GRbQro6HR_go&s=0nEw3l > SrDPhqOGsgc7cUQhvvyh90VzJgqRbMXaEG-B8&e= >
_______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
