> -----Original Message----- > From: [email protected] [mailto:ovs-dev- > [email protected]] On Behalf Of Kevin Traynor > Sent: Friday, May 5, 2017 5:34 PM > To: [email protected] > Subject: [ovs-dev] [RFC PATCH 0/6] Change dpdk rxq scheduling to > incorporate rxq processing cycles. > > Rxqs are scheduled to be handled across available pmds in round robin > order with no weight or priority. > > It can happen that some very busy queues are handled by one pmd which does > not have enough cycles to prevent packets being dropped on them. > While at the same time another pmd which handles queues with no traffic on > them, is essentially idling. > > Rxq scheduling happens as a result of a number of events and when it does, > the same unweighted round robin approach is applied each time. > > This patchset proposes to augment the round robin nature of rxq scheduling > by counting the processing cycles used by the rxqs during their operation > and incorporate it into the rxq scheduling. > > Before distributing in a round robin manner, the rxqs will be sorted in > order of the processing cycles they have been consuming. Assuming multiple > pmds, this ensures that the measured rxqs using most processing cycles > will be distributed to different cores. > > To try out: > This patchset requires the updated pmd counting patch applied as a > prerequisite. https://patchwork.ozlabs.org/patch/729970/ > > Alternatively the series with dependencies can be cloned from here: > https://github.com/kevintraynor/ovs-rxq.git > > Simple way to test is add some dpdk ports, add multiple pmds, vary traffic > rates and rxqs on ports and trigger reschedules e.g. by changing rxqs or > the pmd-cpu-mask. > > Check rxq distribution with ovs-appctl dpif-netdev/pmd-rxq-show and see if > it matches expected. > > todo: > -possibly add a dedicated reschedule trigger command -use consistent type > names -update docs -more testing, especially for dual numa > > thanks, > Kevin. Hi Kevin,
Overall I like the idea, I've done a first pass review and some initial testing. I have a few comments on the patches. In terms of how it interacts with numa, have you looked at Billy's patch for port pmds on non local numa nodes? It touches the same rxq_scheduling code you've modified but it would be useful for DPDK ports and testing numa deployments. It also introduces a log message to flag what pmd and core an rxq is assigned to so the user doesn't need to call ovs-appctl dpif-netdev/pmd-rxq-show, I find that useful for debugging and would be useful here as well I think. https://mail.openvswitch.org/pipermail/ovs-dev/2017-May/332318.html Ian > > Kevin Traynor (6): > dpif-netdev: Add rxq processing cycle counters. > dpif-netdev: Update rxq processing cycles from > cycles_count_intermediate. > dpif-netdev: Change polled_queue to use dp_netdev_rxq. > dpif-netdev: Make dpcls optimization interval more generic. > dpif-netdev: Count the rxq processing cycles for an rxq. > dpif-netdev: Change rxq_scheduling to use rxq processing cycles. > > lib/dpif-netdev.c | 163 ++++++++++++++++++++++++++++++++++++++++++++----- > ----- > 1 file changed, 133 insertions(+), 30 deletions(-) > > -- > 1.8.3.1 > > _______________________________________________ > dev mailing list > [email protected] > https://mail.openvswitch.org/mailman/listinfo/ovs-dev _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
