On 7/22/21 3:25 PM, Kevin Traynor wrote: > The 'Port/Rx Queue Assigment to PMD Threads' section has > expanded over time and now includes info about stats/commands, > manual pinning and different options for OVS assigning Rxqs to > PMDs. > > Split them into different sections with sub-headings and move > the two similar paragraphs about stats together. > > Rename 'Automatic assignment of Port/Rx Queue to PMD Threads' > section to 'PMD Automatic Load Balance'. > > A few other minor cleanups as I was reading. > > Signed-off-by: Kevin Traynor <[email protected]> > --- > Documentation/topics/dpdk/pmd.rst | 101 ++++++++++++++++-------------- > 1 file changed, 54 insertions(+), 47 deletions(-) > > diff --git a/Documentation/topics/dpdk/pmd.rst > b/Documentation/topics/dpdk/pmd.rst > index 95fa7af12..45c24d100 100644 > --- a/Documentation/topics/dpdk/pmd.rst > +++ b/Documentation/topics/dpdk/pmd.rst > @@ -75,8 +75,47 @@ for enabling things like multiqueue for :ref:`physical > <dpdk-phy-multiqueue>` > and :ref:`vhost-user <dpdk-vhost-user>` interfaces. > > -To show port/Rx queue assignment:: > +Rx queues will be assigned to PMD threads by OVS, or they can be manually > +pinned to PMD threads by the user. > +
Nit: double space between "be" and "manually" > +To see the port/Rx queue assignment and current measured usage history of PMD > +core cycles for each Rx queue:: > > $ ovs-appctl dpif-netdev/pmd-rxq-show > > +.. note:: > + > + A history of one minute is recorded and shown for each Rx queue to allow > for > + traffic pattern spikes. Any changes in the Rx queue's PMD core cycles > usage, > + due to traffic pattern or reconfig changes, will take one minute to be > fully > + reflected in the stats. > + > +.. versionchanged:: 2.16.0 > + > + A ``overhead`` statistics is shown per PMD: it represents the number of > + cycles inherently consumed by the OVS PMD processing loop. > + > +Rx queue to PMD assignment takes place whenever there are configuration > changes > +or can be triggered by using:: > + > + $ ovs-appctl dpif-netdev/pmd-rxq-rebalance > + > +.. versionchanged:: 2.6.0 > + > + The ``pmd-rxq-show`` command was added in OVS 2.6.0. > + > +.. versionchanged:: 2.9.0 > + > + Utilization-based allocation of Rx queues to PMDs and the > + ``pmd-rxq-rebalance`` command were added in OVS 2.9.0. Prior to this, > + allocation was round-robin and processing cycles were not taken into > + consideration. > + > + In addition, the output of ``pmd-rxq-show`` was modified to include > + Rx queue utilization of the PMD as a percentage. Prior to this, tracking > of > + stats was not available. > + > + > +Port/Rx Queue assignment to PMD threads by manual pinning > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Rx queues may be manually pinned to cores. This will change the default Rx > queue assignment to PMD threads:: > @@ -117,4 +156,6 @@ If using ``pmd-rxq-assign=group`` PMD threads with > *pinned* Rxqs can be > a *non-isolated* PMD, that will remain *non-isolated*. > > +Automatic Port/Rx Queue assignment to PMD threads > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > If ``pmd-rxq-affinity`` is not set for Rx queues, they will be assigned to > PMDs > (cores) automatically. > @@ -126,7 +167,9 @@ The algorithm used to automatically assign Rxqs to PMDs > can be set by:: > By default, ``cycles`` assignment is used where the Rxqs will be ordered by > their measured processing cycles, and then be evenly assigned in descending > -order to PMDs based on an up/down walk of the PMDs. For example, where there > -are five Rx queues and three cores - 3, 7, and 8 - available and the measured > -usage of core cycles per Rx queue over the last interval is seen to be: > +order to PMDs. The PMD that will be selected for a given Rxq will be the next > +one in alternating ascending/descending order based on core id. For example, > +where there are five Rx queues and three cores - 3, 7, and 8 - available and > +the measured usage of core cycles per Rx queue over the last interval is seen > +to be: > > - Queue #0: 30% > @@ -184,43 +227,6 @@ The Rx queues may be assigned to the cores in the > following order:: > Core 8: P1Q0 | > > -To see the current measured usage history of PMD core cycles for each Rx > -queue:: > - > - $ ovs-appctl dpif-netdev/pmd-rxq-show > - > -.. note:: > - > - A history of one minute is recorded and shown for each Rx queue to allow > for > - traffic pattern spikes. Any changes in the Rx queue's PMD core cycles > usage, > - due to traffic pattern or reconfig changes, will take one minute to be > fully > - reflected in the stats. > - > -.. versionchanged:: 2.16.0 > - > - A ``overhead`` statistics is shown per PMD: it represents the number of > - cycles inherently consumed by the OVS PMD processing loop. > - > -Rx queue to PMD assignment takes place whenever there are configuration > changes > -or can be triggered by using:: > - > - $ ovs-appctl dpif-netdev/pmd-rxq-rebalance > - > -.. versionchanged:: 2.6.0 > - > - The ``pmd-rxq-show`` command was added in OVS 2.6.0. > - > -.. versionchanged:: 2.9.0 > - > - Utilization-based allocation of Rx queues to PMDs and the > - ``pmd-rxq-rebalance`` command were added in OVS 2.9.0. Prior to this, > - allocation was round-robin and processing cycles were not taken into > - consideration. > - > - In addition, the output of ``pmd-rxq-show`` was modified to include > - Rx queue utilization of the PMD as a percentage. Prior to this, tracking > of > - stats was not available. > - > -Automatic assignment of Port/Rx Queue to PMD Threads (experimental) > -------------------------------------------------------------------- > +PMD Automatic Load Balance (experimental) > +----------------------------------------- > > Cycle or utilization based allocation of Rx queues to PMDs gives efficient > @@ -233,8 +239,9 @@ To address this automatic load balancing of PMDs can be > set by:: > $ ovs-vsctl set open_vswitch . other_config:pmd-auto-lb="true" > > -If pmd-auto-lb is set to true AND cycle based assignment is enabled then auto > -load balancing of PMDs is enabled provided there are 2 or more non-isolated > -PMDs and at least one of these PMDs is polling more than one RX queue. So, > -following conditions need to be met to have Auto Load balancing enabled: > +If pmd-auto-lb is set to true AND cycle or group based assignment is enabled > +then auto load balancing of PMDs is enabled provided there are 2 or more > +non-isolated PMDs and at least one of these PMDs is polling more than one RX > +queue. So, following conditions need to be met to have Auto Load balancing > +enabled: > > 1. cycle based assignment of RX queues to PMD is enabled. > Should "group" assignment be added here as well? ^ Overall, this change makes the doc much easier to understand for someone new to the topic (such as myself). Apart from the two small comments: Acked-by: Adrian Moreno <[email protected]> Thanks -- Adrián Moreno _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
