Hello Everyone As a follow up to my previous mail and to add more context, would like to share the observed behavior of OVS 2.13 and OVS 2.15 ALB with the traffic generator[1].
[1] https://github.com/yogananth-subramanian/multiqueue_traffic_gen *OVS 2.13PMD load balancing only triggered when processing cycles goes above 95 %OVS 2.15(openvswitch2.15-2.15.0-2.el8fdp.x86_64)DPDK 20.11.0pmd-auto-lb-load-threshold="70"pmd-auto-lb-improvement-threshold="25"PMD load balancing triggered based on pmd-auto-lb-improvement-threshold and auto-lb-threshold values2 PMD threads and 2 rxq per nicsBefore rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb queue-id: 0 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 24 % port: vhu29ba2330-e2 queue-id: 0 (enabled) pmd usage: 73 % port: vhu9b6ca64c-3c queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu29ba2330-e2 queue-id: 1 (enabled) pmd usage: 98 % port: vhu29ba2330-e2 queue-id: 2 (disabled) pmd usage: 0 % port: vhu9b6ca64c-3c queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 45 % port: vhu17dc0ce6-cb queue-id: 2 (disabled) pmd usage: 0 % port: vhu9b6ca64c-3c queue-id: 2 (enabled) pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: vhu29ba2330-e2 queue-id: 1 (enabled) pmd usage: 98 % port: vhu9b6ca64c-3c queue-id: 1 (enabled) pmd usage: 0 % port: vhu9b6ca64c-3c queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 46 % port: vhu17dc0ce6-cb queue-id: 1 (enabled) pmd usage: 0 % port: vhu9b6ca64c-3c queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 46 % port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb queue-id: 2 (disabled) pmd usage: 0 % port: vhu29ba2330-e2 queue-id: 2 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu17dc0ce6-cb queue-id: 0 (enabled) pmd usage: 0 % port: vhu29ba2330-e2 queue-id: 0 (enabled) pmd usage: 98 %Before rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 21 % port: vhu34d496aa-27 queue-id: 2 (enabled) pmd usage: 0 % port: vhu90528211-c5 queue-id: 2 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 22 % port: vhu34d496aa-27 queue-id: 0 (enabled) pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 1 (enabled) pmd usage: 53 % port: vhu9a0e7eca-68 queue-id: 2 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 33 % port: vhu90528211-c5 queue-id: 0 (enabled) pmd usage: 0 % port: vhu90528211-c5 queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 32 % port: vhu34d496aa-27 queue-id: 1 (enabled) pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 0 (enabled) pmd usage: 53 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: vhu34d496aa-27 queue-id: 0 (enabled) pmd usage: 0 % port: vhu90528211-c5 queue-id: 0 (enabled) pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 1 (enabled) pmd usage: 54 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 19 % port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 29 % port: vhu90528211-c5 queue-id: 2 (disabled) pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 2 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: vhu34d496aa-27 queue-id: 1 (enabled) pmd usage: 0 % port: vhu34d496aa-27 queue-id: 2 (enabled) pmd usage: 0 % port: vhu9a0e7eca-68 queue-id: 0 (enabled) pmd usage: 46 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 19 % port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 30 % port: vhu90528211-c5 queue-id: 1 (enabled) pmd usage: 0 %3 PMD threads and 3 rxq per nicsBefore rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhub5eeedde-9e queue-id: 1 (enabled) pmd usage: 0 % port: vhub5eeedde-9e queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 40 % port: vhuabdd63e5-1d queue-id: 0 (enabled) pmd usage: 57 % port: vhucb9e7d30-fa queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 3 (disabled) pmd usage: 0 % port: vhucb9e7d30-fa queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 2 (enabled) pmd usage: 47 % port: vhucb9e7d30-fa queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 69 % port: vhub5eeedde-9e queue-id: 0 (enabled) pmd usage: 0 % port: vhub5eeedde-9e queue-id: 3 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 77 % port: vhuabdd63e5-1d queue-id: 1 (enabled) pmd usage: 20 % port: vhucb9e7d30-fa queue-id: 2 (enabled) pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 73 % port: vhucb9e7d30-fa queue-id: 2 (enabled) pmd usage: 0 % port: vhucb9e7d30-fa queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 0 (enabled) pmd usage: 81 % port: vhub5eeedde-9e queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 1 (enabled) pmd usage: 82 % port: vhuabdd63e5-1d queue-id: 3 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 73 % port: vhub5eeedde-9e queue-id: 3 (disabled) pmd usage: 0 % port: vhucb9e7d30-fa queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 77 % port: vhub5eeedde-9e queue-id: 0 (enabled) pmd usage: 0 % port: vhucb9e7d30-fa queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhuabdd63e5-1d queue-id: 2 (enabled) pmd usage: 91 % port: vhub5eeedde-9e queue-id: 2 (enabled) pmd usage: 0 %Before rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 21 % port: vhu30e312d0-d2 queue-id: 0 (enabled) pmd usage: 41 % port: vhua7d1a810-86 queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu30e312d0-d2 queue-id: 2 (enabled) pmd usage: 59 % port: vhua7d1a810-86 queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu68c33115-4a queue-id: 1 (enabled) pmd usage: 0 % port: vhu68c33115-4a queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 24 % port: vhu68c33115-4a queue-id: 0 (enabled) pmd usage: 0 % port: vhu68c33115-4a queue-id: 3 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 22 % port: vhu30e312d0-d2 queue-id: 1 (enabled) pmd usage: 57 % port: vhua7d1a810-86 queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhu30e312d0-d2 queue-id: 3 (disabled) pmd usage: 0 % port: vhua7d1a810-86 queue-id: 0 (enabled) pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: vhu30e312d0-d2 queue-id: 0 (enabled) pmd usage: 51 % port: vhu68c33115-4a queue-id: 1 (enabled) pmd usage: 0 % port: vhua7d1a810-86 queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 25 % port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu68c33115-4a queue-id: 3 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: vhu30e312d0-d2 queue-id: 2 (enabled) pmd usage: 59 % port: vhua7d1a810-86 queue-id: 2 (enabled) pmd usage: 0 % port: vhua7d1a810-86 queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: vhu30e312d0-d2 queue-id: 1 (enabled) pmd usage: 64 % port: vhu68c33115-4a queue-id: 0 (enabled) pmd usage: 0 % port: vhua7d1a810-86 queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 26 % port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhu68c33115-4a queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 29 % port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu30e312d0-d2 queue-id: 3 (disabled) pmd usage: 0 %5 Physical queues---5 PMDCores---5 vQueues (3 full throttling traffic streams) Before rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 5 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 73 % port: vhu043b3d09-aa queue-id: 4 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 69 % port: vhu043b3d09-aa queue-id: 3 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 1 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 4 (enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 1 (enabled) pmd usage: 80 % port: vhu956b15b2-d8 queue-id: 2 (enabled) pmd usage: 17 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 0 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 5 (disabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 15 % port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 0 (enabled) pmd usage: 41 % port: vhu956b15b2-d8 queue-id: 3 (enabled) pmd usage: 41 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 97 % port: vhu043b3d09-aa queue-id: 2 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 3 (enabled) pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 75 % port: vhu043b3d09-aa queue-id: 1 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 3 (enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 1 (enabled) pmd usage: 86 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 73 % port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 0 (enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 0 (enabled) pmd usage: 87 % port: vhu956b15b2-d8 queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 74 % port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 3 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 1 (enabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 3 (enabled) pmd usage: 70 % port: vhu956b15b2-d8 queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: vhu043b3d09-aa queue-id: 0 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 2 (enabled) pmd usage: 0 % port: vhu0c6e626a-ff queue-id: 5 (disabled) pmd usage: 0 % port: vhu956b15b2-d8 queue-id: 2 (enabled) pmd usage: 98 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 96 % port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhu043b3d09-aa queue-id: 5 (enabled) pmd usage: 0 %Before rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu261f37a0-be queue-id: 5 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 2 (enabled) pmd usage: 47 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 3 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu261f37a0-be queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 0 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 1 (enabled) pmd usage: 60 %pmd thread numa_id 0 core_id 5: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 38 % port: vhu261f37a0-be queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 29 % port: vhu597bfe26-ed queue-id: 1 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 0 (enabled) pmd usage: 51 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhu261f37a0-be queue-id: 3 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 4 (enabled) pmd usage: 30 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 47 % port: vhu597bfe26-ed queue-id: 2 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 44 % port: vhu261f37a0-be queue-id: 2 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 73 % port: vhu261f37a0-be queue-id: 4 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 3 (enabled) pmd usage: 23 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 31 % port: vhu261f37a0-be queue-id: 0 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 27 % port: vhu261f37a0-be queue-id: 4 (enabled) pmd usage: 0 % port: vhu261f37a0-be queue-id: 5 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 3 (enabled) pmd usage: 53 %pmd thread numa_id 0 core_id 4: isolated : false port: vhu261f37a0-be queue-id: 1 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 2 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 2 (enabled) pmd usage: 69 %pmd thread numa_id 0 core_id 5: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 44 % port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: vhu261f37a0-be queue-id: 2 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 1 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 0 (enabled) pmd usage: 65 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 21 % port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: vhu261f37a0-be queue-id: 3 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 0 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 1 (enabled) pmd usage: 31 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhuc4013bd9-87 queue-id: 4 (enabled) pmd usage: 71 % port: vhuc4013bd9-87 queue-id: 5 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 39 % port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhu597bfe26-ed queue-id: 4 (enabled) pmd usage: 0 %7 PMD threads and 7 rxq per nicsBefore rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port queue-id: 6 (enabled) pmd usage: 0 % port: vhuc12aa03a-ae queue-id: 5 (enabled) pmd usage: 0 % port: vhuc12aa03a-ae queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 6 (enabled) pmd usage: 73 % port: vhuc12aa03a-ae queue-id: 4 (enabled) pmd usage: 0 % port: vhuc12aa03a-ae queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 27 % port: vhuc12aa03a-ae queue-id: 3 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 0 (enabled) pmd usage: 71 %pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 5: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 4 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 6: isolated : false port: dpdk-link1-port queue-id: 5 (enabled) pmd usage: 71 % port: vhuc59a8064-13 queue-id: 5 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 7: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhuc12aa03a-ae queue-id: 2 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 1 (enabled) pmd usage: 92 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id: 5 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 47 % port: vhuc59a8064-13 queue-id: 6 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 5 (enabled) pmd usage: 51 %pmd thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 75 % port: vhuc12aa03a-ae queue-id: 1 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 2 (enabled) pmd usage: 24 %pmd thread numa_id 0 core_id 18: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 7 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 4 (enabled) pmd usage: 30 %pmd thread numa_id 0 core_id 19: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 43 % port: vhuc12aa03a-ae queue-id: 0 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 3 (enabled) pmd usage: 55 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: vhuc12aa03a-ae queue-id: 0 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 7 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 1 (enabled) pmd usage: 88 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 80 % port: vhuc12aa03a-ae queue-id: 1 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link1-port queue-id: 6 (enabled) pmd usage: 78 % port: vhuc12aa03a-ae queue-id: 2 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 5 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 97 % port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 5: isolated : false port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 4 (enabled) pmd usage: 37 % port: vhuf8dcddb7-dd queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 6: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 75 % port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 7: isolated : false port: vhuc12aa03a-ae queue-id: 3 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 4 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 0 (enabled) pmd usage: 59 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 2 (enabled) pmd usage: 88 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0 % port: dpdk-link2-port queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 97 % port: dpdk-link2-port queue-id: 5 (enabled) pmd usage: 0 % port: vhuc12aa03a-ae queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id: 5 (enabled) pmd usage: 97 % port: vhuc12aa03a-ae queue-id: 4 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 18: isolated : false port: vhuc12aa03a-ae queue-id: 6 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 1 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 5 (enabled) pmd usage: 52 %pmd thread numa_id 0 core_id 19: isolated : false port: vhuc12aa03a-ae queue-id: 5 (enabled) pmd usage: 0 % port: vhuc59a8064-13 queue-id: 2 (enabled) pmd usage: 0 % port: vhuf8dcddb7-dd queue-id: 3 (enabled) pmd usage: 97 %Before rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 3 (enabled) pmd usage: 36 % port: vhu745c80e7-c3 queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: dpdk-link2-port queue-id: 6 (enabled) pmd usage: 0 % port: vhu745c80e7-c3 queue-id: 5 (enabled) pmd usage: 0 % port: vhu745c80e7-c3 queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 2 (enabled) pmd usage: 43 % port: vhu745c80e7-c3 queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 5: isolated : false port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 6: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 16 % port: vhu650a7bf5-41 queue-id: 5 (enabled) pmd usage: 26 % port: vhu853f185d-2f queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 7: isolated : false port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: dpdk-link1-port queue-id: 6 (enabled) pmd usage: 16 % port: vhu650a7bf5-41 queue-id: 0 (enabled) pmd usage: 32 % port: vhu745c80e7-c3 queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 15 % port: vhu650a7bf5-41 queue-id: 6 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 5 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 26 % port: vhu745c80e7-c3 queue-id: 4 (enabled) pmd usage: 0 % port: vhu745c80e7-c3 queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 5 (enabled) pmd usage: 23 % port: vhu650a7bf5-41 queue-id: 7 (disabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 4 (enabled) pmd usage: 30 % port: vhu853f185d-2f queue-id: 7 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 18: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 15 % port: vhu650a7bf5-41 queue-id: 1 (enabled) pmd usage: 46 % port: vhu745c80e7-c3 queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 19: isolated : false port: dpdk-link2-port queue-id: 5 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 0 (enabled) pmd usage: 0 %After rebalance :pmd thread numa_id 0 core_id 1: isolated : false port: dpdk-link1-port queue-id: 0 (enabled) pmd usage: 32 % port: vhu745c80e7-c3 queue-id: 5 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 2: isolated : false port: vhu650a7bf5-41 queue-id: 2 (enabled) pmd usage: 69 % port: vhu745c80e7-c3 queue-id: 0 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 7 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 3: isolated : false port: dpdk-link1-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 0 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 4: isolated : false port: vhu650a7bf5-41 queue-id: 0 (enabled) pmd usage: 86 % port: vhu745c80e7-c3 queue-id: 4 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 3 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 5: isolated : false port: dpdk-link1-port queue-id: 1 (enabled) pmd usage: 32 % port: dpdk-link2-port queue-id: 2 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 6: isolated : false port: dpdk-link2-port queue-id: 5 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 5 (enabled) pmd usage: 44 % port: vhu745c80e7-c3 queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 7: isolated : false port: dpdk-link1-port queue-id: 2 (enabled) pmd usage: 44 % port: dpdk-link2-port queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 13: isolated : false port: vhu650a7bf5-41 queue-id: 3 (enabled) pmd usage: 24 % port: vhu745c80e7-c3 queue-id: 2 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 5 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 14: isolated : false port: dpdk-link1-port queue-id: 4 (enabled) pmd usage: 26 % port: dpdk-link2-port queue-id: 4 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 15: isolated : false port: vhu650a7bf5-41 queue-id: 1 (enabled) pmd usage: 32 % port: vhu745c80e7-c3 queue-id: 1 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 6 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 16: isolated : false port: dpdk-link1-port queue-id: 6 (enabled) pmd usage: 46 % port: dpdk-link2-port queue-id: 3 (enabled) pmd usage: 0 % port: vhu650a7bf5-41 queue-id: 7 (disabled) pmd usage: 0 %pmd thread numa_id 0 core_id 17: isolated : false port: dpdk-link1-port queue-id: 5 (enabled) pmd usage: 36 % port: vhu745c80e7-c3 queue-id: 6 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 1 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 18: isolated : false port: vhu650a7bf5-41 queue-id: 4 (enabled) pmd usage: 22 % port: vhu745c80e7-c3 queue-id: 3 (enabled) pmd usage: 0 % port: vhu853f185d-2f queue-id: 4 (enabled) pmd usage: 0 %pmd thread numa_id 0 core_id 19: isolated : false port: dpdk-link2-port queue-id: 0 (enabled) pmd usage: 0 % port: dpdk-link2-port queue-id: 6 (enabled) pmd usage: 0 % * On Fri, Mar 19, 2021 at 2:34 AM Yoga Ananth <[email protected]> wrote: > > Hello Kevin, > On Fri, Mar 5, 2021 at 10:37 PM Kevin Traynor <[email protected]> wrote: > >> Hi All, >> >> PMD auto load balance (ALB) is an experimental feature since OVS 2.11. >> >> In short, RxQs are reassigned to PMDs based on load during a reconfig. >> >> However, there might not be a reconfig for a long time so ALB runs >> periodically and checks PMD thread load and the load variance between >> PMD threads. If load is high and it's estimated that it will be improved >> by a certain amount after reassigning RxQs to PMDs, a reassignment is >> triggered. >> >> ALB calls the reassignment code similar to if there was a reconfig or if >> the rebalance command is used. There are some ideas for how to improve >> RxQ to PMD assignments in general, but that's a slightly different topic >> and can be a later optimization if needed. >> >> ALB is documented more here: >> >> https://docs.openvswitch.org/en/latest/topics/dpdk/pmd/?highlight=rebalance#automatic-assignment-of-port-rx-queue-to-pmd-threads-experimental >> >> I would like to progress it so the experimental tag on ALB can be >> removed. There are a few things I think are needed: >> >> - Need to restructure the code so that ALB reassignment dry run uses the >> same code path as the reassignment code it will call >> >> - ALB operation should be understandable through logs and stats >> e.g. Should be able to confirm setting state/parameters in logs and see >> if ALB is taking some action. Should be able to get more debug if needed >> etc. >> >> - Unit tests >> Some patches here: >> >> http://patchwork.ozlabs.org/project/openvswitch/patch/[email protected]/ >> >> - Iron out if the current ALB enable/disable user interface logging >> based partly min resource requirements is right or we want to it to run >> regardless >> i.e. >> https://mail.openvswitch.org/pipermail/ovs-dev/2021-February/380386.html >> >> - Review/Improve operation documentation >> >> - Further testing >> >> Thoughts? >> >> I was able to evaluate ALB with 2.15 release, by using a traffic > generator[1] that is able to deterministically load only the desired rxqs > associated with a PMD, hence I was able to deterministically evaluate ALB > for different values of pmd-auto-lb-load-threshold and n_rxq. > Currently working on enhancing traffic generator[1] to evaluate ALB with > bidirectional traffic. > If you have any scenario or traffic pattern in mind, happy to adapt the > traffic generator and evaluate ALB with those too. > > [1] https://github.com/yogananth-subramanian/multiqueue_traffic_gen > >> thanks, >> Kevin. >> >> _______________________________________________ >> dev mailing list >> [email protected] >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> > > > -- > Kind Regards > yogananth subramanian > -- Kind Regards yogananth subramanian _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
