On 04.04.2019 22:49, David Marchand wrote: > We currently poll all available queues based on the max queue count > exchanged with the vhost peer and rely on the vhost library in DPDK to > check the vring status beneath. > This can lead to some overhead when we have a lot of unused queues. > This situation happens easily when you provision more queues than you > need for your virtual machines. > > To enhance the situation, we can inform the rxq scheduling algorithm to > skip unused queues. All we need is to catch the per rxq notifications > and trigger a rebalance by asking for a port reconfigure (without > changing the port configuration itself). > > A consequence of this is that before a device is connected to the vhost > port, no rxq is polled at all. > > Signed-off-by: David Marchand <[email protected]> > --- > > We tried to lower the number of rebalances but we don't have a > satisfying solution at the moment, so this patch rebalances on each > update.
Hi. Triggering the reconfiguration on each vring state change is a bad thing. This could be abused by the guest to break the host networking by infinite disabling/enabling queues. Each reconfiguration leads to removing ports from the PMD port caches and their reloads. On rescheduling all the ports could be moved to a different PMD threads resulting in EMC/SMC/dpcls invalidation and subsequent upcalls/packet reorderings. Same issues was discussed previously while looking at possibility of vhost-pmd integration (with some test results): https://mail.openvswitch.org/pipermail/ovs-dev/2016-August/320430.html One more reference: 7f5f2bd0ce43 ("netdev-dpdk: Avoid reconfiguration on reconnection of same vhost device.") One possible way to improve your case where guest never uses all the queues allocated in QEMU is to requesting the reconfiguration only if we're receiving "vring enabled" event for the queue that has higher number than ever seen on this device. i.e.: if (is_rx && enable && qid >= dev->requested_n_rxq) { dev->requested_n_rxq = qid + 1; netdev_request_reconfigure(&dev->up); } At least this will limit number of reconfigurations to the number of queues. But I'm still not completely sure if this change is good enough. (I believe that we discussed this solution somewhere on a list, but can't find right now). Anyway, do you have some numbers of how much time PMD thread spends on polling disabled queues? What the performance improvement you're able to achieve by avoiding that? Best regards, Ilya Maximets. _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
