On Wed, Jul 6, 2022 at 11:00 PM David Marchand <[email protected]> wrote: > > On Fri, Jul 1, 2022 at 5:58 AM Mike Pattrick <[email protected]> wrote: > > > > From: Flavio Leitner <[email protected]> > > > > Now that there is a segmentation in software as a fall back in > > case a netdev doesn't support TCP segmentation offloading (TSO), > > enable it by default on all possible netdevs. > > > > The existing TSO control is inverted, so that now it will disable > > TSO globally, in case TSO is not desired for some deployment. > > > > Signed-off-by: Flavio Leitner <[email protected]> > > Co-authored-by: Mike Pattrick <[email protected]> > > Signed-off-by: Mike Pattrick <[email protected]> > > --- > > Documentation/topics/userspace-tso.rst | 21 ++-- > > NEWS | 4 + > > lib/netdev-linux.c | 133 ++++++++++++++++++++++++- > > lib/userspace-tso.c | 9 +- > > tests/ofproto-macros.at | 1 + > > vswitchd/vswitch.xml | 17 +--- > > 6 files changed, 154 insertions(+), 31 deletions(-) > > We have one issue with vhost user client ports. > > Consider the case with an OVS running before this series is applied, > with userspace tso disabled (which is the case for existing OVS > installation). > I see that qemu negotiates TSO + ECN feature for a virtio port in the > vhost-user backend on OVS side: > 2022-07-06T19:46:38.225Z|00175|dpdk|INFO|VHOST_CONFIG: negotiated > Virtio features: 0x17020a783 > > Next, I apply the whole series and restart OVS: > 2022-07-06T19:53:29.121Z|00069|netdev_dpdk|INFO|vHost User device > 'vhost1' created in 'client' mode, using client socket > '/var/lib/vhost_sockets/vhost1' > 2022-07-06T19:53:29.122Z|00070|dpdk|INFO|VHOST_CONFIG: new device, > handle is 0, path is /var/lib/vhost_sockets/vhost1 > 2022-07-06T19:53:29.122Z|00001|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_FEATURES > 2022-07-06T19:53:29.122Z|00002|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_PROTOCOL_FEATURES > 2022-07-06T19:53:29.122Z|00003|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_PROTOCOL_FEATURES > 2022-07-06T19:53:29.122Z|00004|dpdk|INFO|VHOST_CONFIG: negotiated > Vhost-user protocol features: 0xcbf > 2022-07-06T19:53:29.122Z|00005|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_QUEUE_NUM > 2022-07-06T19:53:29.122Z|00006|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_SLAVE_REQ_FD > 2022-07-06T19:53:29.122Z|00007|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_OWNER > 2022-07-06T19:53:29.122Z|00008|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_GET_FEATURES > 2022-07-06T19:53:29.122Z|00009|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2022-07-06T19:53:29.123Z|00010|dpdk|INFO|VHOST_CONFIG: vring call idx:0 > file:109 > 2022-07-06T19:53:29.123Z|00011|dpdk|INFO|VHOST_CONFIG: read message > VHOST_USER_SET_VRING_CALL > 2022-07-06T19:53:29.123Z|00012|dpdk|INFO|VHOST_CONFIG: vring call idx:1 > file:110 > 2022-07-06T19:53:29.123Z|00013|dpdk|INFO|VHOST_CONFIG: vhost peer closed > > This happens for every vhost ports I have, in a loop flooding OVS logs. > Looking at qemu logs: > 2022-07-06T19:53:17.581363Z qemu-kvm: Unexpected end-of-file before > all data were read > 2022-07-06T19:53:17.583183Z qemu-kvm: Unexpected end-of-file before > all data were read > 2022-07-06T19:53:17.587613Z qemu-kvm: Unexpected end-of-file before > all data were read > 2022-07-06T19:53:17.588464Z qemu-kvm: Unexpected end-of-file before > all data were read > 2022-07-06T19:53:17.641010Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.641023Z qemu-kvm: vhost VQ 0 ring restore failed: > -1: Invalid argument (22) > 2022-07-06T19:53:17.641035Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.641040Z qemu-kvm: vhost VQ 1 ring restore failed: > -1: Invalid argument (22) > 2022-07-06T19:53:17.645027Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.645039Z qemu-kvm: vhost VQ 0 ring restore failed: > -1: Resource temporarily unavailable (11) > 2022-07-06T19:53:17.645047Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.645052Z qemu-kvm: vhost VQ 1 ring restore failed: > -1: Resource temporarily unavailable (11) > 2022-07-06T19:53:17.648953Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.648964Z qemu-kvm: vhost VQ 0 ring restore failed: > -1: Resource temporarily unavailable (11) > 2022-07-06T19:53:17.648971Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.648976Z qemu-kvm: vhost VQ 1 ring restore failed: > -1: Resource temporarily unavailable (11) > 2022-07-06T19:53:17.652951Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.652962Z qemu-kvm: vhost VQ 0 ring restore failed: > -1: Resource temporarily unavailable (11) > 2022-07-06T19:53:17.652970Z qemu-kvm: Failed to set msg fds. > 2022-07-06T19:53:17.652975Z qemu-kvm: vhost VQ 1 ring restore failed: > -1: Resource temporarily unavailable (11) > vhost lacks feature mask 8192 for backend > 2022-07-06T19:53:29.122990Z qemu-kvm: failed to init vhost_net for queue 0 > vhost lacks feature mask 8192 for backend > 2022-07-06T19:53:29.259739Z qemu-kvm: failed to init vhost_net for queue 0 > vhost lacks feature mask 8192 for backend > > Afaiu, 8192 == 0x2000 which translates to bit 13. > VIRTIO_NET_F_HOST_ECN (13) Device can receive TSO with ECN. > > Even though this feature was wrongly enabled, we end up in a kind of > live loop situation. > The only solution I found is to stop qemu and restart the vm. > > I can see it with a OVS upgrade, and I guess it would be the same for > live migration.
For now, the less ugly is probably to incorrectly announce OVS supports those ECN and UFO features, at least it won't break the upgrade. We may support TSO+ECN later (which would probably mean a dpdk api update to expose such a hw offload request). As of UFO, I don't know what to think. -- David Marchand _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
