On Tue, Jul 12, 2022 at 9:54 AM Maxime Coquelin <[email protected]> wrote: > > > > On 7/11/22 23:06, David Marchand wrote: > > On Wed, Jul 6, 2022 at 11:00 PM David Marchand > > <[email protected]> wrote: > >> > >> On Fri, Jul 1, 2022 at 5:58 AM Mike Pattrick <[email protected]> wrote: > >>> > >>> From: Flavio Leitner <[email protected]> > >>> > >>> Now that there is a segmentation in software as a fall back in > >>> case a netdev doesn't support TCP segmentation offloading (TSO), > >>> enable it by default on all possible netdevs. > >>> > >>> The existing TSO control is inverted, so that now it will disable > >>> TSO globally, in case TSO is not desired for some deployment. > >>> > >>> Signed-off-by: Flavio Leitner <[email protected]> > >>> Co-authored-by: Mike Pattrick <[email protected]> > >>> Signed-off-by: Mike Pattrick <[email protected]> > >>> --- > >>> Documentation/topics/userspace-tso.rst | 21 ++-- > >>> NEWS | 4 + > >>> lib/netdev-linux.c | 133 ++++++++++++++++++++++++- > >>> lib/userspace-tso.c | 9 +- > >>> tests/ofproto-macros.at | 1 + > >>> vswitchd/vswitch.xml | 17 +--- > >>> 6 files changed, 154 insertions(+), 31 deletions(-) > >> > >> We have one issue with vhost user client ports. > >> > >> Consider the case with an OVS running before this series is applied, > >> with userspace tso disabled (which is the case for existing OVS > >> installation). > >> I see that qemu negotiates TSO + ECN feature for a virtio port in the > >> vhost-user backend on OVS side: > >> 2022-07-06T19:46:38.225Z|00175|dpdk|INFO|VHOST_CONFIG: negotiated > >> Virtio features: 0x17020a783 > >> > >> Next, I apply the whole series and restart OVS: > >> 2022-07-06T19:53:29.121Z|00069|netdev_dpdk|INFO|vHost User device > >> 'vhost1' created in 'client' mode, using client socket > >> '/var/lib/vhost_sockets/vhost1' > >> 2022-07-06T19:53:29.122Z|00070|dpdk|INFO|VHOST_CONFIG: new device, > >> handle is 0, path is /var/lib/vhost_sockets/vhost1 > >> 2022-07-06T19:53:29.122Z|00001|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_GET_FEATURES > >> 2022-07-06T19:53:29.122Z|00002|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_GET_PROTOCOL_FEATURES > >> 2022-07-06T19:53:29.122Z|00003|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_SET_PROTOCOL_FEATURES > >> 2022-07-06T19:53:29.122Z|00004|dpdk|INFO|VHOST_CONFIG: negotiated > >> Vhost-user protocol features: 0xcbf > >> 2022-07-06T19:53:29.122Z|00005|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_GET_QUEUE_NUM > >> 2022-07-06T19:53:29.122Z|00006|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_SET_SLAVE_REQ_FD > >> 2022-07-06T19:53:29.122Z|00007|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_SET_OWNER > >> 2022-07-06T19:53:29.122Z|00008|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_GET_FEATURES > >> 2022-07-06T19:53:29.122Z|00009|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_SET_VRING_CALL > >> 2022-07-06T19:53:29.123Z|00010|dpdk|INFO|VHOST_CONFIG: vring call idx:0 > >> file:109 > >> 2022-07-06T19:53:29.123Z|00011|dpdk|INFO|VHOST_CONFIG: read message > >> VHOST_USER_SET_VRING_CALL > >> 2022-07-06T19:53:29.123Z|00012|dpdk|INFO|VHOST_CONFIG: vring call idx:1 > >> file:110 > >> 2022-07-06T19:53:29.123Z|00013|dpdk|INFO|VHOST_CONFIG: vhost peer closed > >> > >> This happens for every vhost ports I have, in a loop flooding OVS logs. > >> Looking at qemu logs: > >> 2022-07-06T19:53:17.581363Z qemu-kvm: Unexpected end-of-file before > >> all data were read > >> 2022-07-06T19:53:17.583183Z qemu-kvm: Unexpected end-of-file before > >> all data were read > >> 2022-07-06T19:53:17.587613Z qemu-kvm: Unexpected end-of-file before > >> all data were read > >> 2022-07-06T19:53:17.588464Z qemu-kvm: Unexpected end-of-file before > >> all data were read > >> 2022-07-06T19:53:17.641010Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.641023Z qemu-kvm: vhost VQ 0 ring restore failed: > >> -1: Invalid argument (22) > >> 2022-07-06T19:53:17.641035Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.641040Z qemu-kvm: vhost VQ 1 ring restore failed: > >> -1: Invalid argument (22) > >> 2022-07-06T19:53:17.645027Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.645039Z qemu-kvm: vhost VQ 0 ring restore failed: > >> -1: Resource temporarily unavailable (11) > >> 2022-07-06T19:53:17.645047Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.645052Z qemu-kvm: vhost VQ 1 ring restore failed: > >> -1: Resource temporarily unavailable (11) > >> 2022-07-06T19:53:17.648953Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.648964Z qemu-kvm: vhost VQ 0 ring restore failed: > >> -1: Resource temporarily unavailable (11) > >> 2022-07-06T19:53:17.648971Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.648976Z qemu-kvm: vhost VQ 1 ring restore failed: > >> -1: Resource temporarily unavailable (11) > >> 2022-07-06T19:53:17.652951Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.652962Z qemu-kvm: vhost VQ 0 ring restore failed: > >> -1: Resource temporarily unavailable (11) > >> 2022-07-06T19:53:17.652970Z qemu-kvm: Failed to set msg fds. > >> 2022-07-06T19:53:17.652975Z qemu-kvm: vhost VQ 1 ring restore failed: > >> -1: Resource temporarily unavailable (11) > >> vhost lacks feature mask 8192 for backend > >> 2022-07-06T19:53:29.122990Z qemu-kvm: failed to init vhost_net for queue 0 > >> vhost lacks feature mask 8192 for backend > >> 2022-07-06T19:53:29.259739Z qemu-kvm: failed to init vhost_net for queue 0 > >> vhost lacks feature mask 8192 for backend > >> > >> Afaiu, 8192 == 0x2000 which translates to bit 13. > >> VIRTIO_NET_F_HOST_ECN (13) Device can receive TSO with ECN. > >> > >> Even though this feature was wrongly enabled, we end up in a kind of > >> live loop situation. > >> The only solution I found is to stop qemu and restart the vm. > >> > >> I can see it with a OVS upgrade, and I guess it would be the same for > >> live migration. > > > > For now, the less ugly is probably to incorrectly announce OVS > > supports those ECN and UFO features, at least it won't break the > > upgrade. > > Sadly, I don't think we have over choices than keeping the broken > advertisement of unsupported ECN and UFO. > > > We may support TSO+ECN later (which would probably mean a dpdk api > > update to expose such a hw offload request). > > > > As of UFO, I don't know what to think.
I posted a patch which tries to disable those features and fallback to incorrectly announce them (but disable TSO). Let me know what you think. https://patchwork.ozlabs.org/project/openvswitch/patch/[email protected]/ -- David Marchand _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
