On 7/11/22 23:06, David Marchand wrote:
On Wed, Jul 6, 2022 at 11:00 PM David Marchand
<[email protected]> wrote:
On Fri, Jul 1, 2022 at 5:58 AM Mike Pattrick <[email protected]> wrote:
From: Flavio Leitner <[email protected]>
Now that there is a segmentation in software as a fall back in
case a netdev doesn't support TCP segmentation offloading (TSO),
enable it by default on all possible netdevs.
The existing TSO control is inverted, so that now it will disable
TSO globally, in case TSO is not desired for some deployment.
Signed-off-by: Flavio Leitner <[email protected]>
Co-authored-by: Mike Pattrick <[email protected]>
Signed-off-by: Mike Pattrick <[email protected]>
---
Documentation/topics/userspace-tso.rst | 21 ++--
NEWS | 4 +
lib/netdev-linux.c | 133 ++++++++++++++++++++++++-
lib/userspace-tso.c | 9 +-
tests/ofproto-macros.at | 1 +
vswitchd/vswitch.xml | 17 +---
6 files changed, 154 insertions(+), 31 deletions(-)
We have one issue with vhost user client ports.
Consider the case with an OVS running before this series is applied,
with userspace tso disabled (which is the case for existing OVS
installation).
I see that qemu negotiates TSO + ECN feature for a virtio port in the
vhost-user backend on OVS side:
2022-07-06T19:46:38.225Z|00175|dpdk|INFO|VHOST_CONFIG: negotiated
Virtio features: 0x17020a783
Next, I apply the whole series and restart OVS:
2022-07-06T19:53:29.121Z|00069|netdev_dpdk|INFO|vHost User device
'vhost1' created in 'client' mode, using client socket
'/var/lib/vhost_sockets/vhost1'
2022-07-06T19:53:29.122Z|00070|dpdk|INFO|VHOST_CONFIG: new device,
handle is 0, path is /var/lib/vhost_sockets/vhost1
2022-07-06T19:53:29.122Z|00001|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_GET_FEATURES
2022-07-06T19:53:29.122Z|00002|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_GET_PROTOCOL_FEATURES
2022-07-06T19:53:29.122Z|00003|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_SET_PROTOCOL_FEATURES
2022-07-06T19:53:29.122Z|00004|dpdk|INFO|VHOST_CONFIG: negotiated
Vhost-user protocol features: 0xcbf
2022-07-06T19:53:29.122Z|00005|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_GET_QUEUE_NUM
2022-07-06T19:53:29.122Z|00006|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_SET_SLAVE_REQ_FD
2022-07-06T19:53:29.122Z|00007|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_SET_OWNER
2022-07-06T19:53:29.122Z|00008|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_GET_FEATURES
2022-07-06T19:53:29.122Z|00009|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_SET_VRING_CALL
2022-07-06T19:53:29.123Z|00010|dpdk|INFO|VHOST_CONFIG: vring call idx:0 file:109
2022-07-06T19:53:29.123Z|00011|dpdk|INFO|VHOST_CONFIG: read message
VHOST_USER_SET_VRING_CALL
2022-07-06T19:53:29.123Z|00012|dpdk|INFO|VHOST_CONFIG: vring call idx:1 file:110
2022-07-06T19:53:29.123Z|00013|dpdk|INFO|VHOST_CONFIG: vhost peer closed
This happens for every vhost ports I have, in a loop flooding OVS logs.
Looking at qemu logs:
2022-07-06T19:53:17.581363Z qemu-kvm: Unexpected end-of-file before
all data were read
2022-07-06T19:53:17.583183Z qemu-kvm: Unexpected end-of-file before
all data were read
2022-07-06T19:53:17.587613Z qemu-kvm: Unexpected end-of-file before
all data were read
2022-07-06T19:53:17.588464Z qemu-kvm: Unexpected end-of-file before
all data were read
2022-07-06T19:53:17.641010Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.641023Z qemu-kvm: vhost VQ 0 ring restore failed:
-1: Invalid argument (22)
2022-07-06T19:53:17.641035Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.641040Z qemu-kvm: vhost VQ 1 ring restore failed:
-1: Invalid argument (22)
2022-07-06T19:53:17.645027Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.645039Z qemu-kvm: vhost VQ 0 ring restore failed:
-1: Resource temporarily unavailable (11)
2022-07-06T19:53:17.645047Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.645052Z qemu-kvm: vhost VQ 1 ring restore failed:
-1: Resource temporarily unavailable (11)
2022-07-06T19:53:17.648953Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.648964Z qemu-kvm: vhost VQ 0 ring restore failed:
-1: Resource temporarily unavailable (11)
2022-07-06T19:53:17.648971Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.648976Z qemu-kvm: vhost VQ 1 ring restore failed:
-1: Resource temporarily unavailable (11)
2022-07-06T19:53:17.652951Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.652962Z qemu-kvm: vhost VQ 0 ring restore failed:
-1: Resource temporarily unavailable (11)
2022-07-06T19:53:17.652970Z qemu-kvm: Failed to set msg fds.
2022-07-06T19:53:17.652975Z qemu-kvm: vhost VQ 1 ring restore failed:
-1: Resource temporarily unavailable (11)
vhost lacks feature mask 8192 for backend
2022-07-06T19:53:29.122990Z qemu-kvm: failed to init vhost_net for queue 0
vhost lacks feature mask 8192 for backend
2022-07-06T19:53:29.259739Z qemu-kvm: failed to init vhost_net for queue 0
vhost lacks feature mask 8192 for backend
Afaiu, 8192 == 0x2000 which translates to bit 13.
VIRTIO_NET_F_HOST_ECN (13) Device can receive TSO with ECN.
Even though this feature was wrongly enabled, we end up in a kind of
live loop situation.
The only solution I found is to stop qemu and restart the vm.
I can see it with a OVS upgrade, and I guess it would be the same for
live migration.
For now, the less ugly is probably to incorrectly announce OVS
supports those ECN and UFO features, at least it won't break the
upgrade.
Sadly, I don't think we have over choices than keeping the broken
advertisement of unsupported ECN and UFO.
We may support TSO+ECN later (which would probably mean a dpdk api
update to expose such a hw offload request).
As of UFO, I don't know what to think.
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
