On Fri, Dec 8, 2017 at 6:40 AM, Michal Kubecek <mkube...@suse.cz> wrote:
> On Fri, Dec 08, 2017 at 11:31:50AM +0100, Andreas Hartmann wrote:
>> On 12/08/2017 at 09:47 AM Michal Kubecek wrote:
>> > On Fri, Dec 08, 2017 at 08:21:16AM +0100, Andreas Hartmann wrote:
>> >>
>> >> All my VMs are using virtio_net. BTW: I couldn't see the problems
>> >> (sometimes, the VM couldn't be stopped at all) if all my VMs are using
>> >> e1000 as interface instead.
>> >>
>> >> This finding now matches pretty much the responsible UDP-package which
>> >> caused the stall. I already mentioned it here [2].
>> >>
>> >> To prove it, I reverted from the patch series "[PATCH v2 RFC 0/13]
>> >> Remove UDP Fragmentation Offload support" [3]
>> >>
>> >> 11/13 [v2,RFC,11/13] net: Remove all references to SKB_GSO_UDP. [4]
>> >> 12/13 [v2,RFC,12/13] inet: Remove software UFO fragmenting code. [5]
>> >> 13/13 [v2,RFC,13/13] net: Kill NETIF_F_UFO and SKB_GSO_UDP. [6]
>> >>
>> >> and applied it to Linux 4.14.4. It compiled fine and is running fine.
>> >> The vnet doesn't die anymore. Yet, I can't say if the qemu stop hangs
>> >> are gone, too.
>> >>
>> >> Obviously, there is something broken with the new UDP handling. Could
>> >> you please analyze this problem? I could test some more patches ... .
>> >
>> > Any chance your VMs were live migrated from pre-4.14 host kernel?
>>
>> No - the VMs are not live migrated. They are always running on the same
>> host - either with kernel < 4.14 or with kernel 4.14.x.
>
> This is disturbing... unless I'm mistaken, it shouldn't be possible to
> have UFO enabled on a virtio device in a VM booted on a host with 4.14
> kernel.
Indeed. When working on that revert patch I verified that UFO in
the guest virtio_net was off before the revert patch, on after.
Qemu should check host support with tap_probe_has_ufo
before advertising support to the guest. Indeed, this is exactly
what broke live migration in virtio_net_load_device at
if (qemu_get_byte(f) && !peer_has_ufo(n)) {
error_report("virtio-net: saved image requires TUN_F_UFO support");
return -1;
}
Which follows
peer_has_ufo
qemu_has_ufo
tap_has_ufo
s->has_ufo
where s->has_ufo was set by tap_probe_has_ufo in net_tap_fd_init.
Now, checking my qemu git branch, I ran pretty old 2.7.0-rc3. But this
codepath does not seem to have changed between then and 2.10.1.
I cherry-picked the revert onto 4.14.3. It did not apply cleanly, but the
fix-up wasn't too hard. Compiled and booted, but untested otherwise. At
https://github.com/wdebruij/linux/commits/v4.14.3-aargh-ufo
>
>> > If this is the case, you should try commit 0c19f846d582 ("net:
>> > accept UFO datagrams from tuntap and packet").
>>
>> It doesn't apply to 4.14.4
>>
>> > Or disabling UFO in the guest should
>> > work around the issue.
>>
>> ethtool -K ethX ufo off for each device / bridge in VM.
>>
>> Yes, this seems to work. I'll wait and see if the non stoppable
>> qemu-problem on shutdown will remain.
>>
>> When will there be a fix for 4.14? It is clearly a regression. Is it
>> possible / a good idea to just remove the complete patch series "Remove
>> UDP Fragmentation Offload support"?
>
> I cannot give an exact date but the patch is queued for stable
> (see http://patchwork.ozlabs.org/bundle/davem/stable/?state=* ) so that
> it should land in stable-4.14 in near future (weeks at most).
>
> Michal Kubecek
>
