[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Changed in: ubuntu-z-systems Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
This bug was fixed in the package qemu - 1:4.2-1ubuntu1 --- qemu (1:4.2-1ubuntu1) focal; urgency=medium * Merge with Debian testing, Among many other things this fixes LP Bugs: LP: #1847806 - add mff* instructions to not break on ppc64 with newer glibc LP: #1812822 - avoid crashes on detaching vhost_net interfaces LP: #1852744 - Crypto Passthrough Interrupt Support LP: #1853316 - CCW IPL Support Remaining changes: - qemu-kvm to systemd unit - d/qemu-kvm-init: script for QEMU KVM preparation modules, ksm, hugepages and architecture specifics - d/qemu-system-common.qemu-kvm.service: systemd unit to call qemu-kvm-init - d/qemu-system-common.install: install helper script - d/qemu-system-common.maintscript: clean old sysv and upstart scripts - d/qemu-system-common.qemu-kvm.default: defaults for /etc/default/qemu-kvm - d/rules: call dh_installinit and dh_installsystemd for qemu-kvm - Distribution specific machine type (LP: 1304107 1621042) - d/p/ubuntu/define-ubuntu-machine-types.patch: define distro machine types - d/qemu-system-x86.NEWS Info on fixed machine type definitions for host-phys-bits=true (LP: 1776189) - add an info about -hpb machine type in debian/qemu-system-x86.NEWS - provide pseries-bionic-2.11-sxxm type as convenience with all meltdown/spectre workarounds enabled by default. (LP: 1761372). - Enable nesting by default - d/p/ubuntu/expose-vmx_qemu64cpu.patch: expose nested kvm by default in qemu64 cpu type. - d/p/ubuntu/enable-svm-by-default.patch: Enable nested svm by default in qemu64 on amd [ No more strictly needed, but required for backward compatibility ] - improved dependencies - Make qemu-system-common depend on qemu-block-extra - Make qemu-utils depend on qemu-block-extra - let qemu-utils recommend sharutils - s390x support - Create qemu-system-s390x package - Enable numa support for s390x - d/rules: build s390-ccw.img with upstream Makefile - d/rules: build s390-netboot.img with upstream Makefile - arch aware kvm wrappers - d/control: update VCS links - tolerate ipxe size change on migrations to >=18.04 (LP: 1713490) - d/p/ubuntu/pre-bionic-256k-ipxe-efi-roms.patch: old machine types reference 256k path - d/control-in: depend on ipxe-qemu-256k-compat-efi-roms to be able to handle incoming migrations from former releases. - d/control-in: Disable capstone disassembler library support (universe) - d/control: disable bluetooth being deprecated - d/not-installed: ignore new interop docs and extra icons for now - d/not-installed: do not install elf2dmp until namespaced - d/qemu-utils.install: install new tools qemu-edid and qemu-keymap - d/control-in: promote qemu-efi/ovmf in Ubuntu (LP 1570617) - d/binfmt-update-in: fix binfmt being called in some containers (LP 1840956) - Dropped changes (in Debian) - qemu-guest-agent: freeze-hook fixes (LP: 1484990) - d/qemu-guest-agent.install: provide /etc/qemu/fsfreeze-hook - d/qemu-guest-agent.dirs: provide /etc/qemu/fsfreeze-hook.d - d/control-in: enable RDMA support in qemu (LP: 1692476) - enable RDMA config option - add libibumad-dev build-dep - d/p/ubuntu/lp-1790901-partial-SLOF-for-s390x-netboot.patch: bring back some SLOF bits stripped in DFSG to be able to build s390x-netboot roms As that hack to build s390-ccw.img rom can't build s390x-netboot.img replace it with a build-indep using the upstream makefiles. This is less prone to miss future changes/fixes that are done to the makefiles - remove /dev/kvm permission handling (moved to systemd 239-6) (#892945) - d/p/debianize-qemu-guest-service.patch: fix path of qemu-ga - d/rules: fix qemu-kvm service for debhelper compat >=12 - Refreshed patches for v4.0 context changes - d/control*: remove sdlabi which was removed upstream - d/control*: enable docs (now explicit) and provide new build-dep python3-sphinx - d/qemu-system-data.install: use new paths for formerly used icons - Merge with Upstream release of qemu 4.0 - d/p/ubuntu/lp-1790901-partial-SLOF-for-s390x-netboot.patch - Dropped changes (Upstream) - d/p/ubuntu/lp-1830243-*: s390x Secure Linux Boot Toleration (LP 1830243) - d/p/ubuntu/lp-1830238-*: s390x hardware cpu model (LP 1830238) - d/p/ubuntu/linux-user-fix-__NR_semtimedop-undeclared-error.patch: fix i386 build error - d/p/ubuntu/lp-1836066-s390-cpumodel-fix-description-for-the-new-vector-fac: fix naming of the new vector facitlity (LP 1836066) - d/p/ubuntu/lp-1836159-fix-with-latest-kernel.patch: fix build issues for missing SIOCGSTAMP definition; final fix is still in discussion upstream (LP: 1836159) - d/p/ubuntu/
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Changed in: ubuntu-z-systems Status: Incomplete => In Progress ** Changed in: qemu (Ubuntu) Status: Incomplete => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
The mentioned commit 6ab79a20af3a7b3bf610ba9aebb446a9f0b05930 is in qemu 4.1 and later. Since we will have qemu 4.2 in 20.04 please retry with that once available. I'll set a bug tak reference in the changelog of the update. ** Tags removed: qemu-19.10 ** Tags added: qemu-20.04 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Changed in: linux (Ubuntu) Status: Incomplete => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Tags added: qemu-19.10 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
This may be fixed by https://git.qemu.org/?p=qemu.git;a=commit;h=6ab79a20af3a7b3bf610ba9aebb446a9f0b05930 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
I couldn't get the crash file in the disco system though I set the apport, but get the qemu_system-s390x.crash under /var/crash in the original test system ubuntu16.04(qemu: 2.11.1). ** Attachment added: "_usr_bin_qemu-system-s390x.0.crash" https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+attachment/5238849/+files/_usr_bin_qemu-system-s390x.0.crash -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Changed in: ubuntu-z-systems Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Changed in: ubuntu-z-systems Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
To some extend this feels a bit like: - https://bugzilla.redhat.com/show_bug.cgi?id=1242383 - https://bugzilla.redhat.com/show_bug.cgi?id=1151306 All those got closed as "invalid host config -> won't fix" so we can't find the fix there. But something happened to let it work fine in my case, we need to find that. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
We now need to find out what the difference is: a) your test case is slightly different and you can trigger it on the same SW levels it works for me, then we need to report that to upstream as those are the very latest versions b) your test is good once you use the more recent SW levels, in that case we need to drill down into your crash and identify the fix (that must be in between qemu 2.11 and 3.1 somewhere) to consider backporting it. c) This would be arch dependent (I tested x86), but we would find that further down the road as you'd report (a) to happen. After all TUNSETVNETLE is for setting big/little endian operations for linux tap/macvtap so it could be s390x only after all. I can't re-deploy the system to use Bionic level components that you use at the moment and that also would only answer (b) but not (a). Therefore to differentiate between the above I'd want to ask you if you could re-run your test on Ubuntu 19.04 with Proposed enabled [1] as the new openvswitch still is in proposed for now. Report back if you can still trigger the issue, but then I'll most likely encourage you to report it upstream and I'd then participate in the discussion there - probably building test PPAs for you as needed. Also report back if this SW stack works for you as well, in that case I'd wonder if you get an actual crash in /var/crash that would help where in the qemu code we would look for TUNSETVNETLE ioctl() failed: File descriptor in bad state. I'd assume net/tap-linux.c in tap_fd_set_vnet_le, but let's be sure. [1]: https://wiki.ubuntu.com/Testing/EnableProposed ** Changed in: qemu (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
I used a recent version of the softwrae stack from Disco - qemu 3.1 - libvirt 5.0 - openvswitch 2.11 With that I had a guest with an OVS device like that: Not too different to your's I'd think. The OVS is trivial just having this interface atm. $ sudo ovs-vsctl show 596674ef-e4cd-471f-9708-9caa5737961c Bridge "ovsbr0" Port "eno49" Interface "eno49" Port "ovsbr0" Interface "ovsbr0" type: internal Port "vnet1" Interface "vnet1" ovs_version: "2.11.0" $ ip link show dev vnet1 93: vnet1: mtu 1500 qdisc mq master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000 link/ether fe:54:00:22:57:fd brd ff:ff:ff:ff:ff:ff I have a started a second guest on the same vswitch (to check traffic from the first guest later on). Now lets delete that port: $ sudo ovs-vsctl del-port ovsbr0 vnet1 $ sudo ovs-vsctl show 596674ef-e4cd-471f-9708-9caa5737961c Bridge "ovsbr0" Port "vnet3" Interface "vnet3" Port "eno49" Interface "eno49" Port "ovsbr0" Interface "ovsbr0" type: internal ovs_version: "2.11.0" Ok the OVS device is gone. Obviously traffic on that interface is dead now, but the guest is still alive and happy. The host dev is still there: $ ip link show dev vnet1 93: vnet1: mtu 1500 qdisc mq state UNKNOWN mode DEFAULT group default qlen 1000 link/ether fe:54:00:22:57:fd brd ff:ff:ff:ff:ff:ff Removing that as well as suggested $ sudo ip link del vnet1 $ ip link show dev vnet1 Device "vnet1" does not exist. The guest still is up and running, while traffic still won't work for obvious reasons. Now lets trigger the hot-unplug of the device. $ virsh detach-device guest-openvswitch-1 net.xml Device detached successfully The guest is still happy and alive. It lost the device (since we detached it) but that is ok and intentional. ** Bug watch added: Red Hat Bugzilla #1242383 https://bugzilla.redhat.com/show_bug.cgi?id=1242383 ** Bug watch added: Red Hat Bugzilla #1151306 https://bugzilla.redhat.com/show_bug.cgi?id=1151306 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Package changed: qemu-kvm (Ubuntu) => qemu (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
I couldn't use the command apport-collect 1812822 root@:~# apport-collect 1812822 ERROR: The python-launchpadlib package is not installed. This functionality is not available. root@:~# apt install python-launchpadlib -y Reading package lists... Done Building dependency tree Reading state information... Done python-launchpadlib is already the newest version (1.10.3-3ubuntu0.1). 0 upgraded, 0 newly installed, 0 to remove and 41 not upgraded. root@:~# dpkg -l python-launchpadlib Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-=-=-=- ii python-launchpadl 1.10.3-3ubunt all Launchpad web services client library There's the guest log under /var/log/libvirt/qemu/kvm.log 2019-01-31 10:52:55.667+: starting up libvirt version: 4.0.0, package: 1ubuntu8.5~cloud0 (Openstack Ubuntu Testing Bot Fri, 07 Sep 2018 04:25:04 +), qemu version: 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.5~cloud0), hostname: zfwcec178.boeblingen.de.ibm.com LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-system-s390x -name guest=kvm,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-36-kvm/master-key.aes -machine s390-ccw-virtio-bionic,accel=kvm,usb=off,dump-guest-core=off -m 512 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 59f71b47-16e4-401d-9d33-30bc1605a84a -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-36-kvm/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -drive file=/root/xenial-minimal.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-ccw,scsi=off,devno=fe.0.,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -chardev pty,id=charconsole0 -device sclpconsole,chardev=charconsole0,id=console0 -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0001 -msg timestamp=on 2019-01-31 10:52:55.667+: Domain id=36 is tainted: high-privileges 2019-01-31T10:52:55.749009Z qemu-system-s390x: -chardev pty,id=charconsole0: char device redirected to /dev/pts/1 (label charconsole0) TUNSETVNETLE ioctl() failed: File descriptor in bad state. 2019-01-31 10:57:29.694+: shutting down, reason=crashed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
As of comment #1, please share/attach the relevant logs. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1812822] Re: Guest crashed when detaching the ovs interface device
** Also affects: qemu-kvm (Ubuntu) Importance: Undecided Status: New ** Also affects: linux (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1812822 Title: Guest crashed when detaching the ovs interface device To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-z-systems/+bug/1812822/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs