[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Cannot reproduce this on any recent MAAS. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: maas Status: Incomplete => Invalid -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Setting this to incomplete for maas as weel, since we couldn't reproduce the issue with maas 2.7. Please reopen if it happens again, with steps on how to reproduce ** Changed in: maas Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
As this is subscribed to field-high and field-medium I am unsubscribing field-high. There has been no update for 9 months to this bug. If folks are still seeing this then additional logs and help reproducing would be much appreciated. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
It was already clear on the bug that for qemu I can't do much more as-is. So I had a discussion with the Maas Team today. To core issue for me stays that it seems it is unreproducible outside of Maas. We agree that: - they try to recreate it in a MAAS - Once they can we sync and find day(s) to sit together working on this - The mission then is to eliminate maas components and replace them with "usual" ipxe setups one by one and thereby either identify the maas component to analyze OR get finally a reproducible case for qemu development - There is a fallback if until the next sprint in early 2020 the case is still not reproducible then MAAS will also consider it incomplete/unreproducible -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Thanks Kellen, I took your kernel/initrd and IPXE booted it fine via http from qemu's IPXE on a 2G guest. In the past we tried more (use squash, use tftp instead of http, use the same parmlines as maas, ...), but it all comes down to the same thing every time - only occurs inside Maas. Therefore we are again back at my request to the Maas team to help me with this issue as it continues to not show up without Maas being involved. We had many discussions about this in the past, but I realized that with Mike many of these thoughts might have left. In a perfect world a Maas team member would set up a system using the Artifacts that @Krenshaw provided to reliable recreate the issue (with Maas) on an internal. Then we'd start a full afternoon debugging together (hangout with both of us on the same system) to one by one replace components of the Maas setup in order to finally get this to be debuggable without Maas (or have an idea where/what in side Maas things might break). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "squashfs" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272424/+files/squashfs -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "boot-initrd" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272416/+files/boot-initrd -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "host1_dpkg.txt" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272418/+files/host1_dpkg.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "host2_dpkg.txt" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272419/+files/host2_dpkg.txt -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "vm2.xml" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272422/+files/vm2.xml -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "vm1_console.log" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272421/+files/vm1_console.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "vm2_console.log" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272423/+files/vm2_console.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "vm1.xml" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272420/+files/vm1.xml -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Attachment added: "boot-kernel" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5272417/+files/boot-kernel -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Ran into this today, attaching the information requested in #51. Unable to commission/install 18.04 with "No minimum kernel" MAAS version: maas 2.5.3-7533-g65952b418-0ubuntu1~18.04.1 all Guests kernel paniced on two hosts (host1 and host2), dpkg -l attached for each host. Console output from each failed guest (vm*_console.log) Libvirt xml from each failed guest (vm*.xml) attaching squashfs, boot-kernel, and boot-initrd used by MAAS for the failed boots. Running host kernels: host1: Linux host1 5.0.0-17-generic #18~18.04.1-Ubuntu SMP Wed Jun 5 12:43:23 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux host2: Linux host2 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux regiond log of latest image download: 2019-06-22 23:14:06 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/streams/v1/maas:v2:download.json HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:09 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/ga-18.04-lowlatency/bionic/20190621/boot-initrd HTTP/1 .1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:09 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/ga-18.04-lowlatency/bionic/20190621/boot-kernel HTTP/1 .1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:19 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/ga-18.04-lowlatency/bionic/20190621/squashfs HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:22 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/ga-18.04/bionic/20190621/boot-initrd HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:23 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/ga-18.04/bionic/20190621/boot-kernel HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:27 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04-edge/bionic/20190621/boot-initrd HTTP/1.1 -- > 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:27 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04-edge/bionic/20190621/boot-kernel HTTP/1.1 -- > 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:30 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04-lowlatency-edge/bionic/20190621/boot-initrd HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:31 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04-lowlatency-edge/bionic/20190621/boot-kernel HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:31 regiond: [info] 127.0.0.1 GET /MAAS/rpc/ HTTP/1.1 --> 200 OK (referrer: -; agent: provisioningserver.rpc.clusterservi ce.ClusterClientService) 2019-06-22 23:14:35 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04-lowlatency/bionic/20190621/boot-initrd HTTP/ 1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:35 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04-lowlatency/bionic/20190621/boot-kernel HTTP/ 1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:39 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04/bionic/20190621/boot-initrd HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) 2019-06-22 23:14:39 regiond: [info] 127.0.0.1 GET /MAAS/images-stream/ubuntu/amd64/hwe-18.04/bionic/20190621/boot-kernel HTTP/1.1 --> 200 OK (referrer: -; agent: python-simplestreams/0.1) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added)
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Yeah, it seems to only occur with the TFTP setup that maas creates - and in all former cases it resolved a few days later - most likely by size of images changing. That is the reason we need the files from an affected case. @Reto - please see in the bug description above, there is a section "If you are affected, please attach to the bug" P.S. I'm still waiting on help by the Maas Team to better explain how to get these artifacts, but get what you can and attach it - if it works finally to reproduce it outside of Maas I can work on it, otherwise I'm still bound on waiting for the Maas Team to debug it. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I can also confirm the reported issue. I've tried with Juju and plain MAAS to create a VM with just 2048MB and it fails in both cases. Creating, registering and commissioning the machine works, but it cannot be deployed. System information: 18.04.2 LTS up-to-date MAAS version: 2.5.3 (7533-g65952b418-0ubuntu1~18.04.1) Kernel version: 5.1.3 (Ukuu) I don't see what information we have to provide (last comment from @paelzer)? I've added a diff of the VM configuration ($ virsh dumpxml vm), but I don't see any unexpected difference. The VM with 4096MB memory specification boots and can be deployed while the 2048MB VM cannot. ** Patch added: "Diff of virsh vm configuration" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5270761/+files/vm.diff -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Added to the description what we need from anyone affected. @Maas Team see former comment for the requested guiding of users how to get this data. The Maas task is back from incomplete to confirmed for providing these howto's ** Changed in: maas Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
@sfeole Thanks for the Dup hit that as well now - as all others I ask you to please help to catch the data needed to finally recreate and debug this. >From IRC: [07:00] sfeole: please attach your guest XML, and the used initrd and kernel to the bug [07:02] best would be also the pxeconfig that is provided @MAAS Team So far this only occurred with MAAS and all other tries to recreate worked. I wonder if either TFTP or the PXE loader used could be the trigger of the bug. Therefore (from IRC as well): [07:02] roaksoax: on this bug one of the components we still have to verify if it is related is the way you provide the TFP [07:02] could you outline on the bug how users once they hit the bug could extract that and attach it to the bug [07:03] that way we can hopefully finally recreate the case I tried so back in comment #31, but that was only trying to recreate your setup. For PXEconfig (I was guessing) and TFTP (I used the normal tftpd package) we could still get more similar to your setup. I'll add a section to the description of the bug what users are supposed to attach, it would be great if you could help to outline how users can get those. @MAAS Team [07:03] and IIRC you don't use tftpd or such but something maas internal for tftpd is that right - could you outline how one could use/setup this so that we can compare tftpd vs maas-tftpd as well? That is an extra question to that above, if you could help here as well that would be great. ** Description changed: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. + + + So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. + + If you are affected, please attach to the bug + - the kernel used to boot the guest + - the initrd used to boot the guest + - libvirt guest XML used to define the guest + - PXE config provided to the guest + - the cloud image used to start the guest (That will likely not fit as bug + attachment, consider storing it somewhere and contact us) + + TODO: The Maas Team will outline how to get all these artifacts. + TODO: add reference to the comment outlining this (once added) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Confirmed Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. So far this only occurred in MAAS environments and often went away with e.g. the next daily images. Therefore to reproduce and finally debug/fix this issue we'd need data from anyone being affected. This list will grow as we identify more that is needed. If you are affected, please attach to the bug - the kernel used to boot the guest - the initrd used to boot the guest - libvirt guest XML used to define the guest - PXE config provided to the guest - the cloud image used to start the guest (That will likely not fit as bug attachment, consider storing it somewhere and contact us) TODO: The Maas Team will outline how to get all these artifacts. TODO: add reference to the comment outlining this (once added) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
After a longer session on IRC this is now also for Dmitrii un-reproducible. Lets summarize the current status for the next oen coming by: Current ideas still are: a) at 2G the placement of kernel/initrd is too close (as placed by pxelinux), then when the kernel unpacks it overwrites the initrd b) maas pxe backend might add some bits in transmission that break it => both of the above might depend on there kernel/initrd size (which would be why some kernels/daily images are affected) We would be back needing a case that reliable triggers outside of MAAS. If affected read through the former comments 0. safe and attach here your kernel/initrd/xml to help reproducing it in some debuggable way 0. state the Host OS version and components libvirt/qemu/maas used 1. check if it does reproduce reliably for you (retry a few times) 2. try to do the same without PXE booting (so far always resolved the case) (see above for details) 3a. if you get it recreated without PXE report here how you did so 3b. if only with MAAS then try to modify sizes of the roms (comment #62) ... (per discussion) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Tried using direct kernel boot with QEMU and couldn't reproduce it:
sha256sum
/mnt/libvirt-images/boot-resources-20190419-115735/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-initrd
69ca457a119fe309d315972ca2756a17bd9bc55bc98f2bea5542566a7f41b08f
/mnt/libvirt-images/boot-resources-20190419-115735/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-initrd
30463 qemu-system-x86_64 -enable-kvm -name guest=maas-vhost6,debug-threads=on
-S -object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-224-maas-vhost6/master-key.aes
-machine pc-q35-2.11,accel=kvm,usb=off,vmport=off,dump-guest-core=off -cpu
Haswell-noTSX-IBRS,vme=on,ss=on,vmx=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc_adjust=on,ssbd=on,xsaveopt=on,pdpe1gb=on,abm=on
-m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid
399eae83-f059-4ac0-9609-5bb548f5a90a -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-224-maas-vhost6/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew
-global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global
ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot strict=on -kernel
/mnt/libvirt-images/boot-resources-20190419-115735/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-kernel
-initrd
/mnt/libvirt-images/boot-resources-20190419-115735/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-initrd
-append
BOOT_IMAGE=http://10.10.101.2:5248/images/ubuntu/amd64/ga-18.04/bionic/daily/boot-kernel
nomodeset ro
root=squash:http://10.10.101.2:5248/images/ubuntu/amd64/ga-18.04/bionic/daily/squashfs
ip=maas-vhost6:BOOTIF ip6=off overlayroot=tmpfs
overlayroot_cfgdisk=disabled cc:{'datasource_list': ['MAAS']}end_cc
cloud-config-url=http://10.10.101.2:5248/MAAS/metadata/latest/by-id/xeqrrw/?op=get_preseed
apparmor=0 log_host=10.10.101.2 log_port=5247 --- console=ttyS0,115200
initrd=http://10.10.101.2:5248/images/ubuntu/amd64/ga-18.04/bionic/daily/boot-initrd
BOOTIF=01-52-54-00-3f-ae-46 -device
pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2
-device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1
-device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2
-device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3
-device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4
-device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5
-device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6
-device i82801b11-bridge,id=pci.8,bus=pcie.0,addr=0x1e -device
pci-bridge,chassis_nr=9,id=pci.9,bus=pci.8,addr=0x0 -device
ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2
-device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x0 -device
virtio-scsi-pci,id=scsi1,bus=pci.4,addr=0x0 -device
ahci,id=sata1,bus=pci.9,addr=0x1 -device ahci,id=sata2,bus=pci.9,addr=0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -drive
file=/mnt/libvirt-images/vhost6.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2
-drive
file=/mnt/libvirt-images/maas-vhost6.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-1,serial=disk0
-device
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1
-drive
file=/mnt/libvirt-images/masa-vhost6-scsi21.qcow2,format=qcow2,if=none,id=drive-scsi1-0-0-2,serial=disk1
-device
scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi1-0-0-2,id=scsi1-0-0-2
-drive
file=/mnt/libvirt-images/maas-vhost6-virtio0.qcow2,format=qcow2,if=none,id=drive-virtio-disk0
-device
virtio-blk-pci,scsi=off,bus=pci.6,addr=0x0,drive=drive-virtio-disk0,id=virtio-disk0
-drive
file=/mnt/libvirt-images/maas-vhost6-virtio1.qcow2,format=qcow2,if=none,id=drive-virtio-disk1,serial=disk3
-device
virtio-blk-pci,scsi=off,bus=pci.7,addr=0x0,drive=drive-virtio-disk1,id=virtio-disk1
-drive
file=/mnt/libvirt-images/maas-vhost6-sata0.qcow2,format=qcow2,if=none,id=drive-sata0-0-3,serial=disk4
-device ide-hd,bus=ide.3,drive=drive-sata0-0-3,id=sat
virsh dumpxml exempt:
hvm
/mnt/libvirt-images/boot-resources-20190419-115735/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-kernel
/mnt/libvirt-images/boot-resources-20190419-115735/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-initrd
BOOT_IMAGE=http://10.10.101.2:5248/images/ubuntu/amd64/ga-18.04/bionic/daily/boot-kernel
nomodeset ro
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
The bionic/ga files from #63 need to be placed into both dirs: 1) /var/lib/maas/boot-resources/current/ubuntu/amd64/ga-18.04/bionic/daily 2) /var/lib/maas/boot-resources/current/ubuntu/amd64/generic/bionic/daily The sha256 of the *initrd file* that triggers the issue is 69ca457a119fe309d315972ca2756a17bd9bc55bc98f2bea5542566a7f41b08f Be careful with daily image auto-updates because a recent update have overridden the files for me and the issue was no longer possible to reproduce. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
tar -czvf boot-resources-20190419-115735-amd64-generic-bionic.tar.gz boot-resources/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/ boot-resources/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/ boot-resources/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/ boot-resources/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/squashfs boot-resources/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-kernel boot-resources/snapshot-20190419-115735/ubuntu/amd64/generic/bionic/daily/boot-initrd ** Attachment added: "boot-resources-20190419-115735-amd64-generic-bionic.tar.gz" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5258219/+files/boot-resources-20190419-115735-amd64-generic-bionic.tar.gz -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Fro experimenting with sizes things can easily be padded with zeros: $ truncate -s 8000 bug-1797581-bad-initrd-extended $ truncate -s 900 bug-1797581-xenial-base-extended Boots just as well afterwards. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Ran with: - the cloud-image of Xenial of 20190419 [1] - XML with a memory definition of: 2097152 2097152 - kernel are from the host /var/lib/libvirt/images/bug-1797581-bionic-base /var/lib/libvirt/images/bug-1797581-bad-initrd - The attached broken initrd of comment #56 - Otherwise it is a uvtool kvm guest as it is created on Bionic - Host is Bionic - different kernels: - xenial - xenial hwe - bionic - bionic-hwe Actually after above yielded no useful results I decided to even exclude libvirt from the equation for now (we don't even need a disk, just need to know if the kernel can extract the initrd): $ sudo qemu-system-x86_64 -enable-kvm -m 2048M -nographic -serial mon:stdio -append 'console=ttyS0' -kernel bug-1797581-bionic-hwe -initrd bug-1797581-bad-initrd All kernels loaded the attached broken initrd just fine. :-/ no repro yet @Dmitrii: - in the past we were unsure if this padding would be mangled by bootloaders, tfp or anything like it. Do you reproduce that locally or through maas still? - The ramdisk that you attached is bionic it seems - is that correct? - Please test the same without a disk, it should not matter and saves a lot of space when attaching files here. - Would you mind attaching the full set of a broken case (kernel+initrd+xml) - report the full commandline that the broken boot called qemu with? - If you run this in maas or libvirt still, would you mind to step by step check if the same occurs a) without maas (only in libvirt) b) without libvirt (probably check which args e.g. -m it exactly passes to qemu) [1]: https://cloud-images.ubuntu.com/xenial/20190419/xenial-server- cloudimg-amd64-disk1.img -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
@Dmitrii - we were at the "junk in ..." in comment #21 already. The "broken padding" is interesting but might be the same issue with a different message as the newer kernel understands slightly more about it. That it does only occur with Xenial-HWE but on the same initrd not with the base Xenial kernel is interesting as well. I agree that this is worth the kernel task on this bug (it was the same initrd in this test right)?. The question is if the initrd really is broken (unlikely since it works with other mem sizes) or that it is broken by qemu/seabios/... when passed to the guest (likely). Having such a *full set* of affected files would be great. What we'd need is that you'd copy the kernel+initrd+image+xml used somewhere that we can reach and/or attach it here if it fits - this was asked of all reporters in comment #51 already. I have seen you have attached the initrd of your case at least in comment #56 - thanks! I'll taken another look at the issue with that initrd that you attached, will let you know if I need more. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: qemu (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Tested bionic-hwe - the issue does not occur with 2048 MiB. The closest issue filed upstream I found is this: https://bugzilla.kernel.org/show_bug.cgi?id=199845 ** Bug watch added: Linux Kernel Bug Tracker #199845 https://bugzilla.kernel.org/show_bug.cgi?id=199845 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Found something interesting. Bionic + 2048 MiB of RAM (bad): [1.520243] Unpacking initramfs... [ 14.712821] Initramfs unpacking failed: broken padding [ 14.723088] Freeing initrd memory: 56636K Bionic + 2049 MiB of RAM (good): [0.752624] Unpacking initramfs... [5.572407] Freeing initrd memory: 56636K Xenial HWE + 2048 MiB of RAM (bad): [5.598647] Unpacking initramfs... [ 84.494431] Initramfs unpacking failed: junk in compressed archive [ 84.503565] Freeing initrd memory: 54564K -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I cannot reproduce the same with a xenial (GA kernel) image with 2048 MiB of RAM allocated to a VM. So it seems to me that this is a kernel issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
The kernel code path mentioned in #55 is only executed if there is no
"early userspace init" - in other words, if there is no /init on initrd:
https://git.launchpad.net/~ubuntu-
kernel/ubuntu/+source/linux/+git/bionic/tree/init/main.c?id=Ubuntu-4.15.0-47.50#n1087
/*
* check if there is an early userspace init. If yes, let it do all
* the work
*/
if (!ramdisk_execute_command)
ramdisk_execute_command = "/init"; // <--- if there's no
command specified, default to /init
if (sys_access((const char __user *) ramdisk_execute_command, 0) != 0)
{ // <-- check if /init is present by doing the access syscall
ramdisk_execute_command = NULL;
prepare_namespace(); // <-- call prepare_namespace as
mentioned in #55 which results in an error
}
However, I can see that the initrd used in my case contains the init script (so
sys_access should be successful):
initrd=http://10.10.101.2:5248/images/ubuntu/amd64/generic/bionic/daily
/boot-initrd
lsinitramfs
/var/lib/maas/boot-resources/current/ubuntu/amd64/generic/bionic/daily/boot-initrd
| grep -P ^init$
init
If I increase the memory allocation from 2048 to 2049 MiB the machine
starts to boot just fine.
Unsuccessful boot log (2048 MiB): https://paste.ubuntu.com/p/CY834ZGnvS/
Successful boot log (2049 MiB): https://paste.ubuntu.com/p/3fYfFXW8qR/
Attached boot-initrd.
** Attachment added: "boot-initrd"
https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5257069/+files/boot-initrd
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1797581
Title:
Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to
kernel panic
Status in MAAS:
Incomplete
Status in linux package in Ubuntu:
Confirmed
Status in qemu package in Ubuntu:
Confirmed
Bug description:
Using latest MAAS master, I'm unable to compose a VM over the UI
successfully when composed with 2048 MB of RAM. By that I mean that
the VM is created, but it fails with a kernel panic.
To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
echo -n
'BOOT_IMAGE=http://10.10.101.2:5248/images/ubuntu/amd64/generic/bionic/daily/boot-kernel
nomodeset ro
root=squash:http://10.10.101.2:5248/images/ubuntu/amd64/generic/bionic/daily/squashfs
ip=maas-vhost6:BOOTIF ip6=off overlayroot=tmpfs
overlayroot_cfgdisk=disabled cc:{'datasource_list': ['MAAS']}end_cc
cloud-config-url=http://10.10.101.2:5248/MAAS/metadata/latest/by-id/xeqrrw/?op=get_preseed
apparmor=0 log_host=10.10.101.2 log_port=5247 --- console=ttyS0,115200
initrd=http://10.10.101.2:5248/images/ubuntu/amd64/generic/bionic/daily/boot-initrd
BOOTIF=01-52-54-00-3f-ae-46' | wc -c
576
arch/x86/include/asm/setup.h:
#define COMMAND_LINE_SIZE 2048
#define PARAM_SIZE 4096 /* sizeof(struct boot_params) */
Doesn't look like we are any close to the kernel limits on parameters.
However, the root argument as printed in the panic message looks like a
64-byte string (last byte for null termination):
echo -n 'squash:http://10.10.101.2:5248/images/ubuntu/amd64/generic/bion' | wc
-c
63
It looks like this is coming from the following code (strlcpy into a 64-byte
array):
https://git.launchpad.net/~ubuntu-
kernel/ubuntu/+source/linux/+git/bionic/tree/init/do_mounts.c?id=Ubuntu-4.15.0-47.50#n42
static char __initdata saved_root_name[64];
static int __init root_dev_setup(char *line)
{
strlcpy(saved_root_name, line, sizeof(saved_root_name));
return 1;
}
__setup("root=", root_dev_setup);
And the overall code-path (judging by the md auto-detection log messages):
https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/init/do_mounts.c?id=Ubuntu-4.15.0-47.50#n545
prepare_namespace ->
if (saved_root_name[0]) {
root_device_name = saved_root_name; // < (!) usage of a
cut-down root param
if (!strncmp(root_device_name, "mtd", 3) ||
!strncmp(root_device_name, "ubi", 3)) {
mount_block_root(root_device_name, root_mountflags);
Does not look like we are hitting either EACCES or EINVAL and so we fall
through to panic():
https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux/+git/bionic/tree/init/do_mounts.c?id=Ubuntu-4.15.0-47.50#n381
void __init mount_block_root(char *name, int flags)
// ...
for (p = fs_names; *p; p += strlen(p)+1) {
int err = do_mount_root(name, p, flags, root_mount_data);
switch (err) {
case 0:
goto out;
case -EACCES:
case -EINVAL:
continue;
}
/*
* Allow the user to distinguish between failed sys_open
* and bad superblock on root device.
* and give them a list of the available devices
*/
#ifdef CONFIG_BLOCK
__bdevname(ROOT_DEV, b);
#endif
printk("VFS: Cannot open root device \"%s\" or %s: error %d\n",
root_device_name, b, err);
printk("Please append a correct \"root=\" boot option; here are
the available partitions:\n");
printk_all_partitions();
#ifdef CONFIG_DEBUG_BLOCK_EXT_DEVT
printk("DEBUG_BLOCK_EXT_DEVT is enabled, you need to specify "
"explicit textual name for \"root=\" boot option.\n");
#endif
panic("VFS: Unable to mount root fs on %s", b);
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1797581
Title:
Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to
kernel panic
Status in MAAS:
Incomplete
Status in linux package in Ubuntu:
Confirmed
Status in qemu package in Ubuntu:
Confirmed
Bug description:
Using latest MAAS master, I'm unable to compose a VM over the UI
successfully when composed with 2048 MB of RAM. By that I mean that
the VM is created, but it fails with a kernel panic.
To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Just reproduced it on my env (where things used to work) after updating from MAAS 2.5.0~rc2 to 2.5.2. https://paste.ubuntu.com/p/CY834ZGnvS/ [ 15.458594] VFS: Cannot open root device "squash:http://10.10.101.2:5248/images/ubuntu/amd64/generic/bion; or unknown-block(0,0): error -6 tree /var/lib/maas/boot-resources/ /var/lib/maas/boot-resources/ # ... ├── current -> snapshot-20190419-115735 └── snapshot-20190419-115735 Which binaries do I need to have uploaded? sha256sum /var/lib/maas/boot-resources/snapshot-20190419-115735/ubuntu/amd64/ga-18.04/bionic/daily/* 69ca457a119fe309d315972ca2756a17bd9bc55bc98f2bea5542566a7f41b08f /var/lib/maas/boot-resources/snapshot-20190419-115735/ubuntu/amd64/ga-18.04/bionic/daily/boot-initrd 166853ad9342fdf5be17988e5e18cbf0458ab0da94f18f5e331b3581e3610b97 /var/lib/maas/boot-resources/snapshot-20190419-115735/ubuntu/amd64/ga-18.04/bionic/daily/boot-kernel 1e7841d7fca13ef27c2742cfc2c3a2d59491a74c268b705a02eb4ee8f673d150 /var/lib/maas/boot-resources/snapshot-20190419-115735/ubuntu/amd64/ga-18.04/bionic/daily/squashfs ii seabios1.10.2-1ubuntu1 all Legacy BIOS implementation ii ipxe-qemu 1.0.0+git-20180124.fbe8c52d-0ubuntu2.2 all PXE boot firmware - ROM images for qemu ii qemu-kvm 1:2.11+dfsg-1ubuntu7.12 amd64QEMU Full virtualization on x86 hardware ii libvirt0:amd64 4.0.0-1ubuntu8.8 amd64library for interfacing with different virtualization systems -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Ok, thanks Jason. That is great info, but about as much as we had so far :-/ Can you set your deployment up in a way that if that happens again you can collect these binaries? If it right now still triggers if you (re)deploy a 2G guest use the chance to finally give us the bits that we will need to reach the next stage of debugging. Lesson learned from before on this bug, it might already be gone by tomorrow, so I hope you have time and chance to get those xml+image+kernel+initrd. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
@Christian - release: bionic - seabios: 1.10.2-1ubuntu1 - qemu: 1:2.11+dfsg-1ubuntu7.10 - libvirt: 4.0.0-1ubuntu8.8 - ovmf - this is a uefi thing right? we're not using it. - kernel 2019-03-18T12:17:11+00:00 elastic-2 kernel: [0.00] Linux version 4.15.0-46-generic (buildd@lgw01-amd64-038) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019 (Ubuntu 4.15.0-46.49-generic 4.15.18) I don't have copies of the binaries from this run - it was from daily maas images: 2019-03-18T12:03:37.296671+00:00 leafeon maas.import-images: [info] Region downloading image descriptions from 'http://images.maas.io/ephemeral-v3/daily/'. I don't see anything in the logs to indicate an ID number for the kernel, initrd, or image coming there. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Jason, thanks for chiming in - so far this was non-reproducible every time we looked at it. We had plenty of approaches with the MAAS team to this and later on with Thiago, but still miss the right data to continue. Last time I asked - I think it was still Mike - that I'd really like to get the full set of binaries involved, but then IIRC he couldn't reproduce it with the newer images/kernel of that day. Maybe we can use this issue hitting you now as a new chance to get what we need to continue debugging to reach a resolution. @Jason - can I get from you the full set of elements used when you hit it? That would be: - which release are you on? - which qemu/libvirt/seabios/ovmf do you have installed exactly - can you share the XML that the guest is defined with We have the above from different cases, but it might be important to know which ones exactly are active in your case, so better check them. Finally - and that was what was missing before so far: - can you attach the set of binaries that are used which should be - cloud-img - kernel (as maas provides that directly to the guest in guest XML) - initrd (as maas provides that directly to the guest in guest XML) - describe your KVM Host server HW (so that we can test on something similar) As an alternative - can you provide a login on such a host which has a breaking guest defined. There we could work together fetching the required data? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Bumped to field-high as we ran into this again in testing. We have a workaround, but it's to not use 2G VM's, which is really silly and hard to remember when we go and add new deployments, especially because the failure mode is not obvious at all. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: maas Importance: Critical => Undecided -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Ryan, If I remove the UEFI line (back to BIOS), the machine enters in Kernel Panic during the boot / commissioning. That was actually, the very test that I executed in first place (add UEFI to see)! Then, I came with this UEFI workaround, which is even better for me. Cheers! Thiago -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
Re: [Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
On Mon, Jan 14, 2019 at 1:55 PM Thiago Martins wrote: > @Ryan, > > The working XML file is attached here, with 2048 MB of RAM. > > NOTE: This XML was created using Virt-Manager, then, MaaS took it over > after being "refreshed". > Thanks! If you drop the section (which is what tells libvirt to boot via UEFI) does your VM still work? hvm /usr/share/OVMF/OVMF_CODE.fd /var/lib/libvirt/qemu/nvram/vunft-1_VARS.fd Note that MAAS boots UEFI images with grub2[1], not pxeboot/ipxe/seabios; so I think we can narrow down the error to the non-uefi case which may help find the issue. 1. http://images.maas.io/ephemeral-v3/daily/bootloaders/uefi/amd64/20181123.0/ > > Cheers! > Thiago > > ** Attachment added: "vunft-1.xml" > > https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5229030/+files/vunft-1.xml > > -- > You received this bug notification because you are subscribed to qemu in > Ubuntu. > https://bugs.launchpad.net/bugs/1797581 > > Title: > Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to > kernel panic > > To manage notifications about this bug go to: > https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions > -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
@Ryan, The working XML file is attached here, with 2048 MB of RAM. NOTE: This XML was created using Virt-Manager, then, MaaS took it over after being "refreshed". Cheers! Thiago ** Attachment added: "vunft-1.xml" https://bugs.launchpad.net/maas/+bug/1797581/+attachment/5229030/+files/vunft-1.xml -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
@Thiago, Can you attach your guest XML that's working successfully with 2048MB? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I'm also facing this problem. My workaround is to compose VMs using Virt-Manager, Firmware = UEFI for the VM and then, refreshing the MaaS Pod. There is a need to install ovmf sudo apt install ovmf On MaaS Pod. Then, no more Kernel Panic! Details: https://discourse.maas.io/t/maas-2-5-bionic-pod-composing-guests-kernel- panic-no-init/302 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Reproduced today on MAAS 2.5 latest bionic It is not reproduced if switch from ga-18.04 kernel to ga--18.04-low- latency kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
@Ryan - the resource and other bits being different is because in the bad case the domain was up and in the good case down. That in itself is not a problem. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
Re: [Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
One other oddity in the xml is the cgroup construction in the "bad" case. /machine > > > To manage notifications about this bug go to: > https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions > -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Again it seems to break on initramfs being bad: [0.840153] Initramfs unpacking failed: no cpio magic @Mike/Andres - can you reproduce it on your own test env this time? @Vladimir - could you internally share exactly your kernel and initrd that is tried to be booted in this case? Last time we wondered if the issue could be "in there" so it would be great to have exactly those that fail for you shared to try reproducing on them. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: qemu (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: linux (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Setting this back to incomplete for MAAS because there is not enough information to determine this is a MAAS Bug. That said, given that this work with any VM other than one with 2048 of RAM, the issue may appear to be the kernel or libvirt. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Subscribing field-medium as workaround (e.g not to use 2048mb ram VMs) exists, but solution still need to be found. Also, before yesterday (e.g on libvirt 1:2.11+dfsg-1ubuntu7.8 + maas 2.5rc2) worked fine. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: maas Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Confirmed Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Reproduced today on MAAS 2.5 / latest Bionic. qemu/maas versions: https://paste.ubuntu.com/p/kDk3c9tVFX/ good vm dump: http://paste.ubuntu.com/p/zTW2p6JGQJ/ good vm log: http://paste.ubuntu.com/p/V7GP9KgP8S/ good vm serial output: http://paste.ubuntu.com/p/FDymQcH8qD/ bad vm dump: http://paste.ubuntu.com/p/ZCv5dv3DTS/ bad vm log: http://paste.ubuntu.com/p/4Zk5qq9TfB/ bad vm serial output: http://paste.ubuntu.com/p/QgbmZsjPPP/ ** Changed in: maas Status: Incomplete => Confirmed ** Changed in: linux (Ubuntu) Status: Incomplete => New ** Changed in: qemu (Ubuntu) Status: Incomplete => New ** Tags added: cpe-onsite ** Tags added: field-medium -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: New Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Hi, thanks for your feedback. We are at least much closer to what happened and thereby should be faster if it reoccurs. I don't think it is an interaction of the Kernel and Squashfs as we found that already initramfs unpack was broken. That is before Squash comes into play. I'll keep my test setup until I need to re-deploy the host, which usually is about every 1-2 weeks. If you ever find old kernel/initrd combinations or any new one to trigger it again - please share them via e.g. internal private fileshare - I sent you some details on IRC. Lets see if it comes up again. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Looking again at the date stamps, I don't see any squashfs filesystems older than October 15th. The kernels are all from ~September 25th. So I feel like this must have been an interaction between the kernel September 25th and whatever the previous squashfs was. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
After triaging this again on a call with Andres (who originally reported this) this morning, we determined that this issue is no longer reproducible with MAAS. The only way for me to explain it at this point was that there was something wrong with the daily image MAAS was using last week, and a subsequent update fixed it. (It looks like the images were updated yesterday.) Sorting my /var/lib/maas/boot-resources/cache directory by last- modified, I see the following: https://paste.ubuntu.com/p/mc76rffz5k/ @paelzer, if you'd like to test with any of those images, I've made copies. (Not sure where to put them, though - I wonder if your MAAS synced them as well?) Still very weird that it worked in all cases we tested except when we allocated 2048 MB RAM. I'd like to thank Ryan and Christian for their efforts on this "heisenbug". We should think about how to better handle issues like this, so that it's easier to peel back the layers and get to a point where everyone's environment can be consistent without this much effort. And we'll reopen this if it returns. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: linux (Ubuntu) Status: Confirmed => Incomplete ** Changed in: maas Status: Invalid => Incomplete ** Changed in: qemu (Ubuntu) Status: Invalid => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Incomplete Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
There actually is no need to debug further on my non-maas PXE setup. As identified before - your setup breaks due to [ 20.690329] Initramfs unpacking failed: junk in compressed archive Everything else is a follow on issue, with eventually: [ 22.524541] VFS: Cannot open root device "squash:http://172.16.99.2:5248/images/ubuntu/amd64/generic/bion; or unknown-block(0,0): error -6 But since my setup as outlined in comment #31 PXE-boots the kernel+initramfs just fine I don't have to debug the rest. I already passed the point where it breaks for you. Can you use the above hints to get your breaking setup to also one-by-one remove components. I'd assume that removing libvirt makes no difference to your case. But you could maybe end up with: - PXE-served by MAAS = bad - PXE serverd by tftpd = good Or anything like it. So please follow the hints above (or get in touch if you need me to do so) and convert your setup until you can identify which component makes the decision for good/bad case. Also still I'd be open to get your kernel/initramfs/squash/pxe/ files to implant it into my setup for a test if any of them is the trigger. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Taking the x86 pxelinux.0 from /usr/lib/PXELINUX/pxelinux.0 and otherwise doing the same tftp setup as in [1] and switching from the "maas" to the "default" network where I have set up dhcp to netboot by libvirt as in [1]. I can see netboot happening. With that instead of pushing things from libvirt via kernel/initrd tags I now provide it via PXE similar to your config. TL;DR - tftp serves: lpxelinux.0 + pxe modules + PXEconfig - nginx serves: kernel/initrd/squashfs - qemu started directly without libvirt Still gets the inintial kernel booting fine and then failing to mount the squash: mount: mounting squash:http://192.168.122.1:80/squashfs on /root failed: No such device I can from the initramfs mount it: (initramfs) wget http://192.168.122.1:80/squashfs Connecting to 192.168.122.1:80 (192.168.122.1:80) squashfs 100% |***| 174M 0:00:00 ETA (initramfs) mount -t squashfs squashfs /root/ It is mounted just fine: /dev/loop0 on /root type squashfs (ro,relatime) It is possible that I'm back at the same ordering issue that I had of the IP coming up too late to mount the squashfs, but later works fine. I mounted it with 2047/2048/2049 mb gusts without a problem. Despite all work on this I'd still need a better reproducer :-/ I might take a look at rebuilding a more verbose initramfs for that after lunch. --- config details --- $ find /srv/tftp/ /srv/tftp/ /srv/tftp/squashfs /srv/tftp/boot-kernel /srv/tftp/ldlinux.c32 /srv/tftp/lpxelinux.0 /srv/tftp/pxelinux.cfg /srv/tftp/pxelinux.cfg/01-52-54-00-3b-72-0a /srv/tftp/boot-initrd Kernel/initrd/squash as described in comment #18 # modified to get scrollable direct console (but no iPXE VGA output) #!/bin/bash /usr/bin/qemu-system-x86_64 -name guest=horsea-kvm-pod-2047,debug-threads=on -machine pc-i440fx-bionic,accel=kvm,usb=off,dump-guest-core=off -m 2047 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -cpu kvm64 -rtc base=utc -no-shutdown \ \ -nographic -serial mon:stdio \ \ -drive file=/var/lib/uvtool/libvirt/images/d9658e2f-d4c7-4a55-8ecc-b216612fe410,format=raw,if=none,id=drive-virtio-disk0 \ -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,serial=d9658e2f-d4c7-4a55-8ecc-b216612fe410 \ \ -boot order=n,strict=on \ \ -net bridge,br=virbr0,helper=/usr/lib/qemu/qemu-bridge-helper -net nic,model=virtio,macaddr=52:54:00:3b:72:0a # For access to the iPXE / lpxelinux graphical output #-curses \ #-chardev stdio,mux=on,id=charserial0,logfile=/tmp/test.log \ #-serial chardev:charserial0 \ #-mon chardev=charserial0,mode=readline \ # For access to a scrollable direct console and monitor # -nographic -serial mon:stdio $ cat /srv/tftp/pxelinux.cfg/default DEFAULT execute LABEL execute SAY Booting under MY direction... KERNEL http://192.168.122.1:80/boot-kernel INITRD http://192.168.122.1:80/boot-initrd APPEND console=ttyS0 nomodeset ro root=squash:http://192.168.122.1:80/squashfs ip=horsea-kvm-pod-2047:BOOTIF ip6=off overlayroot=tmpfs overlayroot_cfgdisk=disabled apparmor=0 IPAPPEND 2 [1]: http://ubuntu-on-big-iron.blogspot.com/2017/12/pxe-netboot-kvm-s390x.html -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I didn't let me calm down - I found in the doc [1] that the switch might be on the VLAN and not the subnet (Hrm why?) It was on the vlan http://horsea:5240/MAAS/#/vlan/5021 Not on the subnet http://horsea:5240/MAAS/#/subnet/11 There I found the "provide dhcp" entry \o/ That unlocked some understanding of what the range allocations would be like on vland/subnets. I reconfigured the range allocation and hit the "provide DHCP" switch. Looking back all is reasonable, just some stumbling on my first maas setup from scratch :-/ Must be funny for you seeing me struggle doing so :-) With that in place I created three pods with 2047/2048/2049 MB memory. ubuntu@node-horsea:~/maas$ virsh dumpxml horsea-kvm-pod-2047 | grep emor 2096128 2096128 ubuntu@node-horsea:~/maas$ virsh dumpxml horsea-kvm-pod-2048 | grep emor 2097152 2097152 ubuntu@node-horsea:~/maas$ virsh dumpxml horsea-kvm-pod-2049 | grep emor 2098176 2098176 They all went into the commissioning phase, but who would be surprised got stuck somewhere in there :-/ OTOH the bug says "composing crashes" so we might be at the right place, lets take a look at these three guests. [1]: https://docs.maas.io/devel/en/installconfig-network-dhcp#enabling-dhcp -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I didn't see any guests crashing, instead they all just hang finding nothing to boot. I reduced this to just qemu (for debugging later on) but it reliably breaks finding nothing from PXE. Guest Console: iPXE (PCI 00:03.0) starting execution...ok iPXE initialising devices...ok iPXE 1.0.0+git-20180124.fbe8c52d-0ubuntu2.1 -- Open Source Network Boot Firmware -- http://ipxe.org Features: DNS HTTP HTTPS iSCSI NFS TFTP AoE ELF MBOOT PXE bzImage Menu PXEXT net0: 52:54:00:3b:72:0a using virtio-net on :00:03.0 (open) [Link:up, TX:0 TXE:0 RX:0 RXE:0] Configuring (net0 52:54:00:3b:72:0a).. No configuration methods succeeded (http://ipxe.org/040ee119) No more network devices Corresponding command to start qemu: #!/bin/bash /usr/bin/qemu-system-x86_64 -name guest=horsea-kvm-pod-2047,debug-threads=on -machine pc-i440fx-bionic,accel=kvm,usb=off,dump-guest-core=off -m 2047 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -cpu kvm64 -rtc base=utc -no-shutdown -curses \ -chardev socket,id=charmonitor,path=/tmp/horsea-kvm-pod-2047.monitor.sock,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -chardev stdio,mux=on,id=charserial0,logfile=/tmp/horsea-kvm-pod-2047.log \ -device isa-serial,chardev=charserial0,id=serial0 \ -drive file=/var/lib/uvtool/libvirt/images/d9658e2f-d4c7-4a55-8ecc-b216612fe410,format=raw,if=none,id=drive-virtio-disk0 \ -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,serial=d9658e2f-d4c7-4a55-8ecc-b216612fe410 \ \ -boot order=n,strict=on \ \ -net bridge,br=virbr1,helper=/usr/lib/qemu/qemu-bridge-helper -net nic,model=virtio,macaddr=52:54:00:3b:72:0a This is the mac that is also in the libvirt XML. The bridge is the one served by MAAS and it seems there is just no PXE responding to it. Since the two other guests as started by maas+libvirt hang just the same way it most likely is a general MAAS setup issue. Please help me to get MAAS to reply the way you would usually expect it, to then be able to go on with debugging. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I must admit, compared to you just sharing your kernel/initrd/squash asking to install an own dev maas in comment #23 consumes quite some time :-/ Waiting for a sync here, setting up users there, sorting out how to make us provide PXE and such on the "maas" virtual network, ... - it just isn't one click and ready. Initially things worked other than the unexpected "wait for image sync". I've got a Pod registered and working to get to libvirt data. But compose blocks at: "Pod unable to compose machine: Please add a 'default' or 'maas' network whose bridge is on a MAAS DHCP enabled VLAN. Ensure that libvirt DHCP is not enabled." Well that is clear, but a link where/how to get MAAS dhcp/pxe to own that vlan would be nice. Maybe something small and easy for you that would help to be added. When I go to subnets to create one (to have maas own it) for 172.16.99.0/24 that I created following your link it tells me the subnet already exists "Error: Subnet with this Cidr already exists." - but it isn't in the subnet overview so I can't enable DHCP/PXE on it :-/ My overview suffers a bit from the links you had adding sample-data, I cleaned up a lot of demo-BS and can now see more clearly. I'd want to add a subnet to my fabric that I called "libvirt-maas" being the virbr1 "maas" definition by libvirt. Afterwards I still get "Error: Subnet with this Cidr already exists." Well it might conflict with the "172.16.99.1" set up by. Lets try to set up another subnet, that worked. Also added a range of IPs in the hope that this would switch DHCP on, but it didn't So on the subnet's own page "managed allocation" is enabled. But when clicking on the top menu subnets the column for DHCP in the table is "disabled" Anyway, lets try to compose something ... No, still blocked at "Pod unable to compose machine: Please add a 'default' or 'maas' network whose bridge is on a MAAS DHCP enabled VLAN. Ensure that libvirt DHCP is not enabled." That exceeds what I can find on [2] for DHCP enabling, but still refuses me. Since reusing maas DHCP/PXE setup is exactly what I wanted to debug for/with you I'm stuck, lost time and we are not a bit further on this bug here :-/ Now trying to squeeze your PXE config into tftp/libvirt manually without MAAS - lets see if I can reproduce via that. [1]: https://discourse.maas.io/t/setting-up-a-flexible-virtual-maas-test-environment/142 [2]: https://docs.maas.io/devel/en/installconfig-network-subnet-management -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
... it's interesting that [practically?] the same kernel version that fails consistently with Bionic works just fine with Xenial. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I just tried Xenial with the HWE kernel - same result (success). FYI. https://paste.ubuntu.com/p/WpqHnzbsS7/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Good idea @rharper. It's easy for MAAS to attempt commissioning on Xenial or Bionic, so I gave Xenial a try. It works fine![1] [1]: Console log - https://paste.ubuntu.com/p/58hfBX6BhY/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Does this fail with other releases? like trusty? I was wondering if initrd size plays a factor here: precise/hwe-t: 25M trusty/hwe-x: 35M xenial/ga: 39M xenial/hwe:53M xenial/edge: 53M bionic/ga: 55M cosmic/ga: 57M That might be faster for you to test than for us to replicate setup. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: qemu (Ubuntu) Status: Incomplete => Invalid -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Invalid Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I agree that this doesn't look like a QEMU issue, and I agree that it doesn't look like a [general] networking hiccup. @paelzer, the easiest way to get the exact configs you need would be to install MAAS similar to how I've described on our discourse forum[1]. The PXE config will be /similar/ to this[2] (copied/pasted from /usr/lib/maas/maas-test-enlistment on my MAAS server). [1]: MAAS setup instructions https://discourse.maas.io/t/setting-up-a-flexible-virtual-maas-test-environment/142 [2]: PXE config https://paste.ubuntu.com/p/KBsCHCfKBD/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Changed in: maas Milestone: 2.5.0rc1 => None -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Since it is reproducible maybe not a networking hiccup. But maybe a hiccup of the PXE setup (thats why I asked for it). Or a hiccup of the quite complex shim loading if signed kernels are used. @Mike - in addition to my questions above could you use a non signed/shim load pattern as well to check if that might be part of the reason? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
[0.943808] Unpacking initramfs... [ 20.690329] Initramfs unpacking failed: junk in compressed archive [ 20.703673] Freeing initrd memory: 56612K Looks like the initrd was compromised, possibly a networking hiccup? Can you confirm the checksums on the source and attempt to download the URL ? I can't see why the size of the VMs ram makes a difference though. But I don't think this is a qemu issue any more. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Working on this I found by accident that I actually can reproduce: VFS: Cannot open root device "squash:http://192.168.122.1:80/squashfs; or unknown-block(0,0): error -6 But the way I got there lets assume some more potential reasons. I got there by breaking my initramfs :-) After realizing this I removed the initramfs from the guest definition and got to just the same error. The reason is that without initramfs the kernel is responsible to handle root= and it has no idea of squashfs. With that knowledge I re-checked your log at: https://paste.ubuntu.com/p/mkF6Kp4hx8/ It also has no entry like: Loading, please wait... starting version 237 Which you'd see if systemd in the initrd would take over. So your bad case also fails to load the initramfs! That said, why could that be special for just this memory size? Theories related to the guest size impacting this: - initrd is placed explicit in PXE config now conflicting with kernel allocations - initrd is misplaced/misread by PXE code in qemu @Mike - I'd want to know your exact PXE config @Mike - It would be great to attach your kernel+initrd+squashfs+rootdisk files to the bug. Hopefully we can reproduce by providing kernel+initrd via PXE and varying the guest size. ** Changed in: qemu (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: Incomplete Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
After DHCP is up it works just fine. (initramfs) wget http://192.168.122.1:80/squashfs Connecting to 192.168.122.1:80 (192.168.122.1:80) squashfs 100% |***| 174M 0:00:00 ETA (initramfs) mount -t squashfs squashfs /root (initramfs) mount rootfs on / type rootfs (rw) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) udev on /dev type devtmpfs (rw,nosuid,relatime,size=1007984k,nr_inodes=251996,mode=755) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=204128k,mode=755) tmpfs-root on /media/root-rw type tmpfs (rw,relatime) copymods on /root/lib/modules type tmpfs (rw,relatime) /dev/loop0 on /root type squashfs (ro,relatime) So if anyone has a good hint how to get out of the ip/squash-root ordering issue let me know. That would - as mentioned - help to most likely exclude maas and libvirt from the suspects to be able to debug further. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
As Ryan I can not reproduce locally - hrm. The crash in your log is the root-fs mount. [ 22.524541] VFS: Cannot open root device "squash:http://172.16.99.2:5248/images/ubuntu/amd64/generic/bion; or unknown-block(0,0): error -6 [ 22.575588] Please append a correct "root=" boot option; here are the available partitions: [ 22.583909] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) Also we have to stick to exactly your values (one of the repros had a slightly different value) 2096128 2096128 I tried with the exact numbers above but "normal" cloud image boot is still ok. I wonder if the kernel has an off by one error e.g. aligning the squash at the lowest 2G but with just this amount of memory choosing a place it would not fit. We'd need to set up a local http and serve squashfs, to boot into that. With some luck we can reproduce there and then eliminate libvirt and maas out of the equation. Repro: - I started off as Ryan did with a Cloud Image test via UVTool. - Next I extracted the kernel+initrd from the guest to provide those from the host (as you do via PXE) - installed nginx - made initrd available on /var/www/html/boot-initrd (initrd.img-4.15.0-36-generic) - made kernel available on /var/www/html/boot-kernel (vmlinuz-4.15.0-36-generic) - The address of the Host on the libvirt net is 192.168.122.1, verify the guest can http from there - get matching squash (see below for details) - get an empty qemu disk via qemu-img like the type raw that maas uses sudo qemu-img create -f raw /var/lib/libvirt/images/empty-root.img 10G - With that, modify the guest to use these kernel/initrd/sqashfs/empty-root XML of the guest: http://paste.ubuntu.com/p/PhBn6n8VYH/ I have a dependency issue for my repro, that is IP being configured in /scripts/init-bottom after trying to mount squashfs in what seems /scripts/local-premount. http://paste.ubuntu.com/p/RscHmQqFyY/ I can even fetch the squashfs from the initramfs, wouldn't you be affected by the same ordering issue? I need to find how you usually get around that to continue the repro that hopefully eventually helps to focus on the root cause. I experimented a bit more and asked around on IRC. But so far I can't get past the ordering issue that IP is initialized to late and due to that squash is failing. -- Appendix -- Get Squash: To continue I'd need the current squashfs instead of the disk image. My uvtool spawned this for me: $ uvt-simplestreams-libvirt --verbose query release=bionic arch=amd64 label=daily (20181012) So lets get the mathcing suqash URL and fetch that. $ sstream-query --output-format="%(item_url)s" --no-verify http://cloud-images.ubuntu.com/daily arch=amd64 release=bionic label=daily ftype=squashfs version_name=20181012 http://cloud-images.ubuntu.com/daily/server/bionic/20181012/bionic-server-cloudimg-amd64.squashfs $ sudo wget -O /var/www/html/squashfs http://cloud-images.ubuntu.com/daily/server/bionic/20181012/bionic-server-cloudimg-amd64.squashfs Note: That setup is available on server horsea -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Here's a full console log from the failure. https://paste.ubuntu.com/p/zmS7CP7NKr/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
It's an empty image - MAAS PXE boots the VM. Could you give it a try with MAAS? I can help you with the setup if needed - just ping me on IRC. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
I'm unable to recreate with a daily bionic cloud-image on a bionic host with the same versions. % sudo apt install uvtool libvirt % uvt-simplestreams-libvirt -vv sync --source http://cloud-images.ubuntu.com/daily 'supported=True' arch=amd64 release=bionic % uvt-kvm create --memory 2048 --cpu 1 --disk 10 rharper-b1 label=daily release=bionic % virsh dumpxml rharper-b1 | grep Mem 2097152 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
The backing image: /var/lib/libvirt/maas-images/e5d185a9-8ccb-4ca6-959a-bd8eff0ee184 What boot image is that? Can I get a copy of that from maas-images? or how is it created? On the node with the vm that fails, can you: virsh start --console Assuming it's a normal ubuntu image which has normal console= settings, it should dump the boot console to the terminal so we can capture the full boot to panic. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Here's the log from the failing VM. Doesn't look too unusual to me... https://paste.ubuntu.com/p/JpmSxmwjfM/ ** Description changed: Using latest MAAS master, I'm unable to compose a VM over the UI - successfully. BY this it means that the VM is created, but it fails with - a kernel panic. - - If I compose a VM over the API, the issue does *not* exist. So this is a - UI issue. + successfully when composed with 2048 MB of RAM. By that I mean that the + VM is created, but it fails with a kernel panic. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully when composed with 2048 MB of RAM. By that I mean that the VM is created, but it fails with a kernel panic. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
And /var/log/libvirt/qemu/.log ? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully. BY this it means that the VM is created, but it fails with a kernel panic. If I compose a VM over the API, the issue does *not* exist. So this is a UI issue. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Here's an example of working XML (with 2047 MB RAM) that MAAS generated: https://paste.ubuntu.com/p/mkF6Kp4hx8/ And here's an example of non-working XML (with 2048 MB RAM) that MAAS generated: https://paste.ubuntu.com/p/27HHDrzwm8/ ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully. BY this it means that the VM is created, but it fails with a kernel panic. If I compose a VM over the API, the issue does *not* exist. So this is a UI issue. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Here's the version information. libvirt-bin: Installed: 4.0.0-1ubuntu8.5 Candidate: 4.0.0-1ubuntu8.5 Version table: *** 4.0.0-1ubuntu8.5 500 500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages 100 /var/lib/dpkg/status 4.0.0-1ubuntu8.2 500 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages 4.0.0-1ubuntu8 500 500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages qemu-kvm: Installed: 1:2.11+dfsg-1ubuntu7.6 Candidate: 1:2.11+dfsg-1ubuntu7.6 Version table: *** 1:2.11+dfsg-1ubuntu7.6 500 500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages 100 /var/lib/dpkg/status 1:2.11+dfsg-1ubuntu7.3 500 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages 1:2.11+dfsg-1ubuntu7 500 500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages qemu-system-x86: Installed: 1:2.11+dfsg-1ubuntu7.6 Candidate: 1:2.11+dfsg-1ubuntu7.6 Version table: *** 1:2.11+dfsg-1ubuntu7.6 500 500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages 100 /var/lib/dpkg/status 1:2.11+dfsg-1ubuntu7.3 500 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages 1:2.11+dfsg-1ubuntu7 500 500 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages linux-image-4.15.0-34-generic: Installed: 4.15.0-34.37 Candidate: 4.15.0-34.37 Version table: *** 4.15.0-34.37 500 500 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages 500 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages 100 /var/lib/dpkg/status 4.15.0-34.37~16.04.1 500 500 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Confirmed Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully. BY this it means that the VM is created, but it fails with a kernel panic. If I compose a VM over the API, the issue does *not* exist. So this is a UI issue. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
Can you attach the guest xml and host kernel/qemu/libvirt packages? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully. BY this it means that the VM is created, but it fails with a kernel panic. If I compose a VM over the API, the issue does *not* exist. So this is a UI issue. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1797581] Re: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic
** Summary changed: - Composing a VM in MAAS with exactly 2048 MB RAM causes kernel panic + Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1797581 Title: Composing a VM in MAAS with exactly 2048 MB RAM causes the VM to kernel panic Status in MAAS: Invalid Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: New Bug description: Using latest MAAS master, I'm unable to compose a VM over the UI successfully. BY this it means that the VM is created, but it fails with a kernel panic. If I compose a VM over the API, the issue does *not* exist. So this is a UI issue. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1797581/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
