Public bug reported: This is something to do with the configuration or software versions running here, since we have identical hardware running in an adjacent cloud region but with different versions of the cloud/virt stack.
When we boot bionic arm64 Ubuntu cloud images in "bos01" and they land on the eMAG systems, they always fail like this: ... [ 1.585611] Key type dns_resolver registered [ 1.587408] registered taskstats version 1 [ 1.588913] Loading compiled-in X.509 certificates [ 1.592668] Loaded X.509 cert 'Build time autogenerated kernel key: 4a4a555bc5fd0178c9ab722f3ae7b392f7714ac4' [ 1.598866] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969' [ 1.605389] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19' [ 1.610861] Couldn't get size: 0x800000000000000e [ 1.613413] MODSIGN: Couldn't get UEFI db list [ 1.615920] Couldn't get size: 0x800000000000000e [ 1.618256] MODSIGN: Couldn't get UEFI MokListRT [ 1.620315] Couldn't get size: 0x800000000000000e [ 1.622317] MODSIGN: Couldn't get UEFI dbx list [ 1.624446] zswap: loaded using pool lzo/zbud [ 1.628185] Key type big_key registered [ 1.629937] Key type trusted registered [ 1.632012] Key type encrypted registered [ 1.633668] AppArmor: AppArmor sha1 policy hashing enabled [ 1.635625] ima: No TPM chip found, activating TPM-bypass! (rc=-19) [ 1.638009] ima: Allocated hash algorithm: sha1 [ 1.639272] evm: HMAC attrs: 0x1 [ 1.640875] rtc-efi rtc-efi: setting system clock to 2021-05-11 09:28:43 UTC (1620725323) [ 1.646247] Freeing unused kernel memory: 5824K [ 1.657870] Checked W+X mappings: passed, no W+X pages found [ 1.660250] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000005 [ 1.660250] [ 1.663294] CPU: 1 PID: 1 Comm: init Not tainted 4.15.0-142-generic #146-Ubuntu [ 1.665939] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 [ 1.668204] Call trace: [ 1.668981] dump_backtrace+0x0/0x198 [ 1.670212] show_stack+0x24/0x30 [ 1.671282] dump_stack+0x98/0xc8 [ 1.672433] panic+0x128/0x2b0 [ 1.673502] do_exit+0x75c/0xa80 [ 1.674749] do_group_exit+0x40/0xb0 [ 1.676191] get_signal+0x114/0x6e8 [ 1.677867] do_signal+0x18c/0x240 [ 1.679498] do_notify_resume+0xd0/0x328 [ 1.681302] work_pending+0x8/0x10 [ 1.682771] SMP: stopping secondary CPUs [ 1.684624] Kernel Offset: disabled [ 1.686119] CPU features: 0x04802008 [ 1.687353] Memory Limit: none [ 1.688453] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000005 [ 1.688453] Full example log: https://autopkgtest.ubuntu.com/results/autopkgtest- bionic/bionic/arm64/c/chromium-browser/20210511_093739_ea3f2@/log.gz We tried to get IS to roll things back to match the working bos02, but they said it's too different and not possible. The working cloud is running Mitaka from the cloud archive on Xenial (qemu 2.5) The broken cloud is running Queens from the cloud archive on Xenial (qemu 2.11) The other thing someone suggested we try is that MDS mitigation is enabled in the broken cloud, so we could disable it. No idea if that makes sense tbh. That's about all we have right now. ** Affects: qemu (Ubuntu) Importance: Undecided Status: New ** Description changed: This is something to do with the configuration or software versions running here, since we have identical hardware running in an adjacent cloud region but with different versions of the cloud/virt stack. When we boot bionic Ubuntu cloud images in "bos01", they always fail like this: ... [ 1.585611] Key type dns_resolver registered [ 1.587408] registered taskstats version 1 [ 1.588913] Loading compiled-in X.509 certificates [ 1.592668] Loaded X.509 cert 'Build time autogenerated kernel key: 4a4a555bc5fd0178c9ab722f3ae7b392f7714ac4' [ 1.598866] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969' [ 1.605389] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19' [ 1.610861] Couldn't get size: 0x800000000000000e [ 1.613413] MODSIGN: Couldn't get UEFI db list [ 1.615920] Couldn't get size: 0x800000000000000e [ 1.618256] MODSIGN: Couldn't get UEFI MokListRT [ 1.620315] Couldn't get size: 0x800000000000000e [ 1.622317] MODSIGN: Couldn't get UEFI dbx list [ 1.624446] zswap: loaded using pool lzo/zbud [ 1.628185] Key type big_key registered [ 1.629937] Key type trusted registered [ 1.632012] Key type encrypted registered [ 1.633668] AppArmor: AppArmor sha1 policy hashing enabled [ 1.635625] ima: No TPM chip found, activating TPM-bypass! (rc=-19) [ 1.638009] ima: Allocated hash algorithm: sha1 [ 1.639272] evm: HMAC attrs: 0x1 [ 1.640875] rtc-efi rtc-efi: setting system clock to 2021-05-11 09:28:43 UTC (1620725323) [ 1.646247] Freeing unused kernel memory: 5824K [ 1.657870] Checked W+X mappings: passed, no W+X pages found [ 1.660250] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000005 - [ 1.660250] + [ 1.660250] [ 1.663294] CPU: 1 PID: 1 Comm: init Not tainted 4.15.0-142-generic #146-Ubuntu [ 1.665939] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 [ 1.668204] Call trace: [ 1.668981] dump_backtrace+0x0/0x198 [ 1.670212] show_stack+0x24/0x30 [ 1.671282] dump_stack+0x98/0xc8 [ 1.672433] panic+0x128/0x2b0 [ 1.673502] do_exit+0x75c/0xa80 [ 1.674749] do_group_exit+0x40/0xb0 [ 1.676191] get_signal+0x114/0x6e8 [ 1.677867] do_signal+0x18c/0x240 [ 1.679498] do_notify_resume+0xd0/0x328 [ 1.681302] work_pending+0x8/0x10 [ 1.682771] SMP: stopping secondary CPUs [ 1.684624] Kernel Offset: disabled [ 1.686119] CPU features: 0x04802008 [ 1.687353] Memory Limit: none [ 1.688453] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000005 - [ 1.688453] + [ 1.688453] Full example log: https://autopkgtest.ubuntu.com/results/autopkgtest- bionic/bionic/arm64/c/chromium-browser/20210511_093739_ea3f2@/log.gz We tried to get IS to roll things back to match the working bos02, but they said it's too different and not possible. The working cloud is running Mitaka from the cloud archive on Xenial (qemu 2.5) The broken cloud is running Queens from the cloud archive on Xenial (qemu 2.11) + The other thing someone suggested we try is that MDS mitigation is + enabled in the broken cloud, so we could disable it. No idea if that + makes sense tbh. + That's about all we have right now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1931728 Title: [scalingstack bos01] bionic (arm64) instances always fail to boot on eMAGs in this cloud To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1931728/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
