[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Thanks for the update @davidkrauser, and very happy that groovy is working out. Is the fix due for focal sometime in the coming weeks or months? Any extra info is appreciated. Also happy holidays and end of year, and thank you all for what you do! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in cloud-images: Confirmed Status in grub2 package in Ubuntu: Won't Fix Status in linux-azure package in Ubuntu: Invalid Status in livecd-rootfs package in Ubuntu: Triaged Status in grub2 source package in Focal: Confirmed Status in linux-azure source package in Focal: Invalid Status in livecd-rootfs source package in Focal: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Hey, is there some transparency into an actual fix here? https://cloud- images.ubuntu.com/focal/20201210/focal-server-cloudimg-amd64.img still has this issue. I still cannot use the cloudimg with libvirt / qemu without modification to workaround this issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in cloud-images: Confirmed Status in grub2 package in Ubuntu: Won't Fix Status in linux-azure package in Ubuntu: Invalid Status in livecd-rootfs package in Ubuntu: Triaged Status in grub2 source package in Focal: Confirmed Status in linux-azure source package in Focal: Invalid Status in livecd-rootfs source package in Focal: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Awesome, thanks for categorizing that for me @vorlon! I was trying hard to find the right home for the issue, but didn't even think of livecd- rootfs :) I still am not too sure about the ultimate value offered by the initramfs fallback. It seems like you would: (a) never want initramfs, or (b) always want initramfs. Whereas the initrdfail mechanism isn't either of those, and it introduces complexity and confusion and potentially masks the actual issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Won't Fix Status in linux-azure package in Ubuntu: Confirmed Status in livecd-rootfs package in Ubuntu: Triaged Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Hi! Has any information surfaced about the cloudimg kernel and virtio- scsi? After thinking about this for some time, I propose that the "initrdfail" design isn't ultimately helpful and should be reverted, because: (1) It substantially increases the complexity of grub.cfg, (2) it is not clear that any value is being added (but maybe I am just not aware of the scenarios where it is valuable), and (3) it has the effect of masking legitimate problems (like the ones being discussed in this ticket). More on that here: The actual design intention is to have a kernel that boots correctly on the first try. The cloud images, in fact, should be shipping to boot correctly on the first try. These fail/success/fail/success/etc. boot sequences experienced by myself and @decui are nonsensical and don't really contribute value. A fail/success/fail/success/etc. is not an acceptable workaround, because it breaks things, and it's also just hiding the actual problem: that the kernel hasn't shipped with the right drivers, or the wrong kernel was bundled with an image. These are two definite disadvantages of the "initrdfail" design as it is currently. I'm open to hearing feedback. Thanks, David -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Won't Fix Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
> These images should likewise be bootable under libvirt with, at most, a boot performance penalty because of the second boot. Well, yes, it does boot on the second boot. Not trying to be contrarian here, but isn't a moot point that booting on the first boot is the de facto way things should work? I.e., a panic/reboot shouldn't be considered the "normal" way to boot on libvirt, or anywhere, for that matter. Furthermore, the panic/reboot breaks the virt-install cloud-init workflow, which expects the first boot to succeed. I mean, that's a solid expectation, right? :) What is the cloudimg designed to boot with, if not libvirt? I think virtio-scsi is being used. Here's the virt-install line, if it helps, virt-install \ -n myvm \ --description '' \ --os-type Linux \ --os-variant ubuntu16.04 \ --disk path=/var/lib/libvirt/images/focal-server-cloudimg-amd64.img,bus=scsi,discard=unmap \ --controller type=scsi,model=virtio-scsi \ --cdrom /var/lib/libvirt/images/nocloud.iso \ --virt-type kvm \ --graphics none \ --ram 1024 \ --vcpus 1 \ --network bridge:guestbr,model=virtio,virtualport_type=openvswitch,target=myvm \ --boot hd Once I do (eventually) get the image to boot through arcane ways, then I can confirm it is loading virtio-scsi: $ lsmod | grep virt virtio_net 53248 0 virtio_scsi24576 2 Virtio-scsi seems like it needs to be compiled into the kernel for this to make sense. Am I missing something here? :) Thanks. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Won't Fix Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Thanks for the information @vorlon. I think that makes sense. What would be the solution for booting focal-server-cloudimg-amd64.img with libvirt (virt-install) (so, not on Azure, as the OP was; so, I think the issue is now fractured into two separate issues with same underlying cause)? Thanks. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Won't Fix Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Hey Dan, thanks for getting back. In my case, it's not even my system though: it's Canonical's system; It's a fresh download of a cloudimg from https://cloud-images.ubuntu.com/focal/ which has the issue, so it can't because I didn't build the image right, because I didn't build the image at all. So how would this be able to help? What is the rationale behind the new initrdfail design and 40-force-partuuid.cfg? Thanks. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
I don't know how to escalate attention to this issue. Is emailing the author of commit 6a814c759e10feafb40c3669be30aa51eb5ce39b, which seems to have introduced the issue, acceptable? I'm surprised it's not causing more issues. I guess people don't care about the cloud images much? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
I'm actually not sure the best place to file a ticket for the cloudimg distribution. Any recommendation appreciated. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Hmmm, I am not using the ISO, but rather the pre-installed cloudimg, and I am not so sure it is actually fixed. I just tried out the latest one, and can confirm the issue still persists with it: https://cloud-images.ubuntu.com/focal/20200430.1/focal-server-cloudimg- amd64.img Here is the output from booting: [0.00] Linux version 5.4.0-28-generic (buildd@lgw01-amd64-036) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #32-Ubuntu SMP Wed Apr 22 17:40:10 UTC 2020 (Ubuntu 5.4.0-28.32-generic 5.4.30) [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-28-generic root=PARTUUID=be1f3b17-864d-49f8-8249-5fb0edc7e606 ro console=tty1 console=ttyS0 panic=-1 ... [0.999771] VFS: Cannot open root device "PARTUUID=be1f3b17-864d-49f8-8249-5fb0edc7e606" or unknown-block(0,0): error -6 [1.001738] Please append a correct "root=" boot option; here are the available partitions: [1.003295] 0b00 1048575 sr0 [1.003295] driver: sr [1.004578] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) [1.006119] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.4.0-28-generic #32-Ubuntu [1.007522] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Ubuntu-1.8.2-1ubuntu1 04/01/2014 [1.009254] Call Trace: [1.009790] dump_stack+0x6d/0x9a [1.010488] panic+0x101/0x2e3 [1.011137] mount_block_root+0x23f/0x2e8 [1.011931] mount_root+0x38/0x3a [1.012624] prepare_namespace+0x13f/0x194 [1.013443] kernel_init_freeable+0x231/0x255 [1.014319] ? rest_init+0xb0/0xb0 [1.015018] kernel_init+0xe/0x100 [1.015721] ret_from_fork+0x35/0x40 [1.016842] Kernel Offset: 0x1ac0 from 0x8100 (relocation range: 0x8000-0xbfff) Upon reboot, initrdfail will now be set, and therefore it will actually load the initrd on the second reboot. But, because I am trying to configure the machine on the first boot using cloud-init NoCloud, it needs to actually work on the first boot. (Anyways, I think it's a moot point that first time booting should work.) In the above, the "panic=-1" followed by "Cannot open root device" indicates that there is still something amiss, at least with the cloudimg distribution. I can file a different ticket for cloudimg, if it would be more appropriate. Please advise. Thanks! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
The git commit which introduced the issue seems to be this: commit 6a814c759e10feafb40c3669be30aa51eb5ce39b Author: Mathieu Trudel-Lapierre Date: Tue Jul 16 11:31:29 2019 -0400 Import patches-unapplied version 2.04-1ubuntu1 to ubuntu/eoan-proposed Imported using git-ubuntu import. (at https://git.launchpad.net/ubuntu/+source/grub2/) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Also, this bug is not specific to just Azure. I am using libvirt and experiencing this issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
I've traced the issue back to "00_header" and "10_linux" here: https://git.launchpad.net/ubuntu/+source/grub2/tree/debian/patches /ubuntu-add-initrd-less-boot-fallback.patch?h=ubuntu/focal diffstat debian/patches/ubuntu-add-initrd-less-boot-fallback.patch Makefile.am |3 ++ configure.ac | 10 +++ grub-initrd-fallback.service | 12 + util/grub.d/00_header.in | 27 util/grub.d/10_linux.in | 56 +-- This is still an issue. I have to manually patch the cloud images with guestmount after downloading them in order to make them to work. Please advise. Thank you! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
Hi, this issue needs attention because it renders the cloud images for Ubuntu 20.04 potentially unusable. Thanks. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1870189] Re: initramfs does not get loaded
This bug may also be affecting focal-server-cloudimg builds at: https://cloud-images.ubuntu.com/focal/. See: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1874147 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded Status in grub2 package in Ubuntu: Confirmed Status in linux-azure package in Ubuntu: Confirmed Bug description: A Gen-1 Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d”. Then the latest Ubuntu v5.6 kernel was installed from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.6/. As soon as a reboot was performed, a panic with the v5.6 kernel occured because the rootfs can not be found. It turns out by default, initramfs does not get loaded: /boot/grub/grub.cfg: menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-3d2737e8- b95a-42bf-bac1-bb6fb4cda87f' { … if [ "${initrdfail}" = 1 ]; then linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic initrd/boot/initrd.img-5.6.0-050600-generic else linux /boot/vmlinuz-5.6.0-050600-generic root=PARTUUID=bc3d472f-401e-4774-affa-df1acba65a73 ro console=tty1 console=ttyS0 earlyprintk=ttyS0 ignore_loglevel sysrq_always_enabled unknown_nmi_panic panic=-1 #Dexuan: here the initrd line is missing! fi initrdfail } As we can see, Ubuntu only uses the initrd.img if initrdfail=1. Normally, initrdfail = 0, so when we boot the v5.6 kernel for the first time, we must hit the “fail to mount rootfs” panic and the kernel will automatically reboot…. Also, the “initrdfail” here marks initrdfail=1, so when the kernel boots for the 2nd time, the kernel should successfully boot up. Next, when the kernel boots for the 3rd time, it panics again since the userspace program resets initrdfail to 0, and next time when the kernel boots, it can boot up successfully -- this “panic/success/panic/success” pattern repeats forever… The linux-azure kernels are not affected since they have the vmbus driver and storage drivers built-in (i.e. “=y”): /boot/config-5.3.0-1013-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.3.0-1013-azure:CONFIG_HYPERV=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV_STORAGE=y /boot/config-5.4.0-1006-azure:CONFIG_HYPERV=y /boot/config-5.6.0-050600-generic:CONFIG_HYPERV_STORAGE=m /boot/config-5.6.0-050600-generic:CONFIG_HYPERV=m The v5.6 kernel uses =m rather than =y, so is affected here. It looks the setting may be intentional, but we should not assume a customer kernel must have the necessary vmbus/storage drivers built-in. This issue only happens to the Ubuntu Marketplace image (19.10 and maybe 19.04 as well?) on Azure. We installed a Ubuntu 20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-live/pending/ and don’t see the strange grub issue. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1548985] Re: After linux kernel update system won't boot
*** Update I was able to reproduce this issue in a fresh provisioning from the Kubuntu 15.10 ISO, although the boot log now stops at [OK] Started Accounts Service (See screen shot). Steps to reproduce: 1) Create a new VM (VMWare Fusion 8.1.0, although I doubt this matters) 4 cpu cores, 8GB Ram, 60GB disk 2) Install Kubuntu 15.10 from ISO (x64) 3) Install open-vm-tools sudo apt-get install open-vm-tools sudo apt-get install open-vm-tools-desktop 5) Restart 6) Upgrade distribution sudo apt-get update sudo apt-get upgrade 7) Restart. Machine wont complete the boot process, resulting in a black screen ** Attachment added: "Screen Shot 2016-02-25 at 11.29.52 AM.png" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1548985/+attachment/4580500/+files/Screen%20Shot%202016-02-25%20at%2011.29.52%20AM.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1548985 Title: After linux kernel update system won't boot Status in linux package in Ubuntu: Incomplete Status in linux source package in Wily: Incomplete Bug description: Whenever I try updating packages listed bellow via Software Updater, Ubuntu gets stuck on purple screen after restart. No key combination seems to do anything. I can login to desktop only via recovery and then normal boot option. I'm using Ubuntu 15.10. linux-generic installed version 4.2.0.16.18 available version 4.2.0.30.33 linux-firmware Installed version: 1.149 Available version: 1.149.3 linux-headers-generic Installed version: 4.2.0.16.18 Available version: 4.2.0.30.33 linux-image-generic Installed version: 4.2.0.16.18 Available version: 4.2.0.30.33 linux-headers-4.2.0-30 Installed version: None Available version: 4.2.0-30.35 linux-image-extra-4.2.0-30-generic Installed version: None Available version: 4.2.0-30.35 linux-libc-dev Installed version: 4.2.0-16.19 Available version: 4.2.0-30.35 linux-headers-4.2.0-30-generic Installed version: None Available version: 4.2.0-30.35 linux-image-4.2.0-30-generic Installed version: None Available version: 4.2.0-30.35 I'm not sure if all of them seams to cause the problem. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1548985/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1548985] Re: After linux kernel update system won't boot
I get the same or very similar behaviour running Kubuntu 15.10 running under VMWare Fusion on a Mac. After doing a sudo apt-get dist-upgrade the vm fails to boot, getting stuck on the boot logo. Hitting escape right after the machine starts shows me the boot log, and it gets wedged on the line "starting terminate plymouth boot screen", indicating to me that perhaps it is video related. I have reproduced this twice by taking a copy of another VM and performing the same update. Once wedged, no combination of keys has any effect, and I'm unable to boot to a console because I don't get a grub menu on boot (it's in quiet mode by default?). The HD is not encrypted, but one home directory is encrypted with ecryptfs. Being a bit of a noob, I was unable to rollback to previous kernel version (not sure how to do it if I can't get to a console), but I was able to get my data from the volume and provision another VM. I'll try provisioning a fresh VM from 15.10 iso to see if it is easily reproduced, or if there is something particular about my install. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1548985 Title: After linux kernel update system won't boot Status in linux package in Ubuntu: Incomplete Status in linux source package in Wily: Incomplete Bug description: Whenever I try updating packages listed bellow via Software Updater, Ubuntu gets stuck on purple screen after restart. No key combination seems to do anything. I can login to desktop only via recovery and then normal boot option. I'm using Ubuntu 15.10. linux-generic installed version 4.2.0.16.18 available version 4.2.0.30.33 linux-firmware Installed version: 1.149 Available version: 1.149.3 linux-headers-generic Installed version: 4.2.0.16.18 Available version: 4.2.0.30.33 linux-image-generic Installed version: 4.2.0.16.18 Available version: 4.2.0.30.33 linux-headers-4.2.0-30 Installed version: None Available version: 4.2.0-30.35 linux-image-extra-4.2.0-30-generic Installed version: None Available version: 4.2.0-30.35 linux-libc-dev Installed version: 4.2.0-16.19 Available version: 4.2.0-30.35 linux-headers-4.2.0-30-generic Installed version: None Available version: 4.2.0-30.35 linux-image-4.2.0-30-generic Installed version: None Available version: 4.2.0-30.35 I'm not sure if all of them seams to cause the problem. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1548985/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp