You have been subscribed to a public bug by Mauricio Faria de Oliveira (mfo):
[Impact] Azure VM instances hit I/O error on boot causing kernel crash [Test Plan] Create Ubuntu Marketplace VM on Azure ``` ssh -i .ssh/id_rsa ubuntu@ipaddr<azureVM> ``` Install crash dump utilities (from guide: https://ubuntu.com/server/docs/kernel-crash-dump) apt-get install kdump-tools Say (y) to all questions during install kdump-config show *shows the vm is not yet ready to kdump root@bionic3: kdump-config show * no crashkernel= parameter in the kernel cmdline DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: /var/lib/kdump/vmlinuz kdump initrd: /var/lib/kdump/initrd.img current state: Not ready to kdump kexec command: no kexec command recorded Reboot the VM sudo su As root on the VM after reboot: kdump-config show kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0x32000000 /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-5.4.0-1086-azure kdump initrd: /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-5.4.0-1086-azure current state: ready to kdump kexec command: /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.4.0-1086-azure root=UUID=143c811b-9b9c-48f3-b0c8-040f6e65f50a ro console=tty1 console=ttyS0 earlyprintk=ttyS0 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb ata_piix.prefer_ms_hyperv=0" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz #verify kdump is on cat /proc/cmdline ... crashkernel=512M-:192M dmesg | grep -i crash [ 0.071660] kexec: Reserving the low 1M of memory for crashkernel [ 0.269823] Reserving 192MB of memory at 640MB for crashkernel (System RAM: 4095MB) cat /proc/sys/kernel/sysrq # make sure this value is greater than 0 #set it to 1 sudo sysctl -w kernel.sysrq=1 the directory of /var/crash should have no crashes yet as well. Outcome with "ata_piix.prefer_ms_hyperv=0" in kexec command: # perform crash sudo su echo c > /proc/sysrq-trigger After a couple of minuties Open new terminal and try to ssh to azure VM, It does not succeed Force a reboot of the VM through the portal or serial console #kdump doesnt work and hangs indefinitely #force reboot VM from Azure console # verify package version of kdump-tool # verify parameter is not listed in the kdump-config show output kdump-config unload kdump-config load kdump-config show kdump-config show DUMP_MODE: kdump USE_KDUMP: 1 KDUMP_SYSCTL: kernel.panic_on_oops=1 KDUMP_COREDIR: /var/crash crashkernel addr: 0x32000000 /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-5.4.0-1086-azure kdump initrd: /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-5.4.0-1086-azure current state: ready to kdump kexec command: /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.4.0-1086-azure root=UUID=143c811b-9b9c-48f3-b0c8-040f6e65f50a ro console=tty1 console=ttyS0 earlyprintk=ttyS0 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz #trigger another crash echo c > /proc/sysrq-trigger # open new terminal and ssh back into vm cd /var/crash #verify a dump was created linux-image-5.4.0-1086-azure-202208041658.crash [Where Problems Could Occur] This change modifies the debian/rules. The package could fail to build properly if mistyped. [Other] Back-porting a fix from upstream to remove "ata_piix.prefer_ms_hyperv=0" parameter. target series - Bionic, Focal, Jammy upstream patch https://salsa.debian.org/debian/kdump- tools/-/commit/b1bac9396ddbbce3817c34be3161630698e4a503 *Note: There are two source packages needed changes, kdump-tools for Impish -> Kinetic and makedumpfile for series Focal -> Bionic ** Affects: kdump-tools (Ubuntu) Importance: Undecided Assignee: Heather Lemon (hypothetical-lemon) Status: Fix Committed ** Affects: makedumpfile (Ubuntu) Importance: Undecided Status: Invalid ** Affects: kdump-tools (Ubuntu Bionic) Importance: Undecided Status: Invalid ** Affects: makedumpfile (Ubuntu Bionic) Importance: Undecided Assignee: Heather Lemon (hypothetical-lemon) Status: Fix Committed ** Affects: kdump-tools (Ubuntu Focal) Importance: Undecided Status: Invalid ** Affects: makedumpfile (Ubuntu Focal) Importance: Undecided Assignee: Heather Lemon (hypothetical-lemon) Status: Fix Committed ** Affects: kdump-tools (Ubuntu Impish) Importance: Undecided Status: Invalid ** Affects: makedumpfile (Ubuntu Impish) Importance: Undecided Status: Invalid ** Affects: kdump-tools (Ubuntu Jammy) Importance: Undecided Assignee: Heather Lemon (hypothetical-lemon) Status: Fix Committed ** Affects: makedumpfile (Ubuntu Jammy) Importance: Undecided Status: Invalid ** Affects: kdump-tools (Ubuntu Kinetic) Importance: Undecided Assignee: Heather Lemon (hypothetical-lemon) Status: Fix Committed ** Affects: makedumpfile (Ubuntu Kinetic) Importance: Undecided Status: Invalid ** Tags: patch se-sponsor-halves sts verification-needed verification-needed-bionic verification-needed-focal verification-needed-jammy -- Remove "ata_piix.prefer_ms_hyperv=0" parameter https://bugs.launchpad.net/bugs/1978333 You received this bug notification because you are a member of SE SRU ("STS") Sponsors, which is subscribed to the bug report. -- Mailing list: https://launchpad.net/~sts-sponsors Post to : sts-sponsors@lists.launchpad.net Unsubscribe : https://launchpad.net/~sts-sponsors More help : https://help.launchpad.net/ListHelp
