Detailed steps to repro the issueo on Azure: 1. Create a VM with the image "Ubuntu Server 20.04 LTS - Gen1". Any VM size should be fine. Here I use "Standard E4-2ds_v4 (2 vcpus, 32 GiB memory)".
2. Add an extra disk of 64GB to the VM via Azure portal. 3. Login the VM via ssh and check the kernel version: here I get 5.4.0-1022-azure. 4. In the VM, the 64GB disk can be sdc. Let's create a swap partition in it, i.e. sdc1. 5. mkswap /dev/sdc1 root@decui-tmp-2004:~# mkswap /dev/sdc1 Setting up swapspace version 1, size = 64 GiB (68718424064 bytes) no label, UUID=544831e4-72ab-4d2c-81aa-6dac3a8e20ad 6. Add the swap partition info into /etc/fstab: UUID=544831e4-72ab-4d2c-81aa-6dac3a8e20ad none swap sw 0 0 7. Use "swapon -a; swapon -s" to confirm that the swap partition works. 8. Add the kernel parameter resume= into /etc/default/grub.d/50-cloudimg-settings.cfg: GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0 earlyprintk=ttyS0 resume=UUID=544831e4-72ab-4d2c-81aa-6dac3a8e20ad ignore_loglevel no_console_suspend" Note: here I also add "ignore_loglevel no_console_suspend", which are *required* to see the error messages during hibernation. 9. Comment out the only line in /etc/default/grub.d/40-force-partuuid.cfg: ####GRUB_FORCE_PARTUUID=bf00dea3-136e-49cb-a640-0df7ce49d6db Note: this step is required, otherwise the generated grub.cfg doesn't contain the "initrd ..." line , which is required for resuming to work. 10. Run "update-grub2; reboot". Note: this 'reboot' might be a must, because we'll need to re-generate the initramfs when the running kernel has the resume= parameter. 11. Login the VM again and run "update-initramfs -u". 12. Run "echo disk > /sys/power/state". Note: we'd better run this command from Azure serial console (we need to set a password for root and use that to login via the serial console) so we can easily watch what will be happening. root@decui-tmp-2004:~# echo disk > /sys/power/state [ 67.838749] PM: hibernation entry [ 68.266627] Filesystems sync: 0.041 seconds [ 68.271740] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 68.281528] OOM killer disabled. [ 68.286475] PM: Marking nosave pages: [mem 0x00000000-0x00000fff] [ 68.293459] PM: Marking nosave pages: [mem 0x0009f000-0x000fffff] [ 68.300306] PM: Marking nosave pages: [mem 0x3fff0000-0xffffffff] [ 68.308250] PM: Basic memory bitmaps created [ 68.313082] PM: Preallocating image memory... done (allocated 298659 pages) [ 69.303864] PM: Allocated 1194636 kbytes in 0.98 seconds (1219.01 MB/s) [ 69.311605] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 69.322486] serial 00:04: disabled [ 69.345193] ------------[ cut here ]------------ [ 69.345199] WARNING: CPU: 1 PID: 1495 at kernel/workqueue.c:3040 __flush_work+0x1b5/0x1d0 ... [ 70.047238] CPU1 is up [ 70.054474] hv_utils: KVP IC version 4.0 [ 70.056763] hv_utils: Shutdown IC version 3.2 [ 70.061009] hv_balloon: Using Dynamic Memory protocol version 2.0 It looks the kernel hangs here forever. Normally the VM is expected to save the state to disk and power off and later when we start the VM from the portal, the VM is expected to resume back from the 'echo' command on the serial console. If I build a kernel with the same source code but revert 0a14dbaa0736, the above suspending and resuming work fine. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux-azure in Ubuntu. https://bugs.launchpad.net/bugs/1880032 Title: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels Status in linux-azure package in Ubuntu: Fix Released Status in linux-azure source package in Focal: Fix Released Bug description: Microsoft would like to request commits to enable VM hibernation in the Azure 5.4 kernels for 18.04 and 20.04. Some of the commits needed to enable VM hibernation were included in mainline 5.4 and older. However, 24 commits were added in 5.5 and later, which are required in the 5.4 kernel. The list of commits requested are: 38dce4195f0d x86/hyperv: Properly suspend/resume reenlightenment notifications 2351f8d295ed PM: hibernate: Freeze kernel threads in software_resume() 421f090c819d x86/hyperv: Suspend/resume the VP assist page for hibernation 1a06d017fb3f Drivers: hv: vmbus: Fix Suspend-to-Idle for Generation-2 VM 3704a6a44579 PM: hibernate: Propagate the return value of hibernation_restore() 54e19d34011f hv_utils: Add the support of hibernation ffd1d4a49336 hv_utils: Support host-initiated hibernation request 3e9c72056ed5 hv_utils: Support host-initiated restart request 9fc3c01a1fae6 Tools: hv: Reopen the devices if read() or write() returns 05bd330a7fd8 x86/hyperv: Suspend/resume the hypercall page for hibernation 382a46221757 video: hyperv_fb: Fix hibernation for the deferred IO feature e2379b30324c Input: hyperv-keyboard: Add the support of hibernation ac82fc8327088 PCI: hv: Add hibernation support a8e37506e79a PCI: hv: Reorganize the code in preparation of hibernation 1349401ff1aa4 clocksource/drivers/hyper-v: Suspend/resume Hyper-V clocksource for hibernation af13f9ed6f9a HID: hyperv: Add the support of hibernation 25bd2b2f1f053 hv_balloon: Add the support of hibernation b96f86534fa31 x86/hyperv: Implement hv_is_hibernation_supported() 4df4cb9e99f83 x86/hyperv: Initialize clockevents earlier in CPU onlining 0efeea5fb1535 hv_netvsc: Add the support of hibernation 2194c2eb6717f hv_sock: Add the support of hibernation 1ecf302021040 video: hyperv_fb: Add the support of hibernation 56fb105859345 scsi: storvsc: Add the support of hibernation f2c33ccacb2d4 PCI/PM: Always return devices to D0 when thawing To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp