Just happened for the second time in 46 hours (and it never happened before):
kernel: [167836.884337] INFO: task kcompactd0:63 blocked for more than 120 seconds. kernel: [167836.887341] Not tainted 4.15.0-128-generic #131-Ubuntu kernel: [167836.889880] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kernel: [167836.893754] kcompactd0 D 0 63 2 0x80000000 kernel: [167836.893760] Call Trace: kernel: [167836.894017] __schedule+0x24e/0x880 kernel: [167836.894031] schedule+0x2c/0x80 kernel: [167836.894034] io_schedule+0x16/0x40 kernel: [167836.894160] __lock_page+0xff/0x140 kernel: [167836.894197] ? page_cache_tree_insert+0xe0/0xe0 kernel: [167836.894246] migrate_pages+0x91f/0xb80 kernel: [167836.894269] ? __ClearPageMovable+0x10/0x10 kernel: [167836.894272] ? isolate_freepages_block+0x3b0/0x3b0 kernel: [167836.894276] compact_zone+0x681/0x950 kernel: [167836.894279] kcompactd_do_work+0xfe/0x2a0 kernel: [167836.894282] ? __switch_to_asm+0x35/0x70 kernel: [167836.894284] ? __switch_to_asm+0x41/0x70 kernel: [167836.894288] kcompactd+0x86/0x1c0 kernel: [167836.894292] ? kcompactd+0x86/0x1c0 kernel: [167836.894395] ? wait_woken+0x80/0x80 kernel: [167836.894445] kthread+0x121/0x140 kernel: [167836.894449] ? kcompactd_do_work+0x2a0/0x2a0 kernel: [167836.894452] ? kthread_create_worker_on_cpu+0x70/0x70 kernel: [167836.894454] ret_from_fork+0x35/0x40 Message appears every 2 minutes for 18 minutes (then I guess it stops reporting) ; after that the disk are blocked, then NFS fails, then the whole service I'm running fails. It happened twice with kernel 4.15.0-128-generic. I updated to 4.15.0-137-generic while service was down, let's see how long it will work. HWE kernel is the next step. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1891773 Title: kcompactd0 and btrfs-transaction keep deadlocking with each other Status in linux package in Ubuntu: Confirmed Status in postgresql-11 package in Ubuntu: Confirmed Bug description: Example 1: [346911.187920] INFO: task kcompactd0:53 blocked for more than 120 seconds. [346911.187938] Not tainted 4.15.0-112-generic #113-Ubuntu [346911.187951] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [346911.187968] kcompactd0 D 0 53 2 0x80000000 [346911.187969] Call Trace: [346911.187973] __schedule+0x24e/0x880 [346911.187986] ? btree_releasepage+0x42/0x50 [btrfs] [346911.187987] schedule+0x2c/0x80 [346911.187988] io_schedule+0x16/0x40 [346911.187989] __lock_page+0xff/0x140 [346911.187990] ? page_cache_tree_insert+0xe0/0xe0 [346911.187992] migrate_pages+0x91f/0xb80 [346911.187993] ? __ClearPageMovable+0x10/0x10 [346911.187994] ? isolate_freepages_block+0x3b0/0x3b0 [346911.187995] compact_zone+0x681/0x950 [346911.187995] kcompactd_do_work+0xfe/0x2a0 [346911.187996] ? __switch_to_asm+0x35/0x70 [346911.187997] ? __switch_to_asm+0x41/0x70 [346911.187998] kcompactd+0x86/0x1c0 [346911.187999] ? kcompactd+0x86/0x1c0 [346911.188001] ? wait_woken+0x80/0x80 [346911.188002] kthread+0x121/0x140 [346911.188003] ? kcompactd_do_work+0x2a0/0x2a0 [346911.188003] ? kthread_create_worker_on_cpu+0x70/0x70 [346911.188004] ret_from_fork+0x35/0x40 [346911.188015] INFO: task btrfs-transacti:858 blocked for more than 120 seconds. [346911.188031] Not tainted 4.15.0-112-generic #113-Ubuntu [346911.188043] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [346911.188060] btrfs-transacti D 0 858 2 0x80000000 [346911.188061] Call Trace: [346911.188062] __schedule+0x24e/0x880 [346911.188063] ? bit_wait+0x60/0x60 [346911.188063] schedule+0x2c/0x80 [346911.188064] io_schedule+0x16/0x40 [346911.188065] bit_wait_io+0x11/0x60 [346911.188066] __wait_on_bit+0x4c/0x90 [346911.188066] ? bit_wait+0x60/0x60 [346911.188067] out_of_line_wait_on_bit+0x90/0xb0 [346911.188068] ? bit_waitqueue+0x40/0x40 [346911.188080] lock_extent_buffer_for_io+0x100/0x2a0 [btrfs] [346911.188090] btree_write_cache_pages+0x1b8/0x420 [btrfs] [346911.188092] ? native_sched_clock_from_tsc+0x30/0x70 [346911.188092] ? update_load_avg+0x423/0x780 [346911.188101] btree_writepages+0x5d/0x70 [btrfs] [346911.188102] do_writepages+0x4b/0xe0 [346911.188102] ? enqueue_task_fair+0xb6/0x300 [346911.188111] ? merge_state.part.47+0x44/0x130 [btrfs] [346911.188112] __filemap_fdatawrite_range+0xcf/0x100 [346911.188113] ? __filemap_fdatawrite_range+0xcf/0x100 [346911.188114] filemap_fdatawrite_range+0x13/0x20 [346911.188122] btrfs_write_marked_extents+0x68/0x140 [btrfs] [346911.188129] btrfs_write_and_wait_marked_extents.constprop.20+0x4f/0x90 [btrfs] [346911.188136] btrfs_commit_transaction+0x696/0x910 [btrfs] [346911.188143] ? btrfs_commit_transaction+0x696/0x910 [btrfs] [346911.188144] ? wait_woken+0x80/0x80 [346911.188151] transaction_kthread+0x18d/0x1b0 [btrfs] [346911.188152] kthread+0x121/0x140 [346911.188158] ? btrfs_cleanup_transaction+0x570/0x570 [btrfs] [346911.188159] ? kthread_create_worker_on_cpu+0x70/0x70 [346911.188160] ret_from_fork+0x35/0x40 Example 2: 2020-08-11T20:23:03.533+01:00 <err> kernel: INFO: task kcompactd0:53 blocked for more than 120 seconds. 2020-08-11T20:23:03.533+01:00 <err> kernel: Not tainted 4.15.0-112-generic #113-Ubuntu 2020-08-11T20:23:03.533+01:00 <err> kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 2020-08-11T20:23:03.533+01:00 <info> kernel: kcompactd0 D 0 53 2 0x80000000 2020-08-11T20:23:03.533+01:00 <warning> kernel: Call Trace: 2020-08-11T20:23:03.533+01:00 <warning> kernel: __schedule+0x24e/0x880 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? btree_releasepage+0x42/0x50 [btrfs] 2020-08-11T20:23:03.533+01:00 <warning> kernel: schedule+0x2c/0x80 2020-08-11T20:23:03.533+01:00 <warning> kernel: io_schedule+0x16/0x40 2020-08-11T20:23:03.533+01:00 <warning> kernel: __lock_page+0xff/0x140 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? page_cache_tree_insert+0xe0/0xe0 2020-08-11T20:23:03.533+01:00 <warning> kernel: migrate_pages+0x91f/0xb80 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? __ClearPageMovable+0x10/0x10 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? isolate_freepages_block+0x3b0/0x3b0 2020-08-11T20:23:03.533+01:00 <warning> kernel: compact_zone+0x681/0x950 2020-08-11T20:23:03.533+01:00 <warning> kernel: kcompactd_do_work+0xfe/0x2a0 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? __switch_to_asm+0x35/0x70 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? __switch_to_asm+0x41/0x70 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? __switch_to_asm+0x35/0x70 2020-08-11T20:23:03.533+01:00 <warning> kernel: kcompactd+0x86/0x1c0 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? kcompactd+0x86/0x1c0 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? wait_woken+0x80/0x80 2020-08-11T20:23:03.533+01:00 <warning> kernel: kthread+0x121/0x140 2020-08-11T20:23:03.533+01:00 <warning> kernel: ? kcompactd_do_work+0x2a0/0x2a0 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? kthread_create_worker_on_cpu+0x70/0x70 2020-08-11T20:23:03.534+01:00 <warning> kernel: ret_from_fork+0x35/0x40 2020-08-11T20:23:03.534+01:00 <err> kernel: INFO: task btrfs-transacti:864 blocked for more than 120 seconds. 2020-08-11T20:23:03.534+01:00 <err> kernel: Not tainted 4.15.0-112-generic #113-Ubuntu 2020-08-11T20:23:03.534+01:00 <err> kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 2020-08-11T20:23:03.534+01:00 <info> kernel: btrfs-transacti D 0 864 2 0x80000000 2020-08-11T20:23:03.534+01:00 <warning> kernel: Call Trace: 2020-08-11T20:23:03.534+01:00 <warning> kernel: __schedule+0x24e/0x880 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? bit_wait+0x60/0x60 2020-08-11T20:23:03.534+01:00 <warning> kernel: schedule+0x2c/0x80 2020-08-11T20:23:03.534+01:00 <warning> kernel: io_schedule+0x16/0x40 2020-08-11T20:23:03.534+01:00 <warning> kernel: bit_wait_io+0x11/0x60 2020-08-11T20:23:03.534+01:00 <warning> kernel: __wait_on_bit+0x4c/0x90 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? bit_wait+0x60/0x60 2020-08-11T20:23:03.534+01:00 <warning> kernel: out_of_line_wait_on_bit+0x90/0xb0 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? bit_waitqueue+0x40/0x40 2020-08-11T20:23:03.534+01:00 <warning> kernel: lock_extent_buffer_for_io+0x100/0x2a0 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btree_write_cache_pages+0x1b8/0x420 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btree_writepages+0x5d/0x70 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: do_writepages+0x4b/0xe0 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? enqueue_task_fair+0xb6/0x300 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? merge_state.part.47+0x44/0x130 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: __filemap_fdatawrite_range+0xcf/0x100 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? __filemap_fdatawrite_range+0xcf/0x100 2020-08-11T20:23:03.534+01:00 <warning> kernel: filemap_fdatawrite_range+0x13/0x20 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_write_marked_extents+0x68/0x140 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_write_and_wait_marked_extents.constprop.20+0x4f/0x90 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_commit_transaction+0x696/0x910 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? btrfs_commit_transaction+0x696/0x910 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? start_transaction+0x191/0x430 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: transaction_kthread+0x18d/0x1b0 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: kthread+0x121/0x140 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? btrfs_cleanup_transaction+0x570/0x570 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? kthread_create_worker_on_cpu+0x70/0x70 2020-08-11T20:23:03.534+01:00 <warning> kernel: ret_from_fork+0x35/0x40 2020-08-11T20:23:03.534+01:00 <err> kernel: INFO: task snapperd:5196 blocked for more than 120 seconds. 2020-08-11T20:23:03.534+01:00 <err> kernel: Not tainted 4.15.0-112-generic #113-Ubuntu 2020-08-11T20:23:03.534+01:00 <err> kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 2020-08-11T20:23:03.534+01:00 <info> kernel: snapperd D 0 5196 1 0x00000000 2020-08-11T20:23:03.534+01:00 <warning> kernel: Call Trace: 2020-08-11T20:23:03.534+01:00 <warning> kernel: __schedule+0x24e/0x880 2020-08-11T20:23:03.534+01:00 <warning> kernel: schedule+0x2c/0x80 2020-08-11T20:23:03.534+01:00 <warning> kernel: wait_for_commit+0x46/0x90 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? wait_woken+0x80/0x80 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_commit_transaction+0x287/0x910 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? start_transaction+0x191/0x430 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_mksubvol+0x592/0x5a0 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_ioctl_snap_create_transid+0x18c/0x1a0 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_ioctl_snap_create_v2+0x11e/0x180 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: btrfs_ioctl+0x84f/0x2490 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? do_filp_open+0xaf/0x110 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? _copy_to_user+0x26/0x30 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? cp_new_stat+0x152/0x180 2020-08-11T20:23:03.534+01:00 <warning> kernel: do_vfs_ioctl+0xa8/0x630 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? btrfs_ioctl_get_supported_features+0x30/0x30 [btrfs] 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? do_vfs_ioctl+0xa8/0x630 2020-08-11T20:23:03.534+01:00 <warning> kernel: ? __fput+0x193/0x220 2020-08-11T20:23:03.534+01:00 <warning> kernel: SyS_ioctl+0x79/0x90 2020-08-11T20:23:03.534+01:00 <warning> kernel: do_syscall_64+0x73/0x130 2020-08-11T20:23:03.534+01:00 <warning> kernel: entry_SYSCALL_64_after_hwframe+0x41/0xa6 2020-08-11T20:23:03.534+01:00 <warning> kernel: RIP: 0033:0x7f6dec22d6d7 2020-08-11T20:23:03.534+01:00 <warning> kernel: RSP: 002b:00007f6de6c782c8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 2020-08-11T20:23:03.534+01:00 <warning> kernel: RAX: ffffffffffffffda RBX: 00007f6de6c782d0 RCX: 00007f6dec22d6d7 2020-08-11T20:23:03.534+01:00 <warning> kernel: RDX: 00007f6de6c782d0 RSI: 0000000050009417 RDI: 0000000000000007 2020-08-11T20:23:03.534+01:00 <warning> kernel: RBP: 0000000000000006 R08: 000000000000000f R09: 00007f6de6c79f83 2020-08-11T20:23:03.534+01:00 <warning> kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000007 2020-08-11T20:23:03.534+01:00 <warning> kernel: R13: 00007f6de6c7a450 R14: 0000000000000000 R15: 0000000000000001 ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: linux-image-generic 4.15.0.112.100 ProcVersionSignature: Ubuntu 4.15.0-112.113-generic 4.15.18 Uname: Linux 4.15.0-112-generic x86_64 AlsaVersion: Advanced Linux Sound Architecture Driver Version k4.15.0-112-generic. AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay' ApportVersion: 2.20.9-0ubuntu7.16 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/hwC0D2', '/dev/snd/hwC0D0', '/dev/snd/pcmC0D10p', '/dev/snd/pcmC0D9p', '/dev/snd/pcmC0D8p', '/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/pcmC0D2c', '/dev/snd/pcmC0D0c', '/dev/snd/by-path', '/dev/snd/pcmC0D0p', '/dev/snd/controlC0', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer': 'amixer' Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer': 'amixer' Date: Sat Aug 15 22:39:32 2020 InstallationDate: Installed on 2012-02-01 (3117 days ago) InstallationMedia: Ubuntu-Server 11.10 "Oneiric Ocelot" - Release amd64 (20111011) Lsusb: Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 002: ID 05e3:0608 Genesys Logic, Inc. Hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub MachineType: Gigabyte Technology Co., Ltd. Q370M D3H GSM PLUS ProcFB: 0 inteldrmfb ProcKernelCmdLine: BOOT_IMAGE=/@/boot/vmlinuz-4.15.0-112-generic root=/dev/mapper/lvm_loko0-root ro rootflags=subvol=@ sysrq_always_enabled panic=60 RelatedPackageVersions: linux-restricted-modules-4.15.0-112-generic N/A linux-backports-modules-4.15.0-112-generic N/A linux-firmware 1.173.19 RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill' SourcePackage: linux UpgradeStatus: Upgraded to bionic on 2020-08-07 (8 days ago) dmi.bios.date: 05/11/2020 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: F15c C2 dmi.board.asset.tag: Default string dmi.board.name: Q370M D3H GSM PLUS dmi.board.vendor: Gigabyte Technology Co., Ltd. dmi.board.version: x.x dmi.chassis.asset.tag: Default string dmi.chassis.type: 3 dmi.chassis.vendor: Default string dmi.chassis.version: Default string dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrF15cC2:bd05/11/2020:svnGigabyteTechnologyCo.,Ltd.:pnQ370MD3HGSMPLUS:pvrDefaultstring:rvnGigabyteTechnologyCo.,Ltd.:rnQ370MD3HGSMPLUS:rvrx.x:cvnDefaultstring:ct3:cvrDefaultstring: dmi.product.family: Default string dmi.product.name: Q370M D3H GSM PLUS dmi.product.version: Default string dmi.sys.vendor: Gigabyte Technology Co., Ltd. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1891773/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
