Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Hi, On Fri, May 3, 2019 at 2:14 PM Salvatore Bonaccorso wrote: > Not yet decided, and might need imput from stable release managers as > well, but the issue might be worth fixing before a next point release > already via stretch-updates and a SUA. Agreed. > I have already cherry-picked the commit in the stretch branch in the > kernel-team packaging repo at [1]. Thanks!
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Hi Romain, On Thu, May 02, 2019 at 04:55:25PM +, Romain Francoise wrote: > v4.9.172 is out with the offending commit reverted. Is there a stretch > update with the same revert planned soon to address this? And via which > suite? Not yet decided, and might need imput from stable release managers as well, but the issue might be worth fixing before a next point release already via stretch-updates and a SUA. I have already cherry-picked the commit in the stretch branch in the kernel-team packaging repo at [1]. Regards, Salvatore [1] https://salsa.debian.org/kernel-team/linux/commit/5f66def673547b7d1e7a841937fbd0ab10a925a0
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Hi, v4.9.172 is out with the offending commit reverted. Is there a stretch update with the same revert planned soon to address this? And via which suite? Thanks.
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Control: tags -1 + confirmed upstream Control: forwarded -1 https://bugzilla.kernel.org/show_bug.cgi?id=202985 Hi, Some further updates. On Sun, Apr 28, 2019 at 10:58:21PM +0200, Salvatore Bonaccorso wrote: > Hi Michael, Brad, > > On Sun, Apr 28, 2019 at 08:50:38PM +0200, Michael Biebl wrote: > > Control: reassign -1 linux-image-4.9.0-9-amd64 > > Control: severity -1 important > > > > Am 28.04.19 um 20:32 schrieb Brad Barnett: > > > > > > Ack, should have thought of that. > > > > > > Just tested. OK kernel: > > > > > > 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3 (2019-02-02) x86_64 GNU/Linux > > > > > > Borked kernel: > > > > > > 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1 (2019-04-12) x86_64 > > > > > > Hmm. Just noticed GNU/Linux tag is missing too.. > > > > > > Thanks for checking, Reassigning to the linux package and bumping > > severity a bit. Seems like something that should be fixed in stable. > > And looks already present in 4.9.161-1 which was an intermediate > upload to stretch-proposed-updates before 4.9.168-1. As well > reproducible with the current WIP for 4.9.171-1 as per [1]. Can confirm as well the issue with upstream 4.9.171 directly. Bisecting showed that it's introduced by 3ae3d167f5ec2c7bb5fcd12b7772cfadc93b2305 (v4.9.152~9) and it relates to this thread: https://lore.kernel.org/stable/20190320125806.gd9...@quack2.suse.cz/ https://bugzilla.kernel.org/show_bug.cgi?id=202985 Regards, Salvatore
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Hi Michael, Brad, On Sun, Apr 28, 2019 at 08:50:38PM +0200, Michael Biebl wrote: > Control: reassign -1 linux-image-4.9.0-9-amd64 > Control: severity -1 important > > Am 28.04.19 um 20:32 schrieb Brad Barnett: > > > > Ack, should have thought of that. > > > > Just tested. OK kernel: > > > > 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3 (2019-02-02) x86_64 GNU/Linux > > > > Borked kernel: > > > > 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1 (2019-04-12) x86_64 > > > > Hmm. Just noticed GNU/Linux tag is missing too.. > > > Thanks for checking, Reassigning to the linux package and bumping > severity a bit. Seems like something that should be fixed in stable. And looks already present in 4.9.161-1 which was an intermediate upload to stretch-proposed-updates before 4.9.168-1. As well reproducible with the current WIP for 4.9.171-1 as per [1]. [1] https://salsa.debian.org/kernel-team/linux/merge_requests/141 Regards, Salvatore
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Control: reassign -1 linux-image-4.9.0-9-amd64 Control: severity -1 important Am 28.04.19 um 20:32 schrieb Brad Barnett: > > Ack, should have thought of that. > > Just tested. OK kernel: > > 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3 (2019-02-02) x86_64 GNU/Linux > > Borked kernel: > > 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1 (2019-04-12) x86_64 > > Hmm. Just noticed GNU/Linux tag is missing too.. Thanks for checking, Reassigning to the linux package and bumping severity a bit. Seems like something that should be fixed in stable. > > > On Sun, 28 Apr 2019 19:41:47 +0200 > Michael Biebl wrote: > >> This looks like a kernel regression. >> Do you use a Debian kernel? From which version did you upgrade? >> If you downgrade to the previous kernel version, does the problem go >> away. > > ___ > Pkg-systemd-maintainers mailing list > pkg-systemd-maintain...@alioth-lists.debian.net > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-systemd-maintainers > -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Ack, should have thought of that. Just tested. OK kernel: 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3 (2019-02-02) x86_64 GNU/Linux Borked kernel: 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1 (2019-04-12) x86_64 Hmm. Just noticed GNU/Linux tag is missing too.. On Sun, 28 Apr 2019 19:41:47 +0200 Michael Biebl wrote: > This looks like a kernel regression. > Do you use a Debian kernel? From which version did you upgrade? > If you downgrade to the previous kernel version, does the problem go > away.
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Am 28.04.19 um 19:41 schrieb Michael Biebl: > Do you use a Debian kernel? From which version did you upgrade? > If you downgrade to the previous kernel version, does the problem go away. Btw, please use reportbug next time, so we have this information readily available and don't need to spend time asking for it. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Am 28.04.19 um 19:41 schrieb Michael Biebl: > This looks like a kernel regression. > Do you use a Debian kernel? From which version did you upgrade? > If you downgrade to the previous kernel version, does the problem go away. From https://tracker.debian.org/news/1038093/accepted-linux-49168-1-source-into-proposed-updates-stable-new-proposed-updates/ https://www.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.9.162 - Revert "loop: Fix double mutex_unlock(_ctl_mutex) in loop_control_ioctl()" - Revert "loop: Get rid of loop_index_mutex" - Revert "loop: Fold __loop_release into loop_release" That looks very suspicious. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Am 28.04.19 um 18:00 schrieb Brad Barnett: > > Package: udev > Version: 232-25+deb9u11 > > After today's upgrade to debian 9.9, I attempted to create a disk image. > > I have a script, it does the following: > > ### > dd if=/dev/zero of=/tmp/ff1.raw bs=1G seek=8 count=0 > sync > sleep 1 > parted /tmp/ff1.raw mklabel msdos > parted -s /tmp/ff1.raw mkpart primary linux-swap 1 100 > parted -s -- /tmp/ff1.raw mkpart primary ext2 101 -1 > parted -s -- /tmp/ff1.raw set 2 boot on > sleep 5 > losetup -Pf /tmp/ff1.raw --show > ### > > The above has been run several times per month, for years. > > After today's upgrade I rebooted, ran the above, and it locked on losetup. > > I see this: > > # ps auxwwf > > root 346 0.0 0.1 45656 3420 ?Ss 11:30 0:00 > /lib/systemd/systemd-udevd --daemon > root 2596 0.0 0.0 45564 2120 ?S11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2597 0.0 0.0 45564 2184 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2599 0.0 0.1 45644 3096 ?S11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2602 0.0 0.0 45564 2120 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2603 0.0 0.0 45656 1936 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2607 0.0 0.0 45656 1936 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2609 0.0 0.0 45564 2120 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2610 0.0 0.0 45656 1936 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > root 2612 0.0 0.0 45656 1936 ?D11:31 0:00 \_ > /lib/systemd/systemd-udevd --daemon > > And > > root 2591 0.0 0.0 8064 780 pts/2D11:31 0:00 > /sbin/losetup -Pf /tmp/ff1.raw --show > > I also see this (last 3 'blocks' pasted): > > Apr 28 11:34:17 buildvm kernel: [ 242.831744] INFO: task systemd-udevd:2612 > blocked for more than 120 seconds. > Apr 28 11:34:17 buildvm kernel: [ 242.831807] Not tainted > 4.9.0-9-amd64 #1 Debian 4.9.168-1 > Apr 28 11:34:17 buildvm kernel: [ 242.831859] "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > Apr 28 11:34:17 buildvm kernel: [ 242.831927] systemd-udevd D0 2612 > 346 0x0004 > Apr 28 11:34:17 buildvm kernel: [ 242.831932] 8a2dbb10c400 > 8a2dbacc5240 8a2dbfa18980 > Apr 28 11:34:17 buildvm kernel: [ 242.831939] af411500 > af84409e7b58 aee15a99 1e580f2a959617cf > Apr 28 11:34:17 buildvm kernel: [ 242.831946] 000402c0 > 8a2dbfa18980 8a2dbacc5240 > > Apr 28 11:34:17 buildvm kernel: [ 242.831952] Call Trace: > Apr 28 11:34:17 buildvm kernel: [ 242.831960] [] ? > __schedule+0x239/0x6f0 > Apr 28 11:34:17 buildvm kernel: [ 242.831967] [] ? > schedule+0x32/0x80 > Apr 28 11:34:17 buildvm kernel: [ 242.831973] [] ? > schedule_preempt_disabled+0xa/0x10 > Apr 28 11:34:17 buildvm kernel: [ 242.831977] [] ? > __mutex_lock_slowpath+0xb4/0x130 > Apr 28 11:34:17 buildvm kernel: [ 242.831982] [] ? > mutex_lock+0x1b/0x30 > Apr 28 11:34:17 buildvm kernel: [ 242.831988] [] ? > lo_open+0x15/0x50 [loop] > Apr 28 11:34:17 buildvm kernel: [ 242.831994] [] ? > __blkdev_get+0xd3/0x470 > Apr 28 11:34:17 buildvm kernel: [ 242.832001] [] ? > blkdev_get+0x126/0x330 > Apr 28 11:34:17 buildvm kernel: [ 242.832008] [] ? > unlock_new_inode+0x44/0x70 > Apr 28 11:34:17 buildvm kernel: [ 242.832014] [] ? > bdget+0xfa/0x110 > Apr 28 11:34:17 buildvm kernel: [ 242.832020] [] ? > blkdev_get_by_dev+0x40/0x40 > Apr 28 11:34:17 buildvm kernel: [ 242.832026] [] ? > do_dentry_open+0x234/0x340 > Apr 28 11:34:17 buildvm kernel: [ 242.832030] [] ? > path_openat+0x777/0x15b0 > Apr 28 11:34:17 buildvm kernel: [ 242.832037] [] ? > page_add_file_rmap+0x11/0x110 > Apr 28 11:34:17 buildvm kernel: [ 242.832042] [] ? > do_filp_open+0x91/0x100 > Apr 28 11:34:17 buildvm kernel: [ 242.832047] [] ? > handle_mm_fault+0xcc3/0x1350 > Apr 28 11:34:17 buildvm kernel: [ 242.832052] [] ? > __check_object_size+0xfa/0x1d8 > Apr 28 11:34:17 buildvm kernel: [ 242.832058] [] ? > do_sys_open+0x12e/0x210 > Apr 28 11:34:17 buildvm kernel: [ 242.832064] [] ? > do_syscall_64+0x8d/0xf0 > Apr 28 11:34:17 buildvm kernel: [ 242.832069] [] ? > entry_SYSCALL_64_after_swapgs+0x58/0xc6 > Apr 28 11:36:18 buildvm kernel: [ 363.661408] INFO: task losetup:2591 > blocked for more than 120 seconds. > Apr 28 11:36:18 buildvm kernel: [ 363.661487] Not tainted > 4.9.0-9-amd64 #1 Debian 4.9.168-1 > Apr 28 11:36:18 buildvm kernel: [ 363.661541] "echo 0 > > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > Apr 28 11:36:18 buildvm kernel: [ 363.661610] losetup D0 2591 >1 0x0004 > Apr 28 11:36:18
Bug#928125: losetup causes Dead systemd-udevd processes, blocks forever
Package: udev Version: 232-25+deb9u11 After today's upgrade to debian 9.9, I attempted to create a disk image. I have a script, it does the following: ### dd if=/dev/zero of=/tmp/ff1.raw bs=1G seek=8 count=0 sync sleep 1 parted /tmp/ff1.raw mklabel msdos parted -s /tmp/ff1.raw mkpart primary linux-swap 1 100 parted -s -- /tmp/ff1.raw mkpart primary ext2 101 -1 parted -s -- /tmp/ff1.raw set 2 boot on sleep 5 losetup -Pf /tmp/ff1.raw --show ### The above has been run several times per month, for years. After today's upgrade I rebooted, ran the above, and it locked on losetup. I see this: # ps auxwwf root 346 0.0 0.1 45656 3420 ?Ss 11:30 0:00 /lib/systemd/systemd-udevd --daemon root 2596 0.0 0.0 45564 2120 ?S11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2597 0.0 0.0 45564 2184 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2599 0.0 0.1 45644 3096 ?S11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2602 0.0 0.0 45564 2120 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2603 0.0 0.0 45656 1936 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2607 0.0 0.0 45656 1936 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2609 0.0 0.0 45564 2120 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2610 0.0 0.0 45656 1936 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon root 2612 0.0 0.0 45656 1936 ?D11:31 0:00 \_ /lib/systemd/systemd-udevd --daemon And root 2591 0.0 0.0 8064 780 pts/2D11:31 0:00 /sbin/losetup -Pf /tmp/ff1.raw --show I also see this (last 3 'blocks' pasted): Apr 28 11:34:17 buildvm kernel: [ 242.831744] INFO: task systemd-udevd:2612 blocked for more than 120 seconds. Apr 28 11:34:17 buildvm kernel: [ 242.831807] Not tainted 4.9.0-9-amd64 #1 Debian 4.9.168-1 Apr 28 11:34:17 buildvm kernel: [ 242.831859] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 28 11:34:17 buildvm kernel: [ 242.831927] systemd-udevd D0 2612 346 0x0004 Apr 28 11:34:17 buildvm kernel: [ 242.831932] 8a2dbb10c400 8a2dbacc5240 8a2dbfa18980 Apr 28 11:34:17 buildvm kernel: [ 242.831939] af411500 af84409e7b58 aee15a99 1e580f2a959617cf Apr 28 11:34:17 buildvm kernel: [ 242.831946] 000402c0 8a2dbfa18980 8a2dbacc5240 Apr 28 11:34:17 buildvm kernel: [ 242.831952] Call Trace: Apr 28 11:34:17 buildvm kernel: [ 242.831960] [] ? __schedule+0x239/0x6f0 Apr 28 11:34:17 buildvm kernel: [ 242.831967] [] ? schedule+0x32/0x80 Apr 28 11:34:17 buildvm kernel: [ 242.831973] [] ? schedule_preempt_disabled+0xa/0x10 Apr 28 11:34:17 buildvm kernel: [ 242.831977] [] ? __mutex_lock_slowpath+0xb4/0x130 Apr 28 11:34:17 buildvm kernel: [ 242.831982] [] ? mutex_lock+0x1b/0x30 Apr 28 11:34:17 buildvm kernel: [ 242.831988] [] ? lo_open+0x15/0x50 [loop] Apr 28 11:34:17 buildvm kernel: [ 242.831994] [] ? __blkdev_get+0xd3/0x470 Apr 28 11:34:17 buildvm kernel: [ 242.832001] [] ? blkdev_get+0x126/0x330 Apr 28 11:34:17 buildvm kernel: [ 242.832008] [] ? unlock_new_inode+0x44/0x70 Apr 28 11:34:17 buildvm kernel: [ 242.832014] [] ? bdget+0xfa/0x110 Apr 28 11:34:17 buildvm kernel: [ 242.832020] [] ? blkdev_get_by_dev+0x40/0x40 Apr 28 11:34:17 buildvm kernel: [ 242.832026] [] ? do_dentry_open+0x234/0x340 Apr 28 11:34:17 buildvm kernel: [ 242.832030] [] ? path_openat+0x777/0x15b0 Apr 28 11:34:17 buildvm kernel: [ 242.832037] [] ? page_add_file_rmap+0x11/0x110 Apr 28 11:34:17 buildvm kernel: [ 242.832042] [] ? do_filp_open+0x91/0x100 Apr 28 11:34:17 buildvm kernel: [ 242.832047] [] ? handle_mm_fault+0xcc3/0x1350 Apr 28 11:34:17 buildvm kernel: [ 242.832052] [] ? __check_object_size+0xfa/0x1d8 Apr 28 11:34:17 buildvm kernel: [ 242.832058] [] ? do_sys_open+0x12e/0x210 Apr 28 11:34:17 buildvm kernel: [ 242.832064] [] ? do_syscall_64+0x8d/0xf0 Apr 28 11:34:17 buildvm kernel: [ 242.832069] [] ? entry_SYSCALL_64_after_swapgs+0x58/0xc6 Apr 28 11:36:18 buildvm kernel: [ 363.661408] INFO: task losetup:2591 blocked for more than 120 seconds. Apr 28 11:36:18 buildvm kernel: [ 363.661487] Not tainted 4.9.0-9-amd64 #1 Debian 4.9.168-1 Apr 28 11:36:18 buildvm kernel: [ 363.661541] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Apr 28 11:36:18 buildvm kernel: [ 363.661610] losetup D0 2591 1 0x0004 Apr 28 11:36:18 buildvm kernel: [ 363.661620] 8a2db7926c00 8a2db7956ec0 8a2dbfa18980 Apr 28 11:36:18 buildvm kernel: [ 363.661628] af411500 af8440a7fbd8 aee15a99 aee1a964 Apr 28 11:36:18 buildvm kernel: [ 363.661635] 00ff8a2db7957a00
