[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
This bug fix causes a regression in Xenial when installing initramfs- tools in QEMU, since the version of QEMU in Xenial does not support syncfs(): /bin/sync: error syncing '/boot/initrd.img-4.14.44-v7ooicgsn': Function not implemented -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Released Status in initramfs-tools source package in Bionic: Fix Released Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-4.4.0-63-generic ) points to /boot/initrd.img-4.4.0-63-generic (/boot/initrd.img-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491.
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
This bug was fixed in the package initramfs-tools - 0.122ubuntu8.13 --- initramfs-tools (0.122ubuntu8.13) xenial; urgency=medium [ Colin Watson ] * [b4804dd] Only sync the filesystem containing the initramfs (Closes: #882380) (LP: #1667512) [ Tiago Stürmer Daitx ] * debian/initramfs-tools.postinst: remove orphaned old-dkms initrd files in /boot. (LP: #1791959) -- Tiago Stürmer Daitx Tue, 18 Sep 2018 09:18:47 + ** Changed in: initramfs-tools (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Released Status in initramfs-tools source package in Bionic: Fix Released Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod.
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
This bug was fixed in the package initramfs-tools - 0.130ubuntu3.5 --- initramfs-tools (0.130ubuntu3.5) bionic; urgency=medium * debian/initramfs-tools.postinst: fix version comparison and so it will run for any version earlier than 0.130ubuntu3.5. initramfs-tools (0.130ubuntu3.4) bionic; urgency=medium [ Colin Watson ] * [b4804dd] Only sync the filesystem containing the initramfs (Closes: #882380) (LP: #1667512) [ Tiago Stürmer Daitx ] * debian/initramfs-tools.postinst: remove orphaned old-dkms initrd files in /boot. (LP: #1791959) -- Tiago Stürmer Daitx Thu, 27 Sep 2018 16:21:37 + ** Changed in: initramfs-tools (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Committed Status in initramfs-tools source package in Bionic: Fix Released Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
Did the testcase verification again for the new 0.130ubuntu3.5 bionic- proposed upload and I confirm again that it works as expected. ** Tags removed: verification-needed verification-needed-bionic ** Tags added: verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Committed Status in initramfs-tools source package in Bionic: Fix Committed Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-4.4.0-63-generic ) points to /boot/initrd.img-4.4.0-63-generic (/boot/initrd.img-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491.
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
Hello tbenst, or anyone else affected, Accepted initramfs-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/initramfs- tools/0.130ubuntu3.5 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Tags removed: verification-done-bionic ** Tags added: verification-needed verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Committed Status in initramfs-tools source package in Bionic: Fix Committed Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated)
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
** Merge proposal linked: https://code.launchpad.net/~tdaitx/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+merge/355789 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Committed Status in initramfs-tools source package in Bionic: Fix Committed Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-4.4.0-63-generic ) points to /boot/initrd.img-4.4.0-63-generic (/boot/initrd.img-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491. vmlinuz(/boot/vmlinuz-4.4.0-63-generic ) points to /boot/vmlinuz-4.4.0-63-generic (/boot/vmlinuz-4.4.0-63-generic) -- doing
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
Verification done in Xenial and Bionic as per the test case. It works fine and will prevent update-initramfs from being stuck in the described scenarios. ** Tags removed: verification-needed-bionic verification-needed-xenial ** Tags added: verification-done-bionic verification-done-xenial ** Tags removed: verification-needed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Committed Status in initramfs-tools source package in Bionic: Fix Committed Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-4.4.0-63-generic ) points to /boot/initrd.img-4.4.0-63-generic (/boot/initrd.img-4.4.0-63-generic) -- doing nothing at
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
Hello tbenst, or anyone else affected, Accepted initramfs-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/initramfs- tools/0.130ubuntu3.4 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: initramfs-tools (Ubuntu Bionic) Status: New => Fix Committed ** Tags added: verification-needed verification-needed-bionic ** Changed in: initramfs-tools (Ubuntu Xenial) Status: New => Fix Committed ** Tags added: verification-needed-xenial -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: Fix Committed Status in initramfs-tools source package in Bionic: Fix Committed Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
** Also affects: initramfs-tools (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: initramfs-tools (Ubuntu Xenial) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools source package in Xenial: New Status in initramfs-tools source package in Bionic: New Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-4.4.0-63-generic ) points to /boot/initrd.img-4.4.0-63-generic (/boot/initrd.img-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491. vmlinuz(/boot/vmlinuz-4.4.0-63-generic ) points to /boot/vmlinuz-4.4.0-63-generic
[Touch-packages] [Bug 1667512] Re: [SRU] update-initramfs should sync only the filesystem containing the initramfs image
SRU as a merge proposal for both this and bug 1791959 available at Bionic: https://code.launchpad.net/~tdaitx/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+merge/355189 Xenial: https://code.launchpad.net/~tdaitx/ubuntu/+source/initramfs-tools/+git/initramfs-tools/+merge/355190 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to initramfs-tools in Ubuntu. https://bugs.launchpad.net/bugs/1667512 Title: [SRU] update-initramfs should sync only the filesystem containing the initramfs image Status in initramfs-tools package in Ubuntu: Fix Released Status in initramfs-tools package in Debian: Fix Released Bug description: [Impact] If there is lots of activity on a different filesystem, high io/cache pressure, or a stale mount updating initramfs might hang because the update-initramfs script calls sync. This leaves the system unresponsive and will require the user to forcefully kill dpkg. [Test Case] One easy way to simulate this is by copying a huge file over nfs. Note: do not use localhost, it is too fast for this purpose, a network bridge over qemu is good enough. 1. Setup a nfs server and export a directory with write permissions for the client 2. Copy a big file over NFS (assumes it is mounted at /mnt in the client) $ dd if=/dev/zero of=/mnt/big_zeros bs=512 count=1M status=progress Note: adjust the "count" accordingly to how fast/slow your system is. 3. Call sync and verify that it only finished after "dd" is done. 4. Repeat step #2 and now call "sync -f /boot/". Verify that it finished immediatly. [Regression Potential] Although syncfs(2) man page [2] states that it works just the same as sync - except being limited to a filesystem -, any actual differences in the implementation can affect how the initrd image file is synced to the disk, leading to unexpected behavior. [Other] If syncfs is unsupported then the 'sync' binary will ensure that a normal system-wide sync is called [1]. The proposed patch won't help in scenarios where the initrd file is located in a busy filesystem (eg. no separated /boot partition). The 'sync' and 'syncfs' specifications are pretty much similar and syncfs is expected to work the same as a normal sync regarding data integrity. Also altought the patch introduces the initrd file as an extra argument which would cause 'sync -f' to fail if the file does not exist, the update-initramfs script would fail earlier than that because that file is created by the 'mv' call a few lines above. Thus no regressions are expected. [References] [1] http://git.savannah.gnu.org/cgit/coreutils.git/commit/src/sync.c?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00 [2] http://manpages.ubuntu.com/manpages/xenial/man2/syncfs.2.html [Original bug report] At least four users, including myself, are having an issue with update-initramfs hanging while updating ubuntu 16.04. The bug has been documented while attempting an update to multiple kernel versions ( 4.4.0-24, 4.4.0-62, 4.4.0-63). The bug causes any apt-get update or install to fail, and may also lead to an unbootable system. User #1 (me): $ uname -a Linux 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ sudo apt-get upgrade Fetched 1,571 MB in 2min 9s (12.2 MB/s) Extracting templates from packages: 100% Preconfiguring packages ... (Reading database ... 344634 files and directories currently installed.) Preparing to unpack .../base-files_9.4ubuntu4.4_amd64.deb ... Unpacking base-files (9.4ubuntu4.4) over (9.4ubuntu4.3) ... Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.1) ... update-initramfs: deferring update (trigger activated) Processing triggers for man-db (2.7.5-1) ... Processing triggers for cracklib-runtime (2.9.2-1build2) ... Processing triggers for install-info (6.1.0.dfsg.1-5) ... Processing triggers for initramfs-tools (0.122ubuntu8.8) ... update-initramfs: Generating /boot/initrd.img-4.4.0-63-generic Uh oh! Let's try to troubleshoot. $ sudo killall dpkg $ sudo dpkg --configure -a Setting up initramfs-tools (0.122ubuntu8.8) ... update-initramfs: deferring update (trigger activated) Setting up base-files (9.4ubuntu4.4) ... Installing new version of config file /etc/issue ... Installing new version of config file /etc/issue.net ... Installing new version of config file /etc/lsb-release ... Setting up linux-image-4.4.0-63-generic (4.4.0-63.84) ... Running depmod. update-initramfs: deferring update (hook will be called later) initrd.img(/boot/initrd.img-4.4.0-63-generic ) points to /boot/initrd.img-4.4.0-63-generic (/boot/initrd.img-4.4.0-63-generic) -- doing nothing at /var/lib/dpkg/info/linux-image-4.4.0-63-generic.postinst line 491. vmlinuz(/boot/vmlinuz-4.4.0-63-generic ) points to /boot/vmlinuz-4.4.0-63-generic
