[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: ef0cfc2c0e68299ddfa355732c76cf7d9ed83fcb Author: Sam James gentoo org> AuthorDate: Mon Nov 22 06:30:45 2021 + Commit: Sam James gentoo org> CommitDate: Mon Nov 22 06:31:58 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=ef0cfc2c 2021-09-29-possible-failure-to-preserve-libraries: mention when it's ok to skip -e It should be okay to skip -e @world and instead only rebuild libraries + their consumers in some cases. Bug: https://bugs.gentoo.org/811462 Signed-off-by: Sam James gentoo.org> .../2021-09-29-possible-failure-to-preserve-libraries.en.txt | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt index 7154bc4..68aa6af 100644 --- a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -93,7 +93,12 @@ Step 3. Given that there are possible other side-effects of the corruption/bug, Note that binary packages may need to be discarded given they may contain corrupt metadata. - If no libraries were broken, it is likely safe to skip this step. + If no libraries were broken, it is likely safe to skip this step. It + should be sufficient, for resource-constrained machines, to simply + rebuild any broken libraries and their consumers (reverse-dependencies): + revdep-rebuild may help you do this. + + (If you do not know what that means, please proceed with Step 3.) Please see the wiki [0] for a full description of the background of this problem and handling corner cases such as e.g. already
[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: 8e7f2dd261654f4876ac16cf9f50b24003f4f7e2 Author: Sam James gentoo org> AuthorDate: Wed Oct 13 00:29:17 2021 + Commit: Sam James gentoo org> CommitDate: Wed Oct 13 00:29:38 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=8e7f2dd2 2021-09-29-possible-failure-to-preserve-libraries: clarify world upgrade Signed-off-by: Sam James gentoo.org> ...9-possible-failure-to-preserve-libraries.en.txt | 27 +- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt index cfff106..7154bc4 100644 --- a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -44,7 +44,7 @@ Fixing a broken system is not always straightforward. It is strongly recommended to take a backup of your full system before proceeding, as well as a copy of /var/db/pkg (the VDB): -1. A tool has been developed [5] to attempt to fix the consistency +Step 1. A tool has been developed [5] to attempt to fix the consistency of the Portage database. Using this tool to modify the VDB is NOT mandatory (read the full news item before proceeding) - you can skip to Step 2 if you wish, but fixing the integrity of the VDB @@ -72,23 +72,28 @@ as well as a copy of /var/db/pkg (the VDB): # into your real /var/db/pkg/ directory tree. ``` -2. Attempt to rebuild the affected packages, first upgrading +Step 2. Attempt to rebuild the affected packages, first upgrading app-misc/pax-utils to the latest version: ``` $ emerge --ask --verbose --oneshot ">=app-misc/pax-utils-1.3.3" $ emerge --ask --verbose --oneshot --usepkg=n $(grep -v '#' broken_vdb_packages) ``` -Given that there are possible other side-effects of the corruption/bug, -it is strongly recommended that if any corruption is detected, all -packages on the system should be rebuilt, after following the above -steps: -``` -$ emerge --ask --emptytree --usepkg=n @world -``` + It's possible that the relevant versions have disappeared from the tree, so + if the emerge command fails, please attempt a normal world upgrade. + +Step 3. Given that there are possible other side-effects of the corruption/bug, + it is strongly recommended that if any corruption is detected, all + packages on the system should be rebuilt, after following the above + steps: + ``` + $ emerge --ask --emptytree --usepkg=n @world + ``` + + Note that binary packages may need to be discarded given they may + contain corrupt metadata. -Note that binary packages may need to be discarded given they may -contain corrupt metadata. + If no libraries were broken, it is likely safe to skip this step. Please see the wiki [0] for a full description of the background of this problem and handling corner cases such as e.g. already
[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: b799e9ed123d65d007fa6496e8b5c5ee4073efbd Author: Sam James gentoo org> AuthorDate: Wed Oct 6 01:34:00 2021 + Commit: Sam James gentoo org> CommitDate: Wed Oct 6 01:34:00 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=b799e9ed 2021-09-29-possible-failure-to-preserve-libraries: fix command for comment Bug: https://bugs.gentoo.org/811462 Signed-off-by: Sam James gentoo.org> .../2021-09-29-possible-failure-to-preserve-libraries.en.txt| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt index bf66553..cfff106 100644 --- a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -76,7 +76,7 @@ as well as a copy of /var/db/pkg (the VDB): app-misc/pax-utils to the latest version: ``` $ emerge --ask --verbose --oneshot ">=app-misc/pax-utils-1.3.3" - $ emerge --ask --verbose --oneshot --usepkg=n $(cat broken_vdb_packages) + $ emerge --ask --verbose --oneshot --usepkg=n $(grep -v '#' broken_vdb_packages) ``` Given that there are possible other side-effects of the corruption/bug,
[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: 4d715f694dba3bde1d7235b315d8ab3a654fc22d Author: Sam James gentoo org> AuthorDate: Thu Sep 30 17:16:51 2021 + Commit: Sam James gentoo org> CommitDate: Thu Sep 30 17:16:51 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=4d715f69 2021-09-29-possible-failure-to-preserve-libraries.en.txt: fix typo Thanks Hantezer. Signed-off-by: Sam James gentoo.org> .../2021-09-29-possible-failure-to-preserve-libraries.en.txt| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt index 08f3229..bf66553 100644 --- a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -73,7 +73,7 @@ as well as a copy of /var/db/pkg (the VDB): ``` 2. Attempt to rebuild the affected packages, first upgrading - app-portage/pax-utils to the latest version: + app-misc/pax-utils to the latest version: ``` $ emerge --ask --verbose --oneshot ">=app-misc/pax-utils-1.3.3" $ emerge --ask --verbose --oneshot --usepkg=n $(cat broken_vdb_packages)
[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: 656c8279c5fd1ce6b6f5bfd022d43f1ce0887605 Author: Sam James gentoo org> AuthorDate: Wed Sep 29 17:00:08 2021 + Commit: Sam James gentoo org> CommitDate: Wed Sep 29 17:00:08 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=656c8279 2021-09-29-possible-failure-to-preserve-libraries: note existing fixes Signed-off-by: Sam James gentoo.org> .../2021-09-29-possible-failure-to-preserve-libraries.en.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt index 5891318..08f3229 100644 --- a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -17,8 +17,8 @@ new major version, but pax-utils has not yet been upgraded to a version compatible with it (but at that moment stays undetected). The full technical details and investigation can be found on a Wiki page -[0] and on Bugzilla [1]. Work is underway to prevent this happening -again both within Portage [2] (possibly more to come) and within the +[0] and on Bugzilla [1]. Changes have been made to prevent this happening +again both within Portage [7] (with possibly more to come [2]) and within the glibc and pax-utils ebuilds [3][4]. To detect whether a system is affected, emerge the @@ -101,3 +101,4 @@ being affected by system breakage [6] as a result of the bug. [4] https://bugs.gentoo.org/811462#c7 [5] https://github.com/thesamesam/recover-broken-vdb [6] https://wiki.gentoo.org/wiki/Fix_my_Gentoo +[7] https://gitweb.gentoo.org/proj/portage.git/commit/?id=83af7270fafbd7b1eed0031a5e06836ad1edf06d
[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: 68dc3aea4f603e2f619a609674e316e986f59f09 Author: Sam James gentoo org> AuthorDate: Wed Sep 29 16:58:04 2021 + Commit: Sam James gentoo org> CommitDate: Wed Sep 29 16:58:04 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=68dc3aea 2021-09-29-possible-failure-to-preserve-libraries: only one-off run needed Signed-off-by: Sam James gentoo.org> .../2021-09-29-possible-failure-to-preserve-libraries.en.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt index 904c54a..5891318 100644 --- a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -34,7 +34,9 @@ Then run recover-broken-vdb-find-broken.sh: $ recover-broken-vdb-find-broken.sh | tee broken_vdb_packages ``` -This check should be run on all Gentoo systems. +This check should be run on all Gentoo systems. It is only necessary +to run this as a one-off, as changes have been made to prevent such +problems occurring in future. If you have any output, read on.
[gentoo-commits] data/gentoo-news:master commit in: 2021-09-29-possible-failure-to-preserve-libraries/
commit: 1a196bdef62565e377e90807b8b324547e7d4814 Author: Sam James gentoo org> AuthorDate: Mon Sep 20 21:13:21 2021 + Commit: Sam James gentoo org> CommitDate: Wed Sep 29 14:32:18 2021 + URL:https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=1a196bde 2021-09-24-possible-failure-to-preserve-libraries: add item Bug: https://bugs.gentoo.org/811462 Signed-off-by: Hank Leininger korelogic.com> Signed-off-by: Sam James gentoo.org> ...9-possible-failure-to-preserve-libraries.en.txt | 101 + 1 file changed, 101 insertions(+) diff --git a/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt new file mode 100644 index 000..904c54a --- /dev/null +++ b/2021-09-29-possible-failure-to-preserve-libraries/2021-09-29-possible-failure-to-preserve-libraries.en.txt @@ -0,0 +1,101 @@ +Title: Possible failure to preserve libraries +Author: Sam James +Author: Hank Leininger +Posted: 2021-09-29 +Revision: 1 +News-Item-Format: 2.0 +Display-If-Installed: sys-apps/portage + +We have observed in some cases corruption of Portage's internal database +(VDB), where the libraries provided by a package are not recorded. This +can break the "preserve-libs" functionality, and thus in rare cases +break your system during much later updates (even if you do not use +"preseved-libs" now, but decide to switch it on later). + +The underlying problem occurs usually when glibc has been upgraded to a +new major version, but pax-utils has not yet been upgraded to a version +compatible with it (but at that moment stays undetected). + +The full technical details and investigation can be found on a Wiki page +[0] and on Bugzilla [1]. Work is underway to prevent this happening +again both within Portage [2] (possibly more to come) and within the +glibc and pax-utils ebuilds [3][4]. + +To detect whether a system is affected, emerge the +app-portage/recover-broken-vdb package: +``` +$ emerge --ask --verbose --oneshot app-portage/recover-broken-vdb +``` +which provides two tools: recover-broken-vdb-find-broken.sh and +recover-broken-vdb. + +Then run recover-broken-vdb-find-broken.sh: +``` +$ recover-broken-vdb-find-broken.sh | tee broken_vdb_packages +``` + +This check should be run on all Gentoo systems. + +If you have any output, read on. + +Fixing a broken system is not always straightforward. It is strongly +recommended to take a backup of your full system before proceeding, +as well as a copy of /var/db/pkg (the VDB): + +1. A tool has been developed [5] to attempt to fix the consistency + of the Portage database. Using this tool to modify the VDB is NOT + mandatory (read the full news item before proceeding) - you can skip + to Step 2 if you wish, but fixing the integrity of the VDB + makes it as safe as reasonably possible to proceed with + rebuilding packages. + + Run: + ``` + # Take a backup of /var/db/pkg before proceeding, such as by doing: + $ cp -a /var/db/pkg /var/db/pkg.orig + + # And then: + $ emerge --ask --verbose --oneshot --noreplace \ + app-portage/recover-broken-vdb + + $ recover-broken-vdb + + # The tool will output to a random temporary directory. + # Inspect the results, and then update the real /var/db/pkg/ + # by doing either: + + $ recover-broken-vdb --output /var/db/pkg + + # Or, manually copying the new files from the temporary directory tree + # into your real /var/db/pkg/ directory tree. + ``` + +2. Attempt to rebuild the affected packages, first upgrading + app-portage/pax-utils to the latest version: + ``` + $ emerge --ask --verbose --oneshot ">=app-misc/pax-utils-1.3.3" + $ emerge --ask --verbose --oneshot --usepkg=n $(cat broken_vdb_packages) + ``` + +Given that there are possible other side-effects of the corruption/bug, +it is strongly recommended that if any corruption is detected, all +packages on the system should be rebuilt, after following the above +steps: +``` +$ emerge --ask --emptytree --usepkg=n @world +``` + +Note that binary packages may need to be discarded given they may +contain corrupt metadata. + +Please see the wiki [0] for a full description of the background +of this problem and handling corner cases such as e.g. already +being affected by system breakage [6] as a result of the bug. + +[0] https://wiki.gentoo.org/wiki/Project:Toolchain/Corrupt_VDB_ELF_files +[1] https://bugs.gentoo.org/811462 +[2] https://github.com/gentoo/portage/pull/744 +[3] https://bugs.gentoo.org/811462#c6 +[4] https://bugs.gentoo.org/811462#c7 +[5] https://github.com/thesamesam/recover-broken-vdb +[6] https://wiki.gentoo.org/wiki/Fix_my_Gentoo
