** Description changed: + [Impact] + + Users may have third-party packages unnecessarily removed during their + upgrade to plucky, because ubuntu-release-upgrader does not exclude such + packages from the removal candidates, as it did previously. + + [Test Plan] + + Basically, install a package from a third-party PPA, and do the upgrade. + Confirm that the package is considered foreign at the beginning of the + upgrade, but that it is not later removed. + + 1. Configure a PPA and install a package from it. Using a popular + example that motivated this bug report: + + $ cat > /etc/apt/sources.list.d/code.sources << EOF + Types: deb + URIs: https://packages.microsoft.com/repos/code + Suites: stable + Components: main + Signed-By: . + -----BEGIN PGP PUBLIC KEY BLOCK----- + Version: GnuPG v1.4.7 (GNU/Linux) + . + mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT + LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV + 7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag + OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j + H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr + M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs + ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC + AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH + /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe + MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy + 7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV + KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ + XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+ + NdCFTW7wY0Fb1fWJ+/KTsC4= + =J6gs + -----END PGP PUBLIC KEY BLOCK----- + EOF + $ apt update + $ apt install code -y + + 2. Run the upgrade. Note that due to the fact that plucky is now + released, but upgrades are disabled, we cannot use the --proposed flag + to test upgrades conveniently. Instead, we need to manually download the + tarball from https://changelogs.ubuntu.com/meta-release-proposed, unpack + it, and run the script. + + $ wget http://archive.ubuntu.com/ubuntu/dists/plucky-proposed/main/dist-upgrader-all/current/plucky.tar.gz + $ tar xf plucky.tar.gz + $ sudo -E ./plucky --frontend DistUpgradeViewText + + 3. When the upgrade gets to the 'Remove obsolete packages?' stage, enter + 'd' for details, and verify that 'code' is not suggested for removal. + + 4. Check the upgrade log to ensure that 'code' was considered foreign at + the beginning of the upgrade: + + $ grep "Foreign.*rewrite" /var/log/dist/upgrade/main.log + + [Where problems could occur] + + [Original Description] + This is a bug introduced by commit ba84c16586 ("DistUpgrade: warn about foreign packages after rewriting sources"). This commit is intended to be non-functional with respect to package removals etc., but inadvertently changed the list of packages which should be excluded from consideration for removal. The packages which are considered "foreign" will be different before and after rewriting the new sources. Later, packages which were deemed "foreign" at the beginning of the upgrade will be excluded from removal. The above patch tracks foreign packages before and after rewriting sources for the purposes of supplying the warning, but later uses the post-rewrite list when considering exclusions, whereas it should use the pre-rewrite list. This could be reproduced with any PPA, but to illustrate with a popular example (public key from https://packages.microsoft.com/keys/microsoft.asc): $ cat > /etc/apt/sources.list.d/code.sources << EOF Types: deb URIs: https://packages.microsoft.com/repos/code Suites: stable Components: main Signed-By: . -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.7 (GNU/Linux) . mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV 7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy 7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+ NdCFTW7wY0Fb1fWJ+/KTsC4= =J6gs -----END PGP PUBLIC KEY BLOCK----- EOF $ apt update $ apt install code -y $ do-release-upgrade --devel [...] Eventually, looking at the logs, we can see the different in the pre- vs post-rewrite foreign packages: $ grep "Foreign.*rewriting" /var/log/dist-upgrade/main.log 2025-04-18 15:38:36,057 DEBUG Foreign (before rewriting sources): code 2025-04-18 15:38:36,057 DEBUG Foreign (after rewriting sources): Then, later, code will be shown as a remove candidate when the user is prompted about removals: [...] Remove obsolete packages? 28 packages are going to be removed. Continue [yN] Details [d]d Remove: code <------- Should not be here Remove (was auto installed) humanity-icon-theme libapt-pkg6.0t64 libassuan0 libdrm-nouveau2 libdrm-radeon1 libdw1t64 libfwupd2 libgl1-amber-dri libglapi-amber libgusb2 libicu74 libmodule-scandeps-perl libnl-genl-3-200 libnsl2 libperl5.38t64 libpython3.12-minimal libpython3.12-stdlib libpython3.12t64 libsgutils2-1.46-2 libxcb-dri2-0 perl-modules-5.38 python3-colorama python3.12 python3.12-gdbm python3.12-minimal sosreport ubuntu-mono Continue [yN] Details [d]
** Description changed: [Impact] Users may have third-party packages unnecessarily removed during their upgrade to plucky, because ubuntu-release-upgrader does not exclude such packages from the removal candidates, as it did previously. [Test Plan] Basically, install a package from a third-party PPA, and do the upgrade. Confirm that the package is considered foreign at the beginning of the upgrade, but that it is not later removed. 1. Configure a PPA and install a package from it. Using a popular example that motivated this bug report: $ cat > /etc/apt/sources.list.d/code.sources << EOF Types: deb URIs: https://packages.microsoft.com/repos/code Suites: stable Components: main Signed-By: . -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.7 (GNU/Linux) . mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV 7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy 7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+ NdCFTW7wY0Fb1fWJ+/KTsC4= =J6gs -----END PGP PUBLIC KEY BLOCK----- EOF $ apt update $ apt install code -y 2. Run the upgrade. Note that due to the fact that plucky is now released, but upgrades are disabled, we cannot use the --proposed flag to test upgrades conveniently. Instead, we need to manually download the tarball from https://changelogs.ubuntu.com/meta-release-proposed, unpack it, and run the script. $ wget http://archive.ubuntu.com/ubuntu/dists/plucky-proposed/main/dist-upgrader-all/current/plucky.tar.gz $ tar xf plucky.tar.gz $ sudo -E ./plucky --frontend DistUpgradeViewText 3. When the upgrade gets to the 'Remove obsolete packages?' stage, enter 'd' for details, and verify that 'code' is not suggested for removal. 4. Check the upgrade log to ensure that 'code' was considered foreign at the beginning of the upgrade: $ grep "Foreign.*rewrite" /var/log/dist/upgrade/main.log [Where problems could occur] + + This patch fixes the list of foreign packages used to exclude from + removal during the obsolete removal step. At this point, the package + removals are only for clean up, and should not have any impact on the + upgrade. Any theoretical issue would be seen in the package removal + step. [Original Description] This is a bug introduced by commit ba84c16586 ("DistUpgrade: warn about foreign packages after rewriting sources"). This commit is intended to be non-functional with respect to package removals etc., but inadvertently changed the list of packages which should be excluded from consideration for removal. The packages which are considered "foreign" will be different before and after rewriting the new sources. Later, packages which were deemed "foreign" at the beginning of the upgrade will be excluded from removal. The above patch tracks foreign packages before and after rewriting sources for the purposes of supplying the warning, but later uses the post-rewrite list when considering exclusions, whereas it should use the pre-rewrite list. This could be reproduced with any PPA, but to illustrate with a popular example (public key from https://packages.microsoft.com/keys/microsoft.asc): $ cat > /etc/apt/sources.list.d/code.sources << EOF Types: deb URIs: https://packages.microsoft.com/repos/code Suites: stable Components: main Signed-By: . -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.7 (GNU/Linux) . mQENBFYxWIwBCADAKoZhZlJxGNGWzqV+1OG1xiQeoowKhssGAKvd+buXCGISZJwT LXZqIcIiLP7pqdcZWtE9bSc7yBY2MalDp9Liu0KekywQ6VVX1T72NPf5Ev6x6DLV 7aVWsCzUAF+eb7DC9fPuFLEdxmOEYoPjzrQ7cCnSV4JQxAqhU4T6OjbvRazGl3ag OeizPXmRljMtUUttHQZnRhtlzkmwIrUivbfFPD+fEoHJ1+uIdfOzZX8/oKHKLe2j H632kvsNzJFlROVvGLYAk2WRcLu+RjjggixhwiB+Mu/A8Tf4V6b+YppS44q8EvVr M+QvY7LNSOffSO6Slsy9oisGTdfE39nC7pVRABEBAAG0N01pY3Jvc29mdCAoUmVs ZWFzZSBzaWduaW5nKSA8Z3Bnc2VjdXJpdHlAbWljcm9zb2Z0LmNvbT6JATUEEwEC AB8FAlYxWIwCGwMGCwkIBwMCBBUCCAMDFgIBAh4BAheAAAoJEOs+lK2+EinPGpsH /32vKy29Hg51H9dfFJMx0/a/F+5vKeCeVqimvyTM04C+XENNuSbYZ3eRPHGHFLqe MNGxsfb7C7ZxEeW7J/vSzRgHxm7ZvESisUYRFq2sgkJ+HFERNrqfci45bdhmrUsy 7SWw9ybxdFOkuQoyKD3tBmiGfONQMlBaOMWdAsic965rvJsd5zYaZZFI1UwTkFXV KJt3bp3Ngn1vEYXwijGTa+FXz6GLHueJwF0I7ug34DgUkAFvAs8Hacr2DRYxL5RJ XdNgj4Jd2/g6T9InmWT0hASljur+dJnzNiNCkbn9KbX7J/qK1IbR8y560yRmFsU+ NdCFTW7wY0Fb1fWJ+/KTsC4= =J6gs -----END PGP PUBLIC KEY BLOCK----- EOF $ apt update $ apt install code -y $ do-release-upgrade --devel [...] Eventually, looking at the logs, we can see the different in the pre- vs post-rewrite foreign packages: $ grep "Foreign.*rewriting" /var/log/dist-upgrade/main.log 2025-04-18 15:38:36,057 DEBUG Foreign (before rewriting sources): code 2025-04-18 15:38:36,057 DEBUG Foreign (after rewriting sources): Then, later, code will be shown as a remove candidate when the user is prompted about removals: [...] Remove obsolete packages? 28 packages are going to be removed. Continue [yN] Details [d]d Remove: code <------- Should not be here Remove (was auto installed) humanity-icon-theme libapt-pkg6.0t64 libassuan0 libdrm-nouveau2 libdrm-radeon1 libdw1t64 libfwupd2 libgl1-amber-dri libglapi-amber libgusb2 libicu74 libmodule-scandeps-perl libnl-genl-3-200 libnsl2 libperl5.38t64 libpython3.12-minimal libpython3.12-stdlib libpython3.12t64 libsgutils2-1.46-2 libxcb-dri2-0 perl-modules-5.38 python3-colorama python3.12 python3.12-gdbm python3.12-minimal sosreport ubuntu-mono Continue [yN] Details [d] -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2107657 Title: upgrades to plucky incorrectly remove foreign packages from disabled sources To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ubuntu-release-upgrader/+bug/2107657/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
