Bug#1059533: DEP17: handle /usr-move for gzip and its diversions by zutils
On Thu, Jan 04, 2024 at 10:08:36PM +0100, Helmut Grohne wrote:
> Attached:
> * Fixed testcase.sh
> * Fixed zutils patch (changes 1 line in zutils.preinst)
I eventually figured that there is a way out of the policy violation of
temporarily loosing e.g. /bin/zcat. The scenario was:
echo zutils deinstall | dpkg --set-selections
dpkg --install gzip.deb
In this scenario, Conflicts does not prevent unpack of gzip while zutils
is still unpacked and no amount of changes to zutils can prevent the
loss. What can prevent loss here is changes to gzip.preinst though!
I've implemented this approach for cryptsetup+cryptsetup-nuke-password
and am attaching a very similar implementation for gzip and zutils:
gzip.preinst will now check whether /bin/$tool has been diverted but
/usr/bin/$tool has not. This covers exactly the case above. When this
happens, it'll temporarily duplicate the diversion on behalf of zutils.
If zutils.preinst runs, it'll fix up the diversions, otherwise
gzip.postinst will undo what gzip.preinst did. With these changes, the
code for restoring lost files can go away and we no longer cause any
loss at all, thus that policy violation is removed. The only downside is
that gzip has to help zutils fix up its diversions and thus we
temporarily add zutils-specific code to the gzip package. I think that's
a good trade-off.
Helmut
diff -Nru gzip-1.12/debian/changelog gzip-1.12/debian/changelog
--- gzip-1.12/debian/changelog 2022-04-10 04:22:26.0 +0200
+++ gzip-1.12/debian/changelog 2023-12-23 07:46:32.0 +0100
@@ -1,3 +1,10 @@
+gzip (1.12-1.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * Move files to /usr (closes: #-1)
+
+ -- Helmut Grohne Sat, 23 Dec 2023 07:46:32 +0100
+
gzip (1.12-1) sid; urgency=high
* new upstream release
diff -Nru gzip-1.12/debian/control gzip-1.12/debian/control
--- gzip-1.12/debian/control2022-04-10 04:05:08.0 +0200
+++ gzip-1.12/debian/control2023-12-23 07:27:28.0 +0100
@@ -16,6 +16,7 @@
Pre-Depends: ${shlibs:Depends}
Depends: dpkg (>= 1.15.4) | install-info
Suggests: less
+Conflicts: zutils (<< 1.12-3.1~)
Description: GNU compression utilities
This package provides the standard GNU file compression utilities, which
are also the default compression tools for Debian. They typically operate
diff -Nru gzip-1.12/debian/dirs gzip-1.12/debian/dirs
--- gzip-1.12/debian/dirs 2022-04-09 04:15:18.0 +0200
+++ gzip-1.12/debian/dirs 2023-12-23 07:46:32.0 +0100
@@ -1,3 +1,2 @@
-bin
usr/share/info
usr/share/man/man1
diff -Nru gzip-1.12/debian/gzip.postinst gzip-1.12/debian/gzip.postinst
--- gzip-1.12/debian/gzip.postinst 1970-01-01 01:00:00.0 +0100
+++ gzip-1.12/debian/gzip.postinst 2023-12-23 07:46:32.0 +0100
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+set -e
+
+# begin-remove-after: released:forky
+if [ "$1" = configure ]; then
+ for tool in zcat zcmp zdiff zegrep zfgrep zgrep; do
+ if [ "$(dpkg-divert --truename "/usr/bin/$tool")" =
"/usr/bin/$tool.usr-is-merged" ] &&
+ [ "$(dpkg-divert --listpackage "/usr/bin/$tool")" =
zutils ]; then
+ # This diversion was added by preinst and. This
+ # indicates that zutils was unpacked at preinst time
+ # and is now removed. Thus we clean up the diversion.
+ echo "Removing duplicated diversion of /bin/$tool after
zutils is removed."
+ dpkg-divert --rename --package zutils \
+ --divert "/usr/bin/$tool.usr-is-merged" \
+ --remove "/usr/bin/$tool"
+ fi
+ done
+fi
+# end-remove-after
+
+#DEBHELPER#
diff -Nru gzip-1.12/debian/gzip.preinst gzip-1.12/debian/gzip.preinst
--- gzip-1.12/debian/gzip.preinst 1970-01-01 01:00:00.0 +0100
+++ gzip-1.12/debian/gzip.preinst 2023-12-23 07:46:32.0 +0100
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+set -e
+
+# begin-remove-after: released:forky
+if [ "$1" = upgrade ] || [ "$1" = install ]; then
+ for tool in zcat zcmp zdiff zegrep zfgrep zgrep; do
+ if [ "$(dpkg-divert --truename "/bin/$tool")" =
"/bin/$tool.gzip" ] &&
+ [ "$(dpkg-divert --listpackage "/bin/$tool")" = zutils
] &&
+ [ "$(dpkg-divert --truename "/usr/bin/$tool")" =
"/usr/bin/$tool" ]; then
+ # A pre-/usr-move diversion is installed by zutils.
+ echo "Mitigating diversion of /bin/$tool on behalf of
zutils"
+ dpkg-divert --no-rename --package zutils \
+ --divert "/usr/bin/$tool.usr-is-merged" \
+ --add "/usr/bin/$tool"
+ fi
+ done
+fi
+# end-remove-after
+
+
+#DEBHELPER#
diff -Nru gzip-1.12/debian/rules gzip-1.12/debian/rules
--- gzip-1.12/debian/rules 2022-04-09
Bug#1059533: DEP17: handle /usr-move for gzip and its diversions by zutils
Thanks to Chris for testing my patch and discovering that it was broken.
On Wed, Dec 27, 2023 at 10:27:08PM +0100, Helmut Grohne wrote:
> > So I've developed these patches (both attached). Since piuparts doesn't
> > deal well with testing essential packages, I've developed test cases
> > using mmdebstrap (also attached) and performed the --set-selections test
> > manually. Everything looks fine, but I keep the fingers crossed.
>
> Tests rerun successfully.
The fundamental mistake on my side was with testing. I managed to
comment out the installation of the actual packages. :-(
Then, I didn't see the obvious failure from zutils.preinst which was
setting up one of the diversions incorrectly.
> Sorry for the initially broken gzip patch.
Also sorry for the initially broken zutils patch.
Attached:
* Fixed testcase.sh
* Fixed zutils patch (changes 1 line in zutils.preinst)
Helmut
testcase.sh
Description: Bourne shell script
diff -Nru zutils-1.12/debian/changelog zutils-1.12/debian/changelog
--- zutils-1.12/debian/changelog2023-06-16 11:37:05.0 +0200
+++ zutils-1.12/debian/changelog2023-12-23 07:46:00.0 +0100
@@ -1,3 +1,10 @@
+zutils (1.12-3.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * DEP17 M18: Duplicate aliased diversions (Closes: #-1).
+
+ -- Helmut Grohne Sat, 23 Dec 2023 07:46:00 +0100
+
zutils (1.12-3) sid; urgency=medium
* Uploading to sid.
diff -Nru zutils-1.12/debian/rules zutils-1.12/debian/rules
--- zutils-1.12/debian/rules2023-06-13 08:08:48.0 +0200
+++ zutils-1.12/debian/rules2023-12-23 07:46:00.0 +0100
@@ -6,7 +6,7 @@
dh ${@}
override_dh_auto_configure:
- dh_auto_configure -- --exec-prefix=/ CXX=$(CXX)
+ dh_auto_configure -- CXX=$(CXX)
override_dh_auto_install:
dh_auto_install -- DESTDIR=$(CURDIR)/debian/zutils
diff -Nru zutils-1.12/debian/zutils.postrm zutils-1.12/debian/zutils.postrm
--- zutils-1.12/debian/zutils.postrm2023-06-13 08:08:48.0 +0200
+++ zutils-1.12/debian/zutils.postrm2023-12-23 07:45:29.0 +0100
@@ -6,7 +6,8 @@
remove)
for FILE in zcat zcmp zdiff zegrep zfgrep zgrep
do
- dpkg-divert --package zutils --quiet --remove --rename
--divert /bin/${FILE}.gzip /bin/${FILE}
+ dpkg-divert --package zutils --quiet --remove --rename
--divert "/usr/bin/$FILE.gzip" "/usr/bin/$FILE"
+ dpkg-divert --package zutils --quiet --remove --rename
--divert "/bin/$FILE.gzip.usr-is-merged" "/bin/$FILE"
dpkg-divert --package zutils --quiet --remove --rename
--divert /usr/share/man/man1/${FILE}.gzip.1.gz /usr/share/man/man1/${FILE}.1.gz
done
;;
diff -Nru zutils-1.12/debian/zutils.preinst zutils-1.12/debian/zutils.preinst
--- zutils-1.12/debian/zutils.preinst 2023-06-13 08:08:48.0 +0200
+++ zutils-1.12/debian/zutils.preinst 2023-12-23 07:46:00.0 +0100
@@ -2,15 +2,32 @@
set -e
+# DEP17 M18: Duplicate diversion in aliased location /bin.
+
case "${1}" in
install)
for FILE in zcat zcmp zdiff zegrep zfgrep zgrep
do
- dpkg-divert --package zutils --quiet --add --rename
--divert /bin/${FILE}.gzip /bin/${FILE}
+ dpkg-divert --package zutils --quiet --add --rename
--divert "/usr/bin/$FILE.gzip" "/usr/bin/$FILE"
+ dpkg-divert --package zutils --quiet --add --rename
--divert "/bin/$FILE.gzip.usr-is-merged" "/bin/$FILE"
dpkg-divert --package zutils --quiet --add --rename
--divert /usr/share/man/man1/${FILE}.gzip.1.gz /usr/share/man/man1/${FILE}.1.gz
done
;;
+ upgrade)
+ for FILE in zcat zcmp zdiff zegrep zfgrep zgrep
+ do
+ TRUENAME=$(dpkg-divert --truename "/bin/$FILE")
+ dpkg-divert --package zutils --quiet --add --no-rename
--divert "/usr/bin/$FILE.gzip" "/usr/bin/$FILE"
+ if test "$TRUENAME" != "/bin/$FILE.gzip.usr-is-merged";
then
+ dpkg-divert --package zutils --quiet --remove
--no-rename "/bin/$FILE"
+ dpkg-divert --package zutils --quiet --add
--no-rename --divert "/bin/$FILE.gzip.usr-is-merged" "/bin/$FILE"
+ if test -e "$DPKG_ROOT$TRUENAME" -o -h
"$DPKG_ROOT$TRUENAME"; then
+ mv "$DPKG_ROOT$TRUENAME"
"$DPKG_ROOT/bin/$FILE.gzip.usr-is-merged"
+ fi
+ fi
+ done
+ ;;
abort-upgrade|upgrade)
;;
Bug#1059533: DEP17: handle /usr-move for gzip and its diversions by zutils
Package: gzip
Version: 1.12-1
User: helm...@debian.org
Usertags: dep17m2
Tags: patch
Control: clone -1 -2
Control: reassign -2 zutils/1.12-3
Control: block -1 by -2
Hi,
as part of DEP17, I am looking into moving aliased files in essential
packages from / to /usr. gzip is one such package. Unfortunately, moving
its files from / to /bin causes breakage. zutils diverts e.g. /bin/zcmp
and once gzip moves that to /usr/bin/zcmp, the diversions issued by
zutils become ineffective (DEP17 P3) causing unintended file overwrites.
Mitigating these has turned out to be non-trival and I think we now have
a good understanding of the edge cases having gone through them with
molly-guard. I propose duplicating diversions (DEP17 M18) here as well.
For gzip the story is relatively simple. It moves all the files, but it
must not be unpacked when there is a version of zutils installed that
hasn't duplicated its diversions yet. The best we can do here is adding
versioned Conflicts (not Breaks). I caution that this is not entirely
bullet-proof. If you `echo zutils deinstall | dpkg --set-selections` and
then `dpkg --unpack new_gzip.deb`, it'll unpack the moved gzip first and
then remove zu old zutils that lacks the duplicated diversions. Even in
this case, the gzip package would continue working after the upgrade.
For zutils, the story is less easy. In order to avoid apt issuing a
temporary removal of zutils (and thus trigger the wrongly ordered
unpacks above), zutils must not issue versioned breaks for gzip and
therefore it must carry the aliased diversions during the trixie cycle
(and not just during the upgrade).
So I've developed these patches (both attached). Since piuparts doesn't
deal well with testing essential packages, I've developed test cases
using mmdebstrap (also attached) and performed the --set-selections test
manually. Everything looks fine, but I keep the fingers crossed.
I ask you to upload these changes to experimental (not unstable). Once
both updates are in experimental, dumat will be able to analyze and
we'll also see what other kinds of QA says. Then once that works for
both packages, we can upload zutils to unstable and then gzip.
Thanks for your cooperation
Helmut
diff -Nru gzip-1.12/debian/changelog gzip-1.12/debian/changelog
--- gzip-1.12/debian/changelog 2022-04-10 04:22:26.0 +0200
+++ gzip-1.12/debian/changelog 2023-12-23 07:46:32.0 +0100
@@ -1,3 +1,10 @@
+gzip (1.12-1.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * Move files to /usr (closes: #-1)
+
+ -- Helmut Grohne Sat, 23 Dec 2023 07:46:32 +0100
+
gzip (1.12-1) sid; urgency=high
* new upstream release
diff -Nru gzip-1.12/debian/control gzip-1.12/debian/control
--- gzip-1.12/debian/control2022-04-10 04:05:08.0 +0200
+++ gzip-1.12/debian/control2023-12-23 07:27:28.0 +0100
@@ -16,6 +16,7 @@
Pre-Depends: ${shlibs:Depends}
Depends: dpkg (>= 1.15.4) | install-info
Suggests: less
+Conflicts: zutils (<< 1.12-3.1~)
Description: GNU compression utilities
This package provides the standard GNU file compression utilities, which
are also the default compression tools for Debian. They typically operate
diff -Nru gzip-1.12/debian/rules gzip-1.12/debian/rules
--- gzip-1.12/debian/rules 2022-04-09 04:15:18.0 +0200
+++ gzip-1.12/debian/rules 2023-12-23 07:26:46.0 +0100
@@ -47,7 +47,7 @@
_topdir=$(call shellescape,$(shell pwd))
CONFIGURE_ARGS=--prefix=/usr \
- --bindir=/bin \
+ --bindir=/usr/bin \
--infodir=${_topdir}/debian/gzip/usr/share/info \
--mandir=${_topdir}/debian/gzip/usr/share/man \
--disable-silent-rules
diff -Nru zutils-1.12/debian/changelog zutils-1.12/debian/changelog
--- zutils-1.12/debian/changelog2023-06-16 11:37:05.0 +0200
+++ zutils-1.12/debian/changelog2023-12-23 07:46:00.0 +0100
@@ -1,3 +1,10 @@
+zutils (1.12-3.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * DEP17 M18: Duplicate aliased diversions (Closes: #-1).
+
+ -- Helmut Grohne Sat, 23 Dec 2023 07:46:00 +0100
+
zutils (1.12-3) sid; urgency=medium
* Uploading to sid.
diff -Nru zutils-1.12/debian/rules zutils-1.12/debian/rules
--- zutils-1.12/debian/rules2023-06-13 08:08:48.0 +0200
+++ zutils-1.12/debian/rules2023-12-23 07:46:00.0 +0100
@@ -6,7 +6,7 @@
dh ${@}
override_dh_auto_configure:
- dh_auto_configure -- --exec-prefix=/ CXX=$(CXX)
+ dh_auto_configure -- CXX=$(CXX)
override_dh_auto_install:
dh_auto_install -- DESTDIR=$(CURDIR)/debian/zutils
diff -Nru zutils-1.12/debian/zutils.postrm zutils-1.12/debian/zutils.postrm
--- zutils-1.12/debian/zutils.postrm2023-06-13 08:08:48.0 +0200
+++ zutils-1.12/debian/zutils.postrm2023-12-23 07:45:29.0 +0100
@@ -6,7 +6,8 @@
remove)
for FILE in zcat
