Control: tags -1 patch Hello,
On top of the issue described in this bug, the Debian CTTE in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#110 also ruled as follow because of the same missing functionality in dpkg: "Moratorium on moving files' logical locations into /usr <...> The Technical Committee recommends that during the Debian 12 development cycle, the maintainers of individual packages should not proactively move files from the root filesystem to the corresponding locations in /usr in the data.tar.* of packages. Files that were in /usr in the Debian 11 release should remain in /usr, while files that were in /bin, /lib* or /sbin in the Debian 11 release should remain in those directories. If files were moved from /bin, /lib* or /sbin into /usr since the Debian 11 release, they should be moved back to their Debian 11 locations." The attached patch from user uau is a working solution that would allow to lift this moratorium and also solve this bug. It is shared as-is, with permission and under the same license as src:dpkg. -- Kind regards, Luca Boccassi
diff --git a/README.usrmerge b/README.usrmerge
new file mode 100644
index 000000000..ee8b1646d
--- /dev/null
+++ b/README.usrmerge
@@ -0,0 +1,67 @@
+This is a proof of concept version of dpkg with explicit usrmerge
+support. This means the following file path conversions are done in
+various places:
+/lib -> /usr/lib
+/bin ->/ /usr/bin
+/sbin -> /usr/sbin
+
+Usrmerge also includes other links like "/libx32 -> /usr/libx32"
+depending on platform, but those are not handled. This shouldn't
+matter much for testing since few packages contain those anyway, and
+it does not matter much whether dpkg is aware of aliasing for those
+paths.
+
+The conversion is done at least in the following cases:
+- installing packages: any install path contained in .deb
+- reading list of existing installed files (/var/lib/dpkg/info/*.list)
+- dpkg-query (or dpkg) -S patterns beginning with '/'
+- reading existing statoverride locations, adding/removing
+- dpkg-statoverride --list glob patterns beginning with '/'
+- reading existing divert locations, adding/removing
+- dpkg-divert --list glob patterns beginning with '/'
+- file trigger locations
+- .md5sums file targets
+- .conffiles (should any exist with such paths)
+
+These conversions mean that dpkg should notice any file conflicts
+between /bin/x and /usr/bin/x, since both are now converted to the
+same literal path /usr/bin/x. Most maintainer scripts and tools should
+continue working, as the paths they use are automatically converted.
+However, there is some potential for maintainer scripts or other tools
+to break if they care about the exact paths returned from queries like
+"dpkg-divert --truename" or "dpkg-query -S" (since those will now
+refer to the real location under /usr).
+
+No separate database conversion step is required. Existing paths are
+converted as they are read from disk. Any files generated and written
+by dpkg will have the new paths, since they will have been converted
+before writing. This means newly created .list files use converted
+locations, but .triggers files do not (they are copied verbatim from
+the package, not generated from already parsed data). The
+dpkg-generated triggers/File file will contain converted locations
+when it is written. (Applies at least to udev /lib/udev/hwdb.d).
+
+The query conversions mean that for example "dpkg -S '/bin/ba*'"
+output will include "/usr/bin/bash" (as the query will be converted to
+"/usr/bin/ba*"), but "dpkg -S '/???/bash'" will find no matches since
+the query itself can not be converted and the pattern does not match
+the actual path "/usr/bin/bash".
+
+There's a minor cosmetic issue where the conversion does not add the
+"/usr" directory itself, so theoretically a package containing files
+only under /lib would be listed as containing the directory "/usr/lib"
+but not "/usr", but this shouldn't matter in practice (Debian packages
+contain /usr/share/doc/* anyway). Maybe it could cause issues when
+bootstrapping a new system in case this means trying to create
+/usr/bin before /usr? Could be worked around for example by just
+hardcoding "/usr" as contents of every package (in dpkg), more
+"correct" (and complex) fix, or disabling usrmerge logic for initial
+bootstrap as below.
+
+The usrmerge changes can be disabled by creating the file
+"/var/lib/dpkg/DISABLE_USRMERGE_LOGIC". There's no logic to create
+such a file automatically or to sanity check the existence of usrmerge
+links if the file does not exist. Expect things to break if you run
+this dpkg version on a system without the usrmerge links and without
+creating the file, or if you create the file on an usrmerged system
+after already running this dpkg without it existing.
diff --git a/debian/dpkg.docs b/debian/dpkg.docs
index 308db3568..ebdbd8139 100644
--- a/debian/dpkg.docs
+++ b/debian/dpkg.docs
@@ -1,5 +1,6 @@
AUTHORS
THANKS
+README.usrmerge
debian/README.bug-usertags
usr/share/doc/dpkg/README.api
usr/share/doc/dpkg/README.feature-removal-schedule
diff --git a/lib/dpkg/db-ctrl-access.c b/lib/dpkg/db-ctrl-access.c
index 2787d3977..a61482563 100644
--- a/lib/dpkg/db-ctrl-access.c
+++ b/lib/dpkg/db-ctrl-access.c
@@ -35,21 +35,16 @@
#include <dpkg/fsys.h>
#include <dpkg/db-ctrl.h>
#include <dpkg/debug.h>
+#include <dpkg/file.h>
bool
pkg_infodb_has_file(struct pkginfo *pkg, struct pkgbin *pkgbin,
const char *name)
{
const char *filename;
- struct stat stab;
filename = pkg_infodb_get_file(pkg, pkgbin, name);
- if (lstat(filename, &stab) == 0)
- return true;
- else if (errno == ENOENT)
- return false;
- else
- ohshite(_("unable to check existence of '%.250s'"), filename);
+ return file_exists(filename);
}
void
diff --git a/lib/dpkg/file.c b/lib/dpkg/file.c
index 4f7c3aaa2..4db5ba5d2 100644
--- a/lib/dpkg/file.c
+++ b/lib/dpkg/file.c
@@ -255,3 +255,13 @@ file_show(const char *filename)
ohshite(_("cannot write file %s into the pager"), filename);
}
}
+
+bool file_exists(char *filename) {
+ struct stat stab;
+ if (lstat(filename, &stab) == 0)
+ return true;
+ else if (errno == ENOENT)
+ return false;
+ else
+ ohshite(_("unable to check existence of '%.250s'"), filename);
+}
diff --git a/lib/dpkg/file.h b/lib/dpkg/file.h
index 0136f6901..8b1cfd73a 100644
--- a/lib/dpkg/file.h
+++ b/lib/dpkg/file.h
@@ -65,6 +65,7 @@ void file_lock(int *lockfd, enum file_lock_flags flags, const char *filename,
const char *filedesc);
void file_unlock(int fd, const char *filename, const char *filedesc);
void file_show(const char *filename);
+bool file_exists(char *filename);
/** @} */
diff --git a/lib/dpkg/fsys-hash.c b/lib/dpkg/fsys-hash.c
index 20b755732..f47a2ac87 100644
--- a/lib/dpkg/fsys-hash.c
+++ b/lib/dpkg/fsys-hash.c
@@ -79,7 +79,14 @@ fsys_hash_find_node(const char *name, enum fsys_hash_find_flags flags)
* leading slash. */
name = path_skip_slash_dotslash(name);
- pointerp = bins + (str_fnv_hash(name) % (BINS));
+ unsigned int hashval;
+ bool merge = path_is_usrmerged_nostartslash(name);
+ if (merge) {
+ hashval = str_fnv_hash("usr/");
+ hashval = str_fnv_hash_continue(name, hashval);
+ } else
+ hashval = str_fnv_hash(name);
+ pointerp = bins + (hashval % (BINS));
while (*pointerp) {
/* XXX: This should not be needed, but it has been a constant
* source of assertions over the years. Hopefully with the
@@ -88,8 +95,14 @@ fsys_hash_find_node(const char *name, enum fsys_hash_find_flags flags)
internerr("filename node '%s' does not start with '/'",
(*pointerp)->name);
- if (strcmp((*pointerp)->name + 1, name) == 0)
+ if (!merge) {
+ if (strcmp((*pointerp)->name + 1, name) == 0)
break;
+ } else {
+ if (strncmp((*pointerp)->name, "/usr/", 5) == 0
+ && strcmp((*pointerp)->name + 5, name) == 0)
+ break;
+ }
pointerp = &(*pointerp)->next;
}
if (*pointerp)
@@ -103,11 +116,7 @@ fsys_hash_find_node(const char *name, enum fsys_hash_find_flags flags)
if ((flags & FHFF_NOCOPY) && name > orig_name && name[-1] == '/') {
newnode->name = name - 1;
} else {
- char *newname = nfmalloc(strlen(name) + 2);
-
- newname[0] = '/';
- strcpy(newname + 1, name);
- newnode->name = newname;
+ newnode->name = path_usrmerge_forhash(name);
}
*pointerp = newnode;
nfiles++;
diff --git a/lib/dpkg/path.c b/lib/dpkg/path.c
index 1a4dba182..321c32b27 100644
--- a/lib/dpkg/path.c
+++ b/lib/dpkg/path.c
@@ -25,8 +25,10 @@
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
+#include <stdbool.h>
#include <dpkg/dpkg.h>
+#include <dpkg/dpkg-db.h>
#include <dpkg/string.h>
#include <dpkg/path.h>
@@ -168,3 +170,47 @@ path_quote_filename(char *dst, const char *src, size_t n)
return r;
}
+
+static bool disable_usrmerge;
+
+#define ARRAY_SIZE(s) (sizeof(s) / sizeof((s)[0]))
+#define PREFIX "/usr"
+static char *usrmerged_dirs[] = {
+ "bin",
+ "sbin",
+ "lib",
+};
+
+bool path_is_usrmerged_nostartslash(char *path) {
+ if (disable_usrmerge)
+ return false;
+ for (int i = 0; i < ARRAY_SIZE(usrmerged_dirs); i++) {
+ int l = strlen(usrmerged_dirs[i]);
+ if (strncmp(path, usrmerged_dirs[i], l) == 0
+ && (path[l] == 0 || path[l] == '/'))
+ return true;
+ }
+ return false;
+}
+
+char *path_usrmerge_forhash(char *orig) {
+ char *prefix;
+ if (path_is_usrmerged_nostartslash(orig))
+ prefix = PREFIX "/";
+ else
+ prefix = "/";
+ char *r = nfmalloc(strlen(orig) + strlen(prefix) + 1);
+ strcpy(r, prefix);
+ strcpy(r + strlen(prefix), orig);
+ return r;
+}
+
+char *path_usrmerge_malloc(char *orig) {
+ if (orig[0] == '/' && path_is_usrmerged_nostartslash(orig+1))
+ return str_fmt("%s%s", PREFIX, orig);
+ return m_strdup(orig);
+}
+
+void path_disable_usrmerge(void) {
+ disable_usrmerge = true;
+}
diff --git a/lib/dpkg/path.h b/lib/dpkg/path.h
index 3479a8b3a..4356aaf7b 100644
--- a/lib/dpkg/path.h
+++ b/lib/dpkg/path.h
@@ -24,6 +24,7 @@
#include <sys/stat.h>
#include <stddef.h>
+#include <stdbool.h>
#include <dpkg/macros.h>
@@ -42,6 +43,11 @@ char *path_quote_filename(char *dst, const char *src, size_t size);
char *path_make_temp_template(const char *suffix);
+bool path_is_usrmerged_nostartslash(char *path);
+char *path_usrmerge_forhash(char *orig);
+char *path_usrmerge_malloc(char *orig);
+void path_disable_usrmerge(void);
+
int secure_unlink_statted(const char *pathname, const struct stat *stab);
int secure_unlink(const char *pathname);
int secure_remove(const char *pathname);
diff --git a/lib/dpkg/strhash.c b/lib/dpkg/strhash.c
index ed364695e..5e947f959 100644
--- a/lib/dpkg/strhash.c
+++ b/lib/dpkg/strhash.c
@@ -48,3 +48,11 @@ str_fnv_hash(const char *str)
return h;
}
+
+unsigned int str_fnv_hash_continue(char *str, unsigned int hashval) {
+ while (*str) {
+ hashval ^= *str++;
+ hashval *= FNV_MIXING_PRIME;
+ }
+ return hashval;
+}
diff --git a/lib/dpkg/string.h b/lib/dpkg/string.h
index 47ecd0487..1c001f116 100644
--- a/lib/dpkg/string.h
+++ b/lib/dpkg/string.h
@@ -55,6 +55,7 @@ str_is_set(const char *str)
bool str_match_end(const char *str, const char *end);
unsigned int str_fnv_hash(const char *str);
+unsigned int str_fnv_hash_continue(char *str, unsigned int hashval);
char *str_concat(char *dst, ...) DPKG_ATTR_SENTINEL;
char *str_fmt(const char *fmt, ...) DPKG_ATTR_PRINTF(1);
diff --git a/src/at/divert.at b/src/at/divert.at
index a2f03c48b..8ebe3c11f 100644
--- a/src/at/divert.at
+++ b/src/at/divert.at
@@ -105,7 +105,7 @@ dash
binutils-multiarch
])
-m4_define([di_dash], [diversion of /bin/sh to /bin/sh.distrib by dash
+m4_define([di_dash], [diversion of /usr/bin/sh to /usr/bin/sh.distrib by dash
])
m4_define([di_dashman],
[diversion of /usr/share/man/man1/sh.1.gz to /usr/share/man/man1/sh.distrib.1.gz by dash
@@ -113,21 +113,21 @@ m4_define([di_dashman],
m4_define([di_nm],
[diversion of /usr/bin/nm to /usr/bin/nm.single by binutils-multiarch
])
-m4_define([all_di], [m4_join([], di_nm, di_dashman, di_dash)])
+m4_define([all_di], [m4_join([], di_nm, di_dash, di_dashman)])
AT_CHECK([DPKG_DIVERT --list], [], all_di)
AT_CHECK([DPKG_DIVERT --list '*'], [], all_di)
AT_CHECK([DPKG_DIVERT --list ''])
-AT_CHECK([DPKG_DIVERT --list '???????'], [], di_dash)
+AT_CHECK([DPKG_DIVERT --list '???????????'], [], m4_join([], di_nm, di_dash))
AT_CHECK([DPKG_DIVERT --list '*/sh'], [], di_dash)
-AT_CHECK([DPKG_DIVERT --list '/bin/*'], [], di_dash)
+AT_CHECK([DPKG_DIVERT --list '/bin/*'], [], m4_join([], di_nm, di_dash))
AT_CHECK([DPKG_DIVERT --list binutils-multiarch], [], di_nm)
AT_CHECK([DPKG_DIVERT --list /bin/sh], [], di_dash)
AT_CHECK([DPKG_DIVERT --list -- /bin/sh], [], di_dash)
AT_CHECK([DPKG_DIVERT --list /usr/bin/nm.single], [], di_nm)
AT_CHECK([DPKG_DIVERT --list /bin/sh /usr/share/man/man1/sh.1.gz], [],
- [m4_join([], di_dashman, di_dash)])
+ [m4_join([], di_dash, di_dashman)])
AT_CLEANUP
@@ -148,11 +148,11 @@ AT_CHECK([DPKG_DIVERT --listpackage /bin/true], [], [LOCAL
])
AT_CHECK([DPKG_DIVERT --listpackage /bin/false])
-AT_CHECK([DPKG_DIVERT --truename /bin/sh], [], [/bin/sh.distrib
+AT_CHECK([DPKG_DIVERT --truename /bin/sh], [], [/usr/bin/sh.distrib
])
-AT_CHECK([DPKG_DIVERT --truename /bin/sh.distrib], [], [/bin/sh.distrib
+AT_CHECK([DPKG_DIVERT --truename /bin/sh.distrib], [], [/usr/bin/sh.distrib
])
-AT_CHECK([DPKG_DIVERT --truename /bin/something], [], [/bin/something
+AT_CHECK([DPKG_DIVERT --truename /bin/something], [], [/usr/bin/something
])
AT_CLEANUP
diff --git a/src/divert/main.c b/src/divert/main.c
index dae3ba227..9dc8e5070 100644
--- a/src/divert/main.c
+++ b/src/divert/main.c
@@ -46,6 +46,8 @@
#include <dpkg/buffer.h>
#include <dpkg/options.h>
#include <dpkg/db-fsys.h>
+#include <dpkg/path.h>
+#include <dpkg/file.h>
static const char printforhelp[] = N_(
@@ -697,7 +699,7 @@ diversion_list(const char *const *argv)
const char *pattern;
while ((pattern = *argv++))
- glob_list_prepend(&glob_list, m_strdup(pattern));
+ glob_list_prepend(&glob_list, path_usrmerge_malloc(pattern));
if (glob_list == NULL)
glob_list_prepend(&glob_list, m_strdup("*"));
@@ -735,7 +737,7 @@ diversion_list(const char *const *argv)
static int
diversion_truename(const char *const *argv)
{
- const char *filename = argv[0];
+ char *filename = argv[0];
struct fsys_namenode *namenode;
if (!filename || argv[1])
@@ -743,6 +745,7 @@ diversion_truename(const char *const *argv)
diversion_check_filename(filename);
+ filename = path_usrmerge_malloc(filename);
namenode = fsys_hash_find_node(filename, FHFF_NONE);
/* Print the given name if file is not diverted. */
@@ -751,13 +754,14 @@ diversion_truename(const char *const *argv)
else
printf("%s\n", filename);
+ free(filename);
return 0;
}
static int
diversion_listpackage(const char *const *argv)
{
- const char *filename = argv[0];
+ char *filename = argv[0];
struct fsys_namenode *namenode;
if (!filename || argv[1])
@@ -765,6 +769,7 @@ diversion_listpackage(const char *const *argv)
diversion_check_filename(filename);
+ filename = path_usrmerge_malloc(filename);
namenode = fsys_hash_find_node(filename, FHFF_NONE);
/* Print nothing if file is not diverted. */
@@ -778,6 +783,7 @@ diversion_listpackage(const char *const *argv)
else
printf("%s\n", namenode->divert->pkgset->name);
+ free(filename);
return 0;
}
@@ -855,6 +861,9 @@ main(int argc, const char * const *argv)
instdir = dpkg_fsys_set_dir(instdir);
admindir = dpkg_db_set_dir(admindir);
+ if (file_exists(dpkg_db_get_path("DISABLE_USRMERGE_LOGIC")))
+ path_disable_usrmerge();
+
env_pkgname = getenv("DPKG_MAINTSCRIPT_PACKAGE");
if (opt_pkgname_match_any && env_pkgname)
set_package(NULL, env_pkgname);
diff --git a/src/main/main.c b/src/main/main.c
index 6b43e3e15..9f6ebadda 100644
--- a/src/main/main.c
+++ b/src/main/main.c
@@ -51,6 +51,8 @@
#include <dpkg/pager.h>
#include <dpkg/options.h>
#include <dpkg/db-fsys.h>
+#include <dpkg/path.h>
+#include <dpkg/file.h>
#include "main.h"
#include "filters.h"
@@ -775,6 +777,9 @@ int main(int argc, const char *const *argv) {
ohshite(_("unable to setenv for subprocesses"));
free(force_string);
+ if (file_exists(dpkg_db_get_path("DISABLE_USRMERGE_LOGIC")))
+ path_disable_usrmerge();
+
if (!f_triggers)
f_triggers = (cipaction->arg_int == act_triggers && *argv) ? -1 : 1;
diff --git a/src/query/main.c b/src/query/main.c
index 6e3fe51ef..6898818c0 100644
--- a/src/query/main.c
+++ b/src/query/main.c
@@ -55,6 +55,7 @@
#include <dpkg/options.h>
#include <dpkg/db-ctrl.h>
#include <dpkg/db-fsys.h>
+#include <dpkg/file.h>
#include "actions.h"
@@ -336,7 +337,7 @@ searchfiles(const char *const *argv)
{
struct fsys_namenode *namenode;
struct fsys_hash_iter *iter;
- const char *thisarg;
+ char *thisarg;
int found;
int failures = 0;
struct varbuf path = VARBUF_INIT;
@@ -350,6 +351,7 @@ searchfiles(const char *const *argv)
ensure_diversions();
while ((thisarg = *argv++) != NULL) {
+ thisarg = path_usrmerge_malloc(thisarg);
found= 0;
if (!strchr("*[?/",*thisarg)) {
@@ -385,6 +387,7 @@ searchfiles(const char *const *argv)
} else {
m_output(stdout, _("<standard output>"));
}
+ free(thisarg);
}
modstatdb_shutdown();
@@ -880,6 +883,9 @@ int main(int argc, const char *const *argv) {
instdir = dpkg_fsys_set_dir(instdir);
admindir = dpkg_db_set_dir(admindir);
+ if (file_exists(dpkg_db_get_path("DISABLE_USRMERGE_LOGIC")))
+ path_disable_usrmerge();
+
if (!cipaction) badusage(_("need an action option"));
ret = cipaction->action(argv);
diff --git a/src/statoverride/main.c b/src/statoverride/main.c
index 1b3c998d4..35a92645b 100644
--- a/src/statoverride/main.c
+++ b/src/statoverride/main.c
@@ -46,6 +46,7 @@
#include <dpkg/glob.h>
#include <dpkg/db-fsys.h>
#include <dpkg/options.h>
+#include <dpkg/file.h>
#include "force.h"
#include "actions.h"
@@ -349,8 +350,10 @@ statoverride_list(const char *const *argv)
while ((thisarg = *argv++)) {
char *pattern = path_cleanup(thisarg);
+ char *pattern2 = path_usrmerge_malloc(pattern);
+ free(pattern);
- glob_list_prepend(&glob_list, pattern);
+ glob_list_prepend(&glob_list, pattern2);
}
if (glob_list == NULL)
glob_list_prepend(&glob_list, m_strdup("*"));
@@ -413,6 +416,8 @@ main(int argc, const char *const *argv)
instdir = dpkg_fsys_set_dir(instdir);
admindir = dpkg_db_set_dir(admindir);
+ if (file_exists(dpkg_db_get_path("DISABLE_USRMERGE_LOGIC")))
+ path_disable_usrmerge();
if (!cipaction)
badusage(_("need an action option"));
diff --git a/src/trigger/main.c b/src/trigger/main.c
index e7d589644..105d2a388 100644
--- a/src/trigger/main.c
+++ b/src/trigger/main.c
@@ -40,6 +40,8 @@
#include <dpkg/trigdeferred.h>
#include <dpkg/triglib.h>
#include <dpkg/pkg-spec.h>
+#include <dpkg/file.h>
+#include <dpkg/path.h>
static const char printforhelp[] = N_(
"Type dpkg-trigger --help for help about this utility.");
@@ -225,6 +227,10 @@ main(int argc, const char *const *argv)
instdir = dpkg_fsys_set_dir(instdir);
admindir = dpkg_db_set_dir(admindir);
+ // Not sure if this matters for this program, shouldn't hurt...
+ if (file_exists(dpkg_db_get_path("DISABLE_USRMERGE_LOGIC")))
+ path_disable_usrmerge();
+
if (f_check) {
if (*argv)
badusage(_("--%s takes no arguments"),
signature.asc
Description: This is a digitally signed message part
