Hello community, here is the log from the commit of package coreutils for openSUSE:Factory checked in at 2016-08-05 18:11:47 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/coreutils (Old) and /work/SRC/openSUSE:Factory/.coreutils.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "coreutils" Changes: -------- --- /work/SRC/openSUSE:Factory/coreutils/coreutils-testsuite.changes 2016-07-18 21:17:11.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.coreutils.new/coreutils-testsuite.changes 2016-08-05 18:11:48.000000000 +0200 @@ -1,0 +2,13 @@ +Thu Jul 28 17:02:53 UTC 2016 - m...@bernhard-voelker.de + +- coreutils-maint-fix-dependency-of-man-arch.1.patch: Add Upstream + patch to fix the build dependency between src/arch -> man/arch.1 + which lead to spurious build failures. +- coreutils-df-hash-in-filter.patch: Refresh with -p0. + +------------------------------------------------------------------- +Fri Jul 22 10:48:50 CEST 2016 - p...@suse.de + +- Add coreutils-df-hash-in-filter.patch that speeds up df. + +------------------------------------------------------------------- coreutils.changes: same change New: ---- coreutils-df-hash-in-filter.patch coreutils-maint-fix-dependency-of-man-arch.1.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ coreutils-testsuite.spec ++++++ --- /var/tmp/diff_new_pack.Tg8xN0/_old 2016-08-05 18:11:50.000000000 +0200 +++ /var/tmp/diff_new_pack.Tg8xN0/_new 2016-08-05 18:11:50.000000000 +0200 @@ -132,6 +132,14 @@ # to be removed with coreutils > v8.25. Patch700: coreutils-diagnose-fts-readdir-failure.patch +# Upstream patch to improve df performance with many mount points; +# to be removed with coreutils > v8.25. +Patch705: coreutils-df-hash-in-filter.patch + +# Upstream patch to fix the build dependency between src/arch -> man/arch.1; +# to be removed with coreutils > v8.25. +Patch710: coreutils-maint-fix-dependency-of-man-arch.1.patch + # ================================================ %description These are the GNU core utilities. This package is the union of @@ -175,6 +183,8 @@ %patch501 %patch700 +%patch705 +%patch710 #???## We need to statically link to gmp, otherwise we have a build loop #???#sed -i s,'$(LIB_GMP)',%%{_libdir}/libgmp.a,g Makefile.in ++++++ coreutils.spec ++++++ --- /var/tmp/diff_new_pack.Tg8xN0/_old 2016-08-05 18:11:50.000000000 +0200 +++ /var/tmp/diff_new_pack.Tg8xN0/_new 2016-08-05 18:11:50.000000000 +0200 @@ -132,6 +132,14 @@ # to be removed with coreutils > v8.25. Patch700: coreutils-diagnose-fts-readdir-failure.patch +# Upstream patch to improve df performance with many mount points; +# to be removed with coreutils > v8.25. +Patch705: coreutils-df-hash-in-filter.patch + +# Upstream patch to fix the build dependency between src/arch -> man/arch.1; +# to be removed with coreutils > v8.25. +Patch710: coreutils-maint-fix-dependency-of-man-arch.1.patch + # ================================================ %description These are the GNU core utilities. This package is the union of @@ -175,6 +183,8 @@ %patch501 %patch700 +%patch705 +%patch710 #???## We need to statically link to gmp, otherwise we have a build loop #???#sed -i s,'$(LIB_GMP)',%%{_libdir}/libgmp.a,g Makefile.in ++++++ coreutils-df-hash-in-filter.patch ++++++ # Upstream patch to improve df performance with many mount points; # to be removed with coreutils > v8.25. Upstream patch: http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=1c17f61ef9 Downstream addition: * THANKS: Propagate the addition of Josef Cejka from THANKS.in to this file (usually done via "make dist"). ______________________________________________________________________ >From 1c17f61ef993a5ee5fb0d3bc47b7b25782ae386c Mon Sep 17 00:00:00 2001 From: Philipp Thomas <p...@suse.de> Date: Sun, 31 Jul 2016 21:24:18 +0200 Subject: [PATCH] df: improve performance with many mount points Use hash table for seaching in filter_mount_list() and get_dev() This improves performance for 20K mount entries from: real 0m1.731s user 0m0.532s sys 0m1.188s to: real 0m1.066s user 0m0.028s sys 0m1.032s * src/df.c (devlist_table): Define hash table. (devlist_hash): Add hash function. (devlist_compare): Add hash comparison function. (devlist_for_dev): Add lookup function. (devlist_free): Add cleanup function. (filter_mount_list): Use the above hash table. While at it, rename the variable 'devlist' to 'seen_dev' for better readability. (me_for_dev): Use the above lookup function. NEWS: Mention the improvement. THANKS.in: Remove the committer; add original submitter Josef Cejka. --- NEWS | 3 + THANKS | 1 THANKS.in | 2 - src/df.c | 110 +++++++++++++++++++++++++++++++++++++++++++------------------- 4 files changed, 82 insertions(+), 34 deletions(-) Index: NEWS =================================================================== --- NEWS.orig +++ NEWS @@ -8,6 +8,9 @@ GNU coreutils NEWS introduced in coreutils-8.0. du, chmod, chgrp and chown started using fts in 6.0. chcon was added in coreutils-6.9.91 with fts support. ] + df now filters the system mount list more efficiently, with 20000 + mount entries now being processed in about 1.1s compared to 1.7s. + * Noteworthy changes in release 8.25 (2016-01-20) [stable] Index: THANKS.in =================================================================== --- THANKS.in.orig +++ THANKS.in @@ -322,6 +322,7 @@ Jon Peatfield J.S. Joost van Baal joos...@xs4all.nl Jordi Pujol jordipuj...@gmail.com Jorge Stolfi sto...@ic.unicamp.br +Josef Cejka jce...@suse.com Joseph D. Wagner j...@josephdwagner.info Joseph S. Myers js...@cam.ac.uk Josh Triplett j...@freedesktop.org @@ -514,7 +515,6 @@ Petter Reinholdtsen pere Phelippe Neveu pne...@pcigeomatics.com Phil Richards phil.richa...@vf.vodafone.co.uk Philipp Gortan gor...@gmail.com -Philipp Thomas p...@suse.de Philippe De Muyter p...@macqel.be Philippe Schnoebelen philippe.schnoebe...@imag.fr Phillip Jones mo...@datastacks.com Index: src/df.c =================================================================== --- src/df.c.orig +++ src/df.c @@ -34,6 +34,7 @@ #include "mountlist.h" #include "quote.h" #include "find-mount-point.h" +#include "hash.h" /* The official name of this program (e.g., no 'g' prefix). */ #define PROGRAM_NAME "df" @@ -43,14 +44,16 @@ proper_name ("David MacKenzie"), \ proper_name ("Paul Eggert") -/* Filled with device numbers of examined file systems to avoid - duplicates in output. */ -static struct devlist +struct devlist { dev_t dev_num; struct mount_entry *me; struct devlist *next; -} *device_list; +}; + +/* Filled with device numbers of examined file systems to avoid + duplicates in output. */ +static Hash_table *devlist_table; /* If true, show even file systems with zero size or uninteresting types. */ @@ -603,23 +606,67 @@ excluded_fstype (const char *fstype) return false; } +static size_t +devlist_hash (void const *x, size_t table_size) +{ + struct devlist const *p = x; + return (uintmax_t) p->dev_num % table_size; +} + +static bool +devlist_compare (void const *x, void const *y) +{ + struct devlist const *a = x; + struct devlist const *b = y; + return a->dev_num == b->dev_num; +} + +static struct devlist * +devlist_for_dev (dev_t dev) +{ + if (devlist_table == NULL) + return NULL; + struct devlist dev_entry; + dev_entry.dev_num = dev; + return hash_lookup (devlist_table, &dev_entry); +} + +static void +devlist_free (void *p) +{ + free (p); +} + /* Filter mount list by skipping duplicate entries. In the case of duplicates - based on the device number - the mount entry with a '/' in its me_devname (i.e., not pseudo name like tmpfs) wins. If both have a real devname (e.g. bind mounts), then that with the shorter me_mountdir wins. With DEVICES_ONLY == true (set with df -a), only update - the global device_list, rather than filtering the global mount_list. */ + the global devlist_table, rather than filtering the global mount_list. */ static void filter_mount_list (bool devices_only) { struct mount_entry *me; + /* Temporary list to keep entries ordered. */ + struct devlist *device_list = NULL; + int mount_list_size = 0; + + for (me = mount_list; me; me = me->me_next) + mount_list_size++; + + devlist_table = hash_initialize (mount_list_size, NULL, + devlist_hash, + devlist_compare, + devlist_free); + if (devlist_table == NULL) + xalloc_die (); + /* Sort all 'wanted' entries into the list device_list. */ for (me = mount_list; me;) { struct stat buf; - struct devlist *devlist; struct mount_entry *discard_me = NULL; /* Avoid stating remote file systems as that may hang. @@ -635,21 +682,20 @@ filter_mount_list (bool devices_only) else { /* If we've already seen this device... */ - for (devlist = device_list; devlist; devlist = devlist->next) - if (devlist->dev_num == buf.st_dev) - break; + struct devlist *seen_dev = devlist_for_dev (buf.st_dev); - if (devlist) + if (seen_dev) { - bool target_nearer_root = strlen (devlist->me->me_mountdir) + bool target_nearer_root = strlen (seen_dev->me->me_mountdir) > strlen (me->me_mountdir); /* With bind mounts, prefer items nearer the root of the source */ - bool source_below_root = devlist->me->me_mntroot != NULL + bool source_below_root = seen_dev->me->me_mntroot != NULL && me->me_mntroot != NULL - && (strlen (devlist->me->me_mntroot) + && (strlen (seen_dev->me->me_mntroot) < strlen (me->me_mntroot)); - if (! print_grand_total && me->me_remote && devlist->me->me_remote - && ! STREQ (devlist->me->me_devname, me->me_devname)) + if (! print_grand_total + && me->me_remote && seen_dev->me->me_remote + && ! STREQ (seen_dev->me->me_devname, me->me_devname)) { /* Don't discard remote entries with different locations, as these are more likely to be explicitly mounted. @@ -658,21 +704,21 @@ filter_mount_list (bool devices_only) } else if ((strchr (me->me_devname, '/') /* let "real" devices with '/' in the name win. */ - && ! strchr (devlist->me->me_devname, '/')) + && ! strchr (seen_dev->me->me_devname, '/')) /* let points towards the root of the device win. */ || (target_nearer_root && ! source_below_root) /* let an entry overmounted on a new device win... */ - || (! STREQ (devlist->me->me_devname, me->me_devname) + || (! STREQ (seen_dev->me->me_devname, me->me_devname) /* ... but only when matching an existing mnt point, to avoid problematic replacement when given inaccurate mount lists, seen with some chroot environments for example. */ && STREQ (me->me_mountdir, - devlist->me->me_mountdir))) + seen_dev->me->me_mountdir))) { /* Discard mount entry for existing device. */ - discard_me = devlist->me; - devlist->me = me; + discard_me = seen_dev->me; + seen_dev->me = me; } else { @@ -691,12 +737,14 @@ filter_mount_list (bool devices_only) } else { - /* Add the device number to the global list devlist. */ - devlist = xmalloc (sizeof *devlist); + /* Add the device number to the device_table. */ + struct devlist *devlist = xmalloc (sizeof *devlist); devlist->me = me; devlist->dev_num = buf.st_dev; devlist->next = device_list; device_list = devlist; + if (hash_insert (devlist_table, devlist) == NULL) + xalloc_die (); me = me->me_next; } @@ -711,28 +759,24 @@ filter_mount_list (bool devices_only) me = device_list->me; me->me_next = mount_list; mount_list = me; - /* Free devlist entry and advance. */ - struct devlist *devlist = device_list->next; - free (device_list); - device_list = devlist; + device_list = device_list->next; } + + hash_free (devlist_table); + devlist_table = NULL; } } + /* Search a mount entry list for device id DEV. Return the corresponding mount entry if found or NULL if not. */ static struct mount_entry const * _GL_ATTRIBUTE_PURE me_for_dev (dev_t dev) { - struct devlist *dl = device_list; - - while (dl) - { - if (dl->dev_num == dev) + struct devlist *dl = devlist_for_dev (dev); + if (dl) return dl->me; - dl = dl->next; - } return NULL; } Index: THANKS =================================================================== --- THANKS.orig +++ THANKS @@ -406,6 +406,7 @@ Jon Ringuette jonr Joost van Baal joos...@xs4all.nl Jordi Pujol jordipuj...@gmail.com Jorge Stolfi sto...@ic.unicamp.br +Josef Cejka jce...@suse.com Joseph D. Wagner j...@josephdwagner.info Joseph S. Myers js...@cam.ac.uk Josh Triplett j...@freedesktop.org ++++++ coreutils-maint-fix-dependency-of-man-arch.1.patch ++++++ Upstream patch to fix the build dependency between src/arch -> man/arch.1; to be removed with coreutils > v8.25. Downstream addition: * Propagate the change from local.mk to Makefile.in, which is done upstreams by autoconf. Original patch: http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=a69e54cfdf7e >From a69e54cfdf7e5d8c2c3fe315ded649272d7e8711 Mon Sep 17 00:00:00 2001 From: Bernhard Voelker <m...@bernhard-voelker.de> Date: Tue, 26 Jul 2016 20:58:08 +0200 Subject: [PATCH] maint: fix dependency of man/arch.1 The following is a reproducer for the wrong dependency: $ ./configure --enable-install-program=arch $ make $ rm -f src/arch man/arch.1 $ make man/arch.1 GEN man/arch.1 help2man: can't get `--help' info from man/arch.td/arch Try `--no-discard-stderr' if option outputs to stderr Makefile:14378: recipe for target 'man/arch.1' failed make: *** [man/arch.1] Error 127 * man/local.mk (man/arch.1): Change to depend on src/arch rather than src/uname: while the arch binary depends on uname.c and uname-arch.c, its man page depends on the arch binary. Reported downstream by Rodrigues Goldwyn <rgold...@suse.com> in https://build.opensuse.org/request/show/415172 --- Makefile.in | 2 +- man/local.mk | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) Index: man/local.mk =================================================================== --- man/local.mk.orig +++ man/local.mk @@ -56,10 +56,10 @@ if SINGLE_BINARY mandeps += src/coreutils$(EXEEXT) else # Most prog.1 man pages depend on src/prog. List the exceptions: -man/arch.1: src/uname$(EXEEXT) man/install.1: src/ginstall$(EXEEXT) man/test.1: src/[$(EXEEXT) +man/arch.1: src/arch$(EXEEXT) man/base32.1: src/base32$(EXEEXT) man/base64.1: src/base64$(EXEEXT) man/basename.1: src/basename$(EXEEXT) Index: Makefile.in =================================================================== --- Makefile.in.orig +++ Makefile.in @@ -14276,10 +14276,10 @@ distclean-local: $(ALL_MANS): $(mandeps) # Most prog.1 man pages depend on src/prog. List the exceptions: -@SINGLE_BINARY_FALSE@man/arch.1: src/uname$(EXEEXT) @SINGLE_BINARY_FALSE@man/install.1: src/ginstall$(EXEEXT) @SINGLE_BINARY_FALSE@man/test.1: src/[$(EXEEXT) +@SINGLE_BINARY_FALSE@man/arch.1: src/arch$(EXEEXT) @SINGLE_BINARY_FALSE@man/base32.1: src/base32$(EXEEXT) @SINGLE_BINARY_FALSE@man/base64.1: src/base64$(EXEEXT) @SINGLE_BINARY_FALSE@man/basename.1: src/basename$(EXEEXT)