Re: [bug-gnulib] new module mpsort for faster sorting

2007-01-29 Thread Bruno Haible
Hello Paul,

 Such a function can lead to faster execution
 in some important cases, as I have verified using GNU 'ls' as a guinea pig.

Does the speedup come from the use of the mergesort algorithm, or from
avoiding a function call in the comparison function (with qsort, the
pointer indirection must be done inside the comparison function; here it's
done outside)?

Bruno




Re: new module mpsort for faster sorting

2007-01-29 Thread Jim Meyering
Bruno Haible [EMAIL PROTECTED] wrote:
 Hello Paul,

 Such a function can lead to faster execution
 in some important cases, as I have verified using GNU 'ls' as a guinea pig.

 Does the speedup come from the use of the mergesort algorithm, or from
 avoiding a function call in the comparison function (with qsort, the
 pointer indirection must be done inside the comparison function; here it's
 done outside)?

Hi Bruno,

When sorting records larger than a pointer, reduced data movement is
likely to be the overriding factor: better use of cache.
I.e., setting up the array of pointers costs just O(N) time and memory,
but you save O(N log(N)) time in reduced data copying costs, because
mpsort is swapping only pointers, whereas qsort swaps the entire (larger)
records.

So, for small N and/or very small record size, there may be
a small run-time performance decrease, but for all other cases,
there should be an improvement.

Based on my minimal experience, I suspect that GNU sort will
end up using mpsort, too.




Re: new module mpsort for faster sorting

2007-01-29 Thread Bruno Haible
Jim Meyering wrote:
 When sorting records larger than a pointer, reduced data movement is
 likely to be the overriding factor: better use of cache.
 I.e., setting up the array of pointers costs just O(N) time and memory,
 but you save O(N log(N)) time in reduced data copying costs, because
 mpsort is swapping only pointers, whereas qsort swaps the entire (larger)
 records.

This is undoubtable.

My question is: Once you have switched from an array representation with
sizeof (element)  sizeof (void *) to an array representation with
sizeof (element) == sizeof (void *), you still have 3 ways to sort:

  a) with qsort and a comparison function that does
 int cmp (void **p, void **q) { return element_cmp (*p, *q); }

  b) with a mergesort implementation that looks like mpsort, but calls
   cmp (ba, bb)
 instead of
   cmp (ba, bb)
 and instead does the indirection in the comparison function:
 int cmp (void **p, void **q) { return element_cmp (*p, *q); }

  c) with the mpsort function as it is, and element_cmp as the comparison
 function.

Does the major speedup come from the transition a) - b), or from the
transition b) - c) ?

Bruno




new module mpsort for faster sorting

2007-01-28 Thread Paul Eggert
For quite some time Djamel Belazzougui has been suggesting speed
improvements to glibc's qsort function.  I'm not sure yet that this is
a good idea, but it does seem to me that there is use for a function
that can sort a vector of pointers to data (as opposed to qsort, which
sorts an array of data).  Such a function can lead to faster execution
in some important cases, as I have verified using GNU 'ls' as a guinea
pig.  So I'm adding this module to gnulib, and plan to submit a patch
to coreutils 'ls' shortly.

2007-01-28  Paul Eggert  [EMAIL PROTECTED]

* MODULES.html.sh: New module mpsort.
* lib/mpsort.c, lib/mpsort.h, m4/mpsort.m4, modules/mpsort: New files.

* lib/regex.h (_Restrict_): Renamed from __restrict, to avoid
a circularity problem with HP-UX ia64 reported by Bob Proulx in
http://lists.gnu.org/archive/html/bug-gnulib/2007-01/msg00394.html.
All uses changed.
(_Restrict_arr_): Renamed from __restrict_arr, for similar reasons.
All uses changed.
* lib/regcomp.c, lib/regexec.c: Change all uses from __restrict
to _Restrict_.
* lib/regexec.c (regexec): Declare pmatch with _Restrict_arr_, so that
the parameter matches the prototype.

Index: MODULES.html.sh
===
RCS file: /cvsroot/gnulib/gnulib/MODULES.html.sh,v
retrieving revision 1.177
diff -u -p -r1.177 MODULES.html.sh
--- MODULES.html.sh 27 Jan 2007 01:05:04 -  1.177
+++ MODULES.html.sh 29 Jan 2007 07:23:18 -
@@ -1540,6 +1540,16 @@ func_all_modules ()
   func_module pagealign_alloc
   func_end_table

+  element=Sorting functions stdlib.h
+  element=`printf %s $element | sed -e $sed_lt -e $sed_gt`
+  func_section_wrap ansic_enh_stdlib_sorting
+  func_wrap H3
+  func_echo $element
+
+  func_begin_table
+  func_module mpsort
+  func_end_table
+
   element=Date and time time.h
   element=`printf %s $element | sed -e $sed_lt -e $sed_gt`
   func_section_wrap ansic_enh_time_datetime
Index: lib/mpsort.c
===
RCS file: lib/mpsort.c
diff -N lib/mpsort.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ lib/mpsort.c29 Jan 2007 07:23:18 -
@@ -0,0 +1,157 @@
+/* Sort a vector of pointers to data.
+
+   Copyright (C) 2007 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 2, or (at your option)
+   any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License along
+   with this program; if not, write to the Free Software Foundation,
+   Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.  */
+
+/* Written by Paul Eggert.  */
+
+#include config.h
+
+#include mpsort.h
+
+#include string.h
+
+/* The type of qsort-style comparison functions.  */
+
+typedef int (*comparison_function) (void const *, void const *);
+
+static void mpsort_with_tmp (void const **restrict, size_t,
+void const **restrict, comparison_function);
+
+/* Sort a vector BASE containing N pointers, placing the sorted array
+   into TMP.  Compare pointers with CMP.  N must be at least 2.  */
+
+static void
+mpsort_into_tmp (void const **restrict base, size_t n,
+void const **restrict tmp,
+comparison_function cmp)
+{
+  size_t n1 = n / 2;
+  size_t n2 = n - n1;
+  size_t a = 0;
+  size_t alim = n1;
+  size_t b = n1;
+  size_t blim = n;
+  void const *ba;
+  void const *bb;
+
+  mpsort_with_tmp (base + n1, n2, tmp, cmp);
+  mpsort_with_tmp (base, n1, tmp, cmp);
+
+  ba = base[a];
+  bb = base[b];
+
+  for (;;)
+if (cmp (ba, bb) = 0)
+  {
+   *tmp++ = ba;
+   a++;
+   if (a == alim)
+ {
+   a = b;
+   alim = blim;
+   break;
+ }
+   ba = base[a];
+  }
+else
+  {
+   *tmp++ = bb;
+   b++;
+   if (b == blim)
+ break;
+   bb = base[b];
+  }
+
+  memcpy (tmp, base + a, (alim - a) * sizeof *base);
+}
+
+/* Sort a vector BASE containing N pointers, in place.  Use TMP
+   (containing N / 2 pointers) for temporary storage.  Compare
+   pointers with CMP.  */
+
+static void
+mpsort_with_tmp (void const **restrict base, size_t n,
+void const **restrict tmp,
+comparison_function cmp)
+{
+  if (n = 2)
+{
+  if (n == 2)
+   {
+ void const *p0 = base[0];
+ void const *p1 = base[1];
+ if (! (cmp (p0, p1) = 0))
+   {
+ base[0] = p1;
+ base[1] = p0;