Re: optimize lt_combine
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Gary V. Vaughan on 5/16/2008 11:24 AM: | | Yes, please apply! | | * libltdl/m4/ltsugar.m4 (lt_combine): Mirror Autoconf 2.62 | improvements. Applied. - -- Don't work too hard, make some time for fun as well! Eric Blake [EMAIL PROTECTED] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkgu/fYACgkQ84KuGfSFAYBfugCfbeQa+fWTt1yQqlEKg0Awz4OP N4MAoIvlHA8WhCRE7BTKZJ+shnrUbGAk =wouP -END PGP SIGNATURE-
Re: optimize lt_combine
Hi Eric, On 14 May 2008, at 17:04, Eric Blake wrote: Revisiting an old thread: http://lists.gnu.org/archive/html/libtool-patches/2007-10/ msg00016.html Thanks for the ping :-) The patch was previously deferred because 2.2 was so close to release; now that it is out, is this okay to apply? so close == 87.5% of the way through the 4 years it took to get there :-) Yes, please apply! * libltdl/m4/ltsugar.m4 (lt_combine): Mirror Autoconf 2.62 improvements. Includes a semantic change where the separator can now be empty, and where an empty fourth argument is now treated as a valid suffix. * libltdl/m4/libtool.m4 (_lt_decl_varnames_tagged): Adjust to new semantics of lt_combine. (lt_decl_varnames_tagged): Fix quoting and optimize. Cheers, Gary -- ())_. Email me: [EMAIL PROTECTED] ( '/ Read my blog: http://blog.azazil.net / )= ...and my book: http://sources.redhat.com/autobook `(_~)_ PGP.sig Description: This is a digitally signed message part
optimize lt_combine
Revisiting an old thread: http://lists.gnu.org/archive/html/libtool-patches/2007-10/msg00016.html Here's an optimization to lt_combine, as well as its current clients to match the slight semantic changes. If we ever start requiring autoconf 2.62, we can then s/lt_combine/m4_combine/ without impact. I've updated the patch with the conceptual m4sugar improvements that have been made in the meantime, while still avoiding m4sugar constructs that were not available prior to 2.62. Some statistics: pre-patch: $ rm -Rf autom4te.cache $ autoconf --trace lt_combine | wc 3 31715 $ time autoconf --trace m4_shift | wc 23859 289406 20200516 real1m18.914s user1m16.929s sys 0m1.168s $ wc autom4te.cache/traces.0 81870 1123343 23163745 autom4te.cache/traces.0 post-patch: $ rm -Rf autom4te.cache $ time autoconf --trace m4_shift | wc 20974 281283 16654686 real1m3.968s user1m2.462s sys 0m1.288s $ wc autom4te.cache/traces.0 79178 948394 19087599 autom4te.cache/traces.0 Tracing m4_shift is inherently slow (mostly I/O time in generating the trace file), since it is used so heavily ;) But these numbers show that autoconf expands roughly 3000 fewer m4_shift calls (not to mention other macros that weren't traced), and shaves off more than 4 megabytes of parsing with m4 1.4.x (it is not until m4 1.6 that shift is optimized to avoid reparsing everything), just by optimizing the three lt_combine calls that occur during libtool's configure.ac (not to mention the subdirectories that also run autoconf during bootstrap time). The patch was previously deferred because 2.2 was so close to release; now that it is out, is this okay to apply? From 9bdee069949f55656d0c8bf47914ea809e498592 Mon Sep 17 00:00:00 2001 From: Eric Blake [EMAIL PROTECTED] Date: Wed, 16 Apr 2008 14:48:47 -0600 Subject: [PATCH] Speed up bootstrap by improving lt_combine. * libltdl/m4/ltsugar.m4 (lt_combine): Mirror Autoconf 2.62 improvements. Includes a semantic change where the separator can now be empty, and where an empty fourth argument is now treated as a valid suffix. * libltdl/m4/libtool.m4 (_lt_decl_varnames_tagged): Adjust to new semantics of lt_combine. (lt_decl_varnames_tagged): Fix quoting and optimize. Signed-off-by: Eric Blake [EMAIL PROTECTED] --- ChangeLog | 11 +++ libltdl/m4/libtool.m4 | 12 ++-- libltdl/m4/ltsugar.m4 | 22 +++--- 3 files changed, 28 insertions(+), 17 deletions(-) diff --git a/ChangeLog b/ChangeLog index e556e7c..58ebb19 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,14 @@ +2008-05-14 Eric Blake [EMAIL PROTECTED] + + Speed up bootstrap by improving lt_combine. + * libltdl/m4/ltsugar.m4 (lt_combine): Mirror Autoconf 2.62 + improvements. Includes a semantic change where the separator can + now be empty, and where an empty fourth argument is now treated as + a valid suffix. + * libltdl/m4/libtool.m4 (_lt_decl_varnames_tagged): Adjust to new + semantics of lt_combine. + (lt_decl_varnames_tagged): Fix quoting and optimize. + 2008-05-13 Eric Blake [EMAIL PROTECTED] Kill _LT_LIBSOURCES; it wasn't checking anything useful. diff --git a/libltdl/m4/libtool.m4 b/libltdl/m4/libtool.m4 index 4d31e80..1144395 100644 --- a/libltdl/m4/libtool.m4 +++ b/libltdl/m4/libtool.m4 @@ -380,12 +380,12 @@ m4_define([lt_decl_dquote_varnames], # lt_decl_varnames_tagged([SEPARATOR], [VARNAME1...]) # --- m4_define([lt_decl_varnames_tagged], -[_$0(m4_quote(m4_default([$1], [[, ]])), - m4_quote(m4_if([$2], [], -m4_quote(lt_decl_tag_varnames), - m4_quote(m4_shift($@, - m4_split(m4_normalize(m4_quote(_LT_TAGS]) -m4_define([_lt_decl_varnames_tagged], [lt_combine([$1], [$2], [_], $3)]) +[m4_assert([$# = 2])dnl +_$0(m4_quote(m4_default([$1], [[, ]])), +m4_ifval([$2], [[$2]], [m4_dquote(lt_decl_tag_varnames)]), +m4_split(m4_normalize(m4_quote(_LT_TAGS)), [ ]))]) +m4_define([_lt_decl_varnames_tagged], +[m4_ifval([$3], [lt_combine([$1], [$2], [_], $3)])]) # lt_decl_all_varnames([SEPARATOR], [VARNAME1...]) diff --git a/libltdl/m4/ltsugar.m4 b/libltdl/m4/ltsugar.m4 index 0d258e0..3e576d5 100644 --- a/libltdl/m4/ltsugar.m4 +++ b/libltdl/m4/ltsugar.m4 @@ -1,13 +1,13 @@ # ltsugar.m4 -- libtool m4 base layer. -*-Autoconf-*- # -# Copyright (C) 2004, 2005, 2007 Free Software Foundation, Inc. -# Written by Gary V. Vaughan, 2004 +# Copyright (C) 2004, 2005, 2007, 2008 Free Software Foundation, Inc. +# Written by Gary V. Vaughan, 2004 # # This file is free software; the Free Software Foundation gives # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. -# serial 5 ltsugar.m4 +# serial 6 ltsugar.m4 # This is to help aclocal find these macros, as it can't see
Re: optimize lt_combine
Hi Eric, On 17 Oct 2007, at 19:25, Eric Blake wrote: I liked the concept of lt_combine so much that I added m4_combine to m4sugar. In the process, Thanks :-) A while ago I considered shipping the useful stuff from m4sugar and ltsugar with m4 2.0... it might also give us some scope for speeding up some of the stickier implementations by moving them into a module in C. I made some optimizations and some slight semantic changes: - the separator is now used literally, rather than losing whitespace after comma - the infix argument is now used literally, rather than undergoing expansion (consider m4_define([_],[oops])...) - the quoting in m4_foreach is fixed (consider m4_define ([_Lt_suffix])...) - useless use of m4_car is avoided, and the m4_shiftn(3, $@) is now inlined and done once up front, rather than once per _Lt_prefix - the separator is no longer defaulted, so you can use an empty separator - the suffix list can now include an empty suffix, so the caller must be taught not to pass an empty suffix to preserve previous usage pattern + _lt_decl_varnames_tagged was the only caller for now, so I optimized it as well + right now, lt_decl_varnames_tagged is never called with more than one argument. It looks like it may have been intended to be called with multiple VARNAME arguments, rather than a single argument consisting of multiple VARNAMEs, but changing that is more invasive. So for now I just added an assert which will trigger if we ever change our minds, to remind us to fix the usage patterns appropriately. All good! OK to apply? I'm writing the testcase for the last release blocker today, and would rather hold off on this until we've rolled the release personally. But it is no more intrusive than the lt_join optimisations Ralf agreed to, so if someone else wants to overrule and let you apply, I won't be heartbroken :-) Cheers, Gary -- ())_. Email me: [EMAIL PROTECTED] ( '/ Read my blog: http://blog.azazil.net / )= ...and my book: http://sources.redhat.com/autobook `(_~)_ Join my AGLOCO Network: http://www.agloco.com/r/BBBS7912 PGP.sig Description: This is a digitally signed message part
optimize lt_combine
I liked the concept of lt_combine so much that I added m4_combine to m4sugar. In the process, I made some optimizations and some slight semantic changes: - the separator is now used literally, rather than losing whitespace after comma - the infix argument is now used literally, rather than undergoing expansion (consider m4_define([_],[oops])...) - the quoting in m4_foreach is fixed (consider m4_define([_Lt_suffix])...) - useless use of m4_car is avoided, and the m4_shiftn(3, $@) is now inlined and done once up front, rather than once per _Lt_prefix - the separator is no longer defaulted, so you can use an empty separator - the suffix list can now include an empty suffix, so the caller must be taught not to pass an empty suffix to preserve previous usage pattern + _lt_decl_varnames_tagged was the only caller for now, so I optimized it as well + right now, lt_decl_varnames_tagged is never called with more than one argument. It looks like it may have been intended to be called with multiple VARNAME arguments, rather than a single argument consisting of multiple VARNAMEs, but changing that is more invasive. So for now I just added an assert which will trigger if we ever change our minds, to remind us to fix the usage patterns appropriately. OK to apply? $ echo 'm4_debugmode(qt)lt_combine([, ],[a,b],[_],[1],[2])' | m4 -Ilib - Ilib/m4sugar m4sugar.m4 ltsugar.pre - 21 |wc 172 5163788 $ echo 'm4_debugmode(qt)lt_combine([, ],[a,b],[_],[1],[2])' | m4 -Ilib - Ilib/m4sugar m4sugar.m4 ltsugar.post - 21 |wc 101 3032241 $ echo 'm4_divert(0)lt_combine([, ],[a,b],[_],[1],[2])' | m4 -Ilib - Ilib/m4sugar m4sugar.m4 ltsugar.pre - a_1,a_2,b_1,b_2 $ echo 'm4_divert(0)lt_combine([, ],[a,b],[_],[1],[2])' | m4 -Ilib - Ilib/m4sugar m4sugar.m4 ltsugar.post - a_1, a_2, b_1, b_2 2007-10-17 Eric Blake [EMAIL PROTECTED] Speed up bootstrap by improving lt_combine. * libltdl/m4/ltsugar.m4 (lt_combine): Mirror Autoconf 2.62 improvements. Includes a semantic change where the separator can now be empty, and where an empty fourth argument is now treated as a valid suffix. * libltdl/m4/libtool.m4 (_lt_decl_varnames_tagged): Adjust to new semantics of lt_combine. (lt_decl_varnames_tagged): Fix quoting and optimize. Index: libltdl/m4/libtool.m4 === RCS file: /sources/libtool/libtool/libltdl/m4/libtool.m4,v retrieving revision 1.120 diff -u -p -r1.120 libtool.m4 --- libltdl/m4/libtool.m4 12 Oct 2007 20:54:44 - 1.120 +++ libltdl/m4/libtool.m4 17 Oct 2007 17:29:37 - @@ -367,12 +367,12 @@ m4_define([lt_decl_dquote_varnames], # lt_decl_varnames_tagged([SEPARATOR], [VARNAME1...]) # --- m4_define([lt_decl_varnames_tagged], -[_$0(m4_quote(m4_default([$1], [[, ]])), - m4_quote(m4_if([$2], [], -m4_quote(lt_decl_tag_varnames), - m4_quote(m4_shift($@, - m4_split(m4_normalize(m4_quote(_LT_TAGS]) -m4_define([_lt_decl_varnames_tagged], [lt_combine([$1], [$2], [_], $3)]) +[m4_assert([$# = 2])dnl +_$0(m4_quote(m4_default([$1], [[, ]])), +m4_ifval([$2], [[$2]], [m4_dquote(lt_decl_tag_varnames)]), +m4_split(m4_normalize(m4_quote(_LT_TAGS)), [ ]))]) +m4_define([_lt_decl_varnames_tagged], +[m4_ifval([$3], [lt_combine([$1], [$2], [_], $3)])]) # lt_decl_all_varnames([SEPARATOR], [VARNAME1...]) Index: libltdl/m4/ltsugar.m4 === RCS file: /sources/libtool/libtool/libltdl/m4/ltsugar.m4,v retrieving revision 1.7 diff -u -p -r1.7 ltsugar.m4 --- libltdl/m4/ltsugar.m4 16 Oct 2007 22:45:49 - 1.7 +++ libltdl/m4/ltsugar.m4 17 Oct 2007 17:29:37 - @@ -7,7 +7,7 @@ # unlimited permission to copy and/or distribute it, with or without # modifications, as long as this notice is preserved. -# serial 5 ltsugar.m4 +# serial 6 ltsugar.m4 # This is to help aclocal find these macros, as it can't see m4_define. AC_DEFUN([LTSUGAR_VERSION], [m4_if([0.1])]) @@ -63,14 +63,16 @@ m4_define([lt_append], # Produce a SEP delimited list of all paired combinations of elements of # PREFIX-LIST with SUFFIX1 through SUFFIXn. Each element of the list # has the form PREFIXmINFIXSUFFIXn. +# Needed until we can rely on m4_combine added in Autoconf 2.62. m4_define([lt_combine], -[m4_if([$2], [], [], - [m4_if([$4], [], [], -[lt_join(m4_quote(m4_default([$1], [[, ]])), - lt_unquote(m4_split(m4_normalize(m4_foreach(_Lt_prefix, [$2], - [m4_foreach(_Lt_suffix, lt_car([m4_shiftn(3, $@)]), - [_Lt_prefix[]$3[]_Lt_suffix ])])])])dnl -]) +[m4_if(m4_eval([$# 3]), [1], + [m4_pushdef([_Lt_sep], [m4_define([_Lt_sep], [[$1]])])]]dnl +[[m4_foreach([_Lt_prefix], [$2], +[m4_foreach([_Lt_suffix], +
