Re: [PATCH] Re: libtool-2.4.2 is fine but libtool-2.4.6 very slow.

2015-09-24 Thread Pavel Raiskup 
On Wednesday 23 of September 2015 16:09:49 Eric Blake wrote:
> On 09/23/2015 03:37 PM, Pavel Raiskup wrote:
> 
> > 
> >>From 5e8a4c5173f1aa0786e8eba15fb07bfe04b83862 Mon Sep 17 00:00:00 2001
> > From: Pavel Raiskup 
> > Date: Fri, 18 Sep 2015 23:17:07 +0200
> > Subject: [PATCH] libtoolize: fix infinite recursion in m4
> > 
> > Some projects use this construct in configure.ac:
> > 
> >   m4_define([version], m4_include([version])
> 
> Missing a )
> 
> The faulty package used:
> 
> m4_define([version], m4_include(version))

Thanks for catching the ')' typo!

> and the infinite recursion occurred _because_ the usage was underquoted.
> But as you wrote things here, you have sufficient quoting that you
> won't trigger infinite recursion.
> You need to drop the second [] for
> this to be an accurate representation of the failure.

I intentionally added the '[ ]' quotes there as that is something which
would look like correct usage, but it still goes to infinite recursion.

The problem is that while defining 'version':

  m4_define([version], m4_include([version]))

The expansion is like 'version => m4_include(version)'.  Then, usage of
'version' goes to infinite recursion.

> >   pkg_version=version
> > 
> > When the m4_include builtin is undefined (as was done in
> > libtoolize and extract-trace scripts), the call to this 'version'
> > macro gone to infinite recursion (until ENOMEM).  So rather
> 
> s/gone to infinite/enters an infinite/
> 
> > re-define all potentially dangerous macros by empty strings,
> > suggested by Eric Blake.
> > 
> > While we are on it, merge the macro-"blacklist" with similar list
> > implemented in gettext, except the 'm4_esyscmd'.  It's kept
> 
> s/except the/except for/
> 
> > defined because we already trace AC_INIT macro for package
> > version, while it is often specified by
> > m4_esyscmd(git-version-gen).  Similarly to m4_include, m4_esyscmd
> > might be opt-in-blacklisted in future.
> > 
> > References:
> > http://lists.gnu.org/archive/html/libtool/2015-09/msg0.html
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764580
> > 
> 
> With the commit message touched up, it looks okay to me.

I'll fix the typos, thanks!

Pavel

Re: [PATCH] Re: libtool-2.4.2 is fine but libtool-2.4.6 very slow.

2015-09-24 Thread Hiroyuki Sato 
Hello Pavel.

Thank you for your advice.
Workaround worked perfectly!!.
https://github.com/groonga/groonga/commit/7d7a62a2fba74334b23d0021a9a9e78da046b21c

Thanks.


2015年9月19日(土) 17:09 Pavel Raiskup :

> Hi Hiroyuki Sato,
>
> On Wednesday 02 of September 2015 16:00:34 Hiroyuki Sato wrote:
> > This configure.ac is extreme slow on libtool-2.4.6.
> > But It run smoothly on libtool-2.4.2.
> > https://github.com/groonga/groonga/blob/master/configure.ac
>
> thanks for reproducer!
>
> This _really_ looks like issue mentioned [1], though the thread is
> believed to be resolved (with existing small slowdown between
> libtool-2.4.5 and 2.4.2).  Let me CC Robert whether this patch does not
> actually fix the "1 sec" slowdown of libtoolize.
>
> > Could you tell me how to fix this problem?.
>
> You can workaround/fix this by:
> -m4_define([nginx_version], m4_include(nginx_version))
> -NGINX_VERSION=nginx_version
> +NGINX_VERSION=m4_include([nginx_version])
>
> See the "recursion" in 'nginx_version' definition.  Its not recursion
> usually (because nginx_version is not defined in the time of m4_define)
> -- but libtoolize script does 'm4_undefine(m4_include)' before
> evaluating configure.ac.
>
> > Is this libtool's Bug?
>
> I would say so, see the attached patch I'll push very soon (if there are
> no objections).
>
> > * OS OSX 10.10.5. and CentOS7
>
> Should be IMO reproducible everywhere.
>
> [1] http://lists.gnu.org/archive/html/libtool/2015-02/msg0.html
>
> Thanks, Pavel
> --
>
> > Thanks.
> >
> > libtool-2.4.6
> >
> >   time /tmp/libtool246/bin/libtoolize --verbose --copy --force
> >   libtoolize: found 'configure.ac'
> >   libtoolize: export M4='/usr/bin/gm4'
> >
> >   It takes around 10 minutes.
> >
> > libtool-2.4.2
> >
> >   time /tmp/libtool242/bin/libtoolize --verbose --copy --force
> >   libtoolize: rm -f './ltmain.sh'
> >   libtoolize: putting auxiliary files in `.'.
> >   libtoolize: copying file from
> `/tmp/libtool242/share/libtool/config/ltmain.sh'
> >   libtoolize: rm -f './libtool.m4'
> >   libtoolize: rm -f './ltoptions.m4'
> >   libtoolize: rm -f './ltsugar.m4'
> >   libtoolize: rm -f './ltversion.m4'
> >   libtoolize: rm -f './lt~obsolete.m4'
> >   libtoolize: Not copying `./argz.m4', libltdl not used.
> >   libtoolize: putting macros in `.'.
> >   libtoolize: copying file from
> `/tmp/libtool242/share/aclocal/libtool.m4'
> >   libtoolize: Not copying `./ltdl.m4', libltdl not used.
> >   libtoolize: copying file from
> `/tmp/libtool242/share/aclocal/ltoptions.m4'
> >   libtoolize: copying file from
> `/tmp/libtool242/share/aclocal/ltsugar.m4'
> >   libtoolize: copying file from
> `/tmp/libtool242/share/aclocal/ltversion.m4'
> >   libtoolize: copying file from
> `/tmp/libtool242/share/aclocal/lt~obsolete.m4'
> >   libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to
> configure.ac and
> >   libtoolize: rerunning libtoolize, to keep the correct libtool macros
> in-tree.
> >
> >   real 0m1.003s
> >   user 0m0.364s
> >   sys 0m0.540s
> >
> >
> >
>

Re: [PATCH] Re: libtool-2.4.2 is fine but libtool-2.4.6 very slow.

2015-09-23 Thread Pavel Raiskup 
Hi Eric, thanks for your review!

On Saturday 19 of September 2015 15:05:07 Eric Blake wrote:
> On 09/19/2015 02:09 AM, Pavel Raiskup wrote:
> > Hi Hiroyuki Sato,
> > 
> > On Wednesday 02 of September 2015 16:00:34 Hiroyuki Sato wrote:
> >> This configure.ac is extreme slow on libtool-2.4.6.
> >> But It run smoothly on libtool-2.4.2.
> >> https://github.com/groonga/groonga/blob/master/configure.ac
> > 
> > thanks for reproducer!
> > 
> > This _really_ looks like issue mentioned [1], though the thread is
> > believed to be resolved (with existing small slowdown between
> > libtool-2.4.5 and 2.4.2).  Let me CC Robert whether this patch does not
> > actually fix the "1 sec" slowdown of libtoolize.
> 
> This looks very much like the same bug that gettext had:
> http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=d75090f2
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764580

Indeed, I missed that report.

> Your proposed solution doesn't seem quite right:
> 
> >  # Disable these macros.
> >  m4_undefine([m4_dnl])
> > -m4_undefine([m4_include])
> >  m4_undefine([m4_m4exit])
> >  m4_undefine([m4_m4wrap])
> >  m4_undefine([m4_maketemp])
> > +# Rather do not use undefine here because people tend to define
> > +# macros of the same name as file included in their bodies - which
> > +# results in infinite recursion.
> > +m4_define([m4_include], [])
> 
> I'd recommend that you use the same fix as gettext, and define ALL of
> these macros to an empty string, rather than special-casing m4_include
> as the only one that does not get undefined.

Agreed, m4_pushdef([trace_breaking_macro]) should not cause any harm.
I fixed the patch (attached).

Pavel
>From 5e8a4c5173f1aa0786e8eba15fb07bfe04b83862 Mon Sep 17 00:00:00 2001
From: Pavel Raiskup 
Date: Fri, 18 Sep 2015 23:17:07 +0200
Subject: [PATCH] libtoolize: fix infinite recursion in m4

Some projects use this construct in configure.ac:

  m4_define([version], m4_include([version])
  pkg_version=version

When the m4_include builtin is undefined (as was done in
libtoolize and extract-trace scripts), the call to this 'version'
macro gone to infinite recursion (until ENOMEM).  So rather
re-define all potentially dangerous macros by empty strings,
suggested by Eric Blake.

While we are on it, merge the macro-"blacklist" with similar list
implemented in gettext, except the 'm4_esyscmd'.  It's kept
defined because we already trace AC_INIT macro for package
version, while it is often specified by
m4_esyscmd(git-version-gen).  Similarly to m4_include, m4_esyscmd
might be opt-in-blacklisted in future.

References:
http://lists.gnu.org/archive/html/libtool/2015-09/msg0.html
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764580

* gl/build-aux/extract-trace (_G_mini): Redefine trace-breaking
macros to empty strings rather than undefining those.  Use 'dnl'
for comments.
* bootstrap: Likewise, sync with extract-trace.
* NEWS: Document.
* NO-THANKS: Mention Hiroyuki Sato.
---
 NEWS   |  4 
 NO-THANKS  |  1 +
 bootstrap  | 44 
 gl/build-aux/extract-trace | 44 
 4 files changed, 61 insertions(+), 32 deletions(-)

diff --git a/NEWS b/NEWS
index c5c9023..82dd037 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,10 @@ NEWS - list of user-visible changes between releases of GNU Libtool
 
 * Noteworthy changes in release ?.? (-??-??) [?]
 
+** Bug fixes:
+
+  - Fix significant slowdown of libtoolize for certain projects (regression
+introduced in 2.4.3 release) caused by infinite m4 macro recursion.
 
 * Noteworthy changes in release 2.4.6 (2015-02-15) [stable]
 
diff --git a/NO-THANKS b/NO-THANKS
index 10de16b..51ec8eb 100644
--- a/NO-THANKS
+++ b/NO-THANKS
@@ -82,6 +82,7 @@ Donn Washburn			n5...@comcast.net
 Erik van Pienbroek		erik-...@vanpienbroek.nl
 Ethan Mallove			ethan.mall...@sun.com
 Fred Cox			sailorf...@yahoo.com
+Hiroyuki Sato			hiroys...@gmail.com
 Jakub Bogusz			qbo...@pld-linux.org
 James Su			james...@gmail.com
 Jay Krell			jay.kr...@cornell.edu
diff --git a/bootstrap b/bootstrap
index 4596413..17fb169 100755
--- a/bootstrap
+++ b/bootstrap
@@ -2472,29 +2472,41 @@ func_extract_trace ()
 # arguments to Autocof functions, but without following
 # 'm4_s?include' files.
 _G_mini='
-# Initialisation.
+dnl  Initialisation.
 m4_changequote([,])
 m4_define([m4_copy],   [m4_define([$2], m4_defn([$1]))])
 m4_define([m4_rename], [m4_copy([$1], [$2])m4_undefine([$1])])
 
-# Disable these macros.
-m4_undefine([m4_dnl])
-m4_undefine([m4_include])
-m4_undefine([m4_m4exit])
-m4_undefine([m4_m4wrap])
-m4_undefine([m4_maketemp])
-
-# Copy and rename macros not handled by "m4 --prefix".
+dnl  Replace macros which may abort m4

Re: [PATCH] Re: libtool-2.4.2 is fine but libtool-2.4.6 very slow.

2015-09-23 Thread Eric Blake 
On 09/23/2015 03:37 PM, Pavel Raiskup wrote:

> 
>>From 5e8a4c5173f1aa0786e8eba15fb07bfe04b83862 Mon Sep 17 00:00:00 2001
> From: Pavel Raiskup 
> Date: Fri, 18 Sep 2015 23:17:07 +0200
> Subject: [PATCH] libtoolize: fix infinite recursion in m4
> 
> Some projects use this construct in configure.ac:
> 
>   m4_define([version], m4_include([version])

Missing a )

The faulty package used:

m4_define([version], m4_include(version))

and the infinite recursion occurred _because_ the usage was underquoted.
 But as you wrote things here, you have sufficient quoting that you
won't trigger infinite recursion.  You need to drop the second [] for
this to be an accurate representation of the failure.

>   pkg_version=version
> 
> When the m4_include builtin is undefined (as was done in
> libtoolize and extract-trace scripts), the call to this 'version'
> macro gone to infinite recursion (until ENOMEM).  So rather

s/gone to infinite/enters an infinite/

> re-define all potentially dangerous macros by empty strings,
> suggested by Eric Blake.
> 
> While we are on it, merge the macro-"blacklist" with similar list
> implemented in gettext, except the 'm4_esyscmd'.  It's kept

s/except the/except for/

> defined because we already trace AC_INIT macro for package
> version, while it is often specified by
> m4_esyscmd(git-version-gen).  Similarly to m4_include, m4_esyscmd
> might be opt-in-blacklisted in future.
> 
> References:
> http://lists.gnu.org/archive/html/libtool/2015-09/msg0.html
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764580
> 

With the commit message touched up, it looks okay to me.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [PATCH] Re: libtool-2.4.2 is fine but libtool-2.4.6 very slow.

2015-09-19 Thread Eric Blake 
On 09/19/2015 02:09 AM, Pavel Raiskup wrote:
> Hi Hiroyuki Sato,
> 
> On Wednesday 02 of September 2015 16:00:34 Hiroyuki Sato wrote:
>> This configure.ac is extreme slow on libtool-2.4.6.
>> But It run smoothly on libtool-2.4.2.
>> https://github.com/groonga/groonga/blob/master/configure.ac
> 
> thanks for reproducer!
> 
> This _really_ looks like issue mentioned [1], though the thread is
> believed to be resolved (with existing small slowdown between
> libtool-2.4.5 and 2.4.2).  Let me CC Robert whether this patch does not
> actually fix the "1 sec" slowdown of libtoolize.

This looks very much like the same bug that gettext had:
http://git.savannah.gnu.org/cgit/gettext.git/commit/?id=d75090f2
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764580

Your proposed solution doesn't seem quite right:

>  # Disable these macros.
>  m4_undefine([m4_dnl])
> -m4_undefine([m4_include])
>  m4_undefine([m4_m4exit])
>  m4_undefine([m4_m4wrap])
>  m4_undefine([m4_maketemp])
> +# Rather do not use undefine here because people tend to define
> +# macros of the same name as file included in their bodies - which
> +# results in infinite recursion.
> +m4_define([m4_include], [])

I'd recommend that you use the same fix as gettext, and define ALL of
these macros to an empty string, rather than special-casing m4_include
as the only one that does not get undefined.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[PATCH] Re: libtool-2.4.2 is fine but libtool-2.4.6 very slow.

2015-09-19 Thread Pavel Raiskup 
Hi Hiroyuki Sato,

On Wednesday 02 of September 2015 16:00:34 Hiroyuki Sato wrote:
> This configure.ac is extreme slow on libtool-2.4.6.
> But It run smoothly on libtool-2.4.2.
> https://github.com/groonga/groonga/blob/master/configure.ac

thanks for reproducer!

This _really_ looks like issue mentioned [1], though the thread is
believed to be resolved (with existing small slowdown between
libtool-2.4.5 and 2.4.2).  Let me CC Robert whether this patch does not
actually fix the "1 sec" slowdown of libtoolize.

> Could you tell me how to fix this problem?.

You can workaround/fix this by:
-m4_define([nginx_version], m4_include(nginx_version))
-NGINX_VERSION=nginx_version
+NGINX_VERSION=m4_include([nginx_version])

See the "recursion" in 'nginx_version' definition.  Its not recursion
usually (because nginx_version is not defined in the time of m4_define)
-- but libtoolize script does 'm4_undefine(m4_include)' before
evaluating configure.ac.

> Is this libtool's Bug?

I would say so, see the attached patch I'll push very soon (if there are
no objections).

> * OS OSX 10.10.5. and CentOS7

Should be IMO reproducible everywhere.

[1] http://lists.gnu.org/archive/html/libtool/2015-02/msg0.html

Thanks, Pavel
--

> Thanks.
> 
> libtool-2.4.6
> 
>   time /tmp/libtool246/bin/libtoolize --verbose --copy --force
>   libtoolize: found 'configure.ac'
>   libtoolize: export M4='/usr/bin/gm4'
> 
>   It takes around 10 minutes.
> 
> libtool-2.4.2
> 
>   time /tmp/libtool242/bin/libtoolize --verbose --copy --force
>   libtoolize: rm -f './ltmain.sh'
>   libtoolize: putting auxiliary files in `.'.
>   libtoolize: copying file from 
> `/tmp/libtool242/share/libtool/config/ltmain.sh'
>   libtoolize: rm -f './libtool.m4'
>   libtoolize: rm -f './ltoptions.m4'
>   libtoolize: rm -f './ltsugar.m4'
>   libtoolize: rm -f './ltversion.m4'
>   libtoolize: rm -f './lt~obsolete.m4'
>   libtoolize: Not copying `./argz.m4', libltdl not used.
>   libtoolize: putting macros in `.'.
>   libtoolize: copying file from `/tmp/libtool242/share/aclocal/libtool.m4'
>   libtoolize: Not copying `./ltdl.m4', libltdl not used.
>   libtoolize: copying file from `/tmp/libtool242/share/aclocal/ltoptions.m4'
>   libtoolize: copying file from `/tmp/libtool242/share/aclocal/ltsugar.m4'
>   libtoolize: copying file from `/tmp/libtool242/share/aclocal/ltversion.m4'
>   libtoolize: copying file from `/tmp/libtool242/share/aclocal/lt~obsolete.m4'
>   libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and
>   libtoolize: rerunning libtoolize, to keep the correct libtool macros 
> in-tree.
> 
>   real 0m1.003s
>   user 0m0.364s
>   sys 0m0.540s
> 
> 
> 
>From 40d297de7502f93a546126ff69d3f687f3d3ca1a Mon Sep 17 00:00:00 2001
From: Pavel Raiskup 
Date: Fri, 18 Sep 2015 23:17:07 +0200
Subject: [PATCH] libtoolize: fix infinite recursion in m4

Some projects use this construct in configure.ac:

  m4_define([version], m4_include([version])
  pkg_version=version

When the m4_include builtin is undefined (as is done in libtoolize
and extract-trace scripts), the call to 'version' macro goes to
infinite recursion (until ENOMEM).  Due to theirs usual
semantics/evaluation, other undefined macros/builtins should not
cause troubles here.

References:
http://lists.gnu.org/archive/html/libtool/2015-09/msg0.html

* gl/build-aux/extract-trace (_G_mini): Redefine m4_include to
empty string rather than undefine it completely.
* NEWS: Document.
* NO-THANKS: Mention Hiroyuki Sato.
---
 NEWS   | 4 
 NO-THANKS  | 1 +
 gl/build-aux/extract-trace | 5 -
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index c5c9023..e68c38c 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,10 @@ NEWS - list of user-visible changes between releases of GNU Libtool
 
 * Noteworthy changes in release ?.? (-??-??) [?]
 
+** Bug fixes:
+
+  - Fix significant slowdown of libtoolize for certain projects (regression
+introduced in 2.4.3 release).
 
 * Noteworthy changes in release 2.4.6 (2015-02-15) [stable]
 
diff --git a/NO-THANKS b/NO-THANKS
index 10de16b..51ec8eb 100644
--- a/NO-THANKS
+++ b/NO-THANKS
@@ -82,6 +82,7 @@ Donn Washburn			n5...@comcast.net
 Erik van Pienbroek		erik-...@vanpienbroek.nl
 Ethan Mallove			ethan.mall...@sun.com
 Fred Cox			sailorf...@yahoo.com
+Hiroyuki Sato			hiroys...@gmail.com
 Jakub Bogusz			qbo...@pld-linux.org
 James Su			james...@gmail.com
 Jay Krell			jay.kr...@cornell.edu
diff --git a/gl/build-aux/extract-trace b/gl/build-aux/extract-trace
index 315a32a..ebbf42a 100755
--- a/gl/build-aux/extract-trace
+++ b/gl/build-aux/extract-trace
@@ -336,10 +336,13 @@ func_extract_trace ()
 
 # Disable these macros.
 m4_undefine([m4_dnl])
-m4_undefine([m4_include])
 m4_undefine([m4_m4exit])
 m4_undefine([m4_m4wrap])
 m4_undefine([m4_maketemp])
+# Rather do not use undefine here because people tend to define
+