Re: libtool-2.5.1 released [beta]

2024-08-09 Thread Richard Purdie
On Fri, 2024-07-26 at 19:13 +0300, Ileana Dumitrescu wrote:
> On 26/07/2024 03:26, Bruno Haible wrote:
> > Hi Ileana,
> > 
> > I tested a GNU gettext tarball, built with libtool-2.5.1, on
> > several platforms,
> > including on Solaris 11.3 (where libtool-2.4.7 had a problem). No
> > issues found;
> > everything is fine.
> > 
> > Bruno
> > 
> 
> I am happy to hear that, and thank you for testing the beta release!
> 
> I hope to release a stable version soon after fixing a few more bugs
> that have already been reported.

I can report that Yocto Project/OpenEmbedded were able to update to
2.5.1 and haven't seen any regressions so far.

https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/libtool

Our patchset (in the libtool dir) is looking much reduced too so thanks
for the help in sorting some of the issues out!

Cheers,

Richard





Re: rpath stripping

2022-04-18 Thread Richard Purdie
On Mon, 2022-04-18 at 07:39 -0400, Carlos O'Donell wrote:
> On 4/17/22 10:06, Bob Friesenhahn wrote:
> > The libtool I was using (originating from Ubuntu Linux) stripped the
> > rpath (which was provided like '-Wl,rpath=/usr/lib') so I was unable
> > to embed an rpath in the libcurl I built so that applications linked
> > with that libcurl would find it.
> 
> I agree with our position.
> 
> The behaviour of stripping '-Wl,-rpath' is incorrect.
> 
> With new DT_RUNPATH semantics (DT_RPATH being deprecated and binutils having
> switched defaults), each shared object, including the binary, must correctly
> specify the search path for the immediate needed objects. Stripping this off
> will result in incorrectly built shared objects and binaries that don't
> operate correctly.
> 
> I'm curious what justification is given for this behaviour.

As I understand it the dynamic loader has a set of search paths it falls back to
(sys_lib_dlsearch_path in libtool). An RPATH or RUNPATH entry matching a system
loader path isn't usually of much use, it just takes up space.

I can imagine some cases where that might not be true such as linking with "-z
nodeflib" or some fairly convoluted setups but I suspect those would have other
issues too.

Cheers,

Richard





Submission process for libtool?

2022-03-31 Thread Richard Purdie
Hi,

It was great to see a libtool release, thanks for that!

I upgraded Yocto Project to it in time for our LTS release:

https://git.yoctoproject.org/poky/commit/?id=ff7b41573842a403c81f58bee41fc8163a9d7754

so far things seem reasonable, we've had a few minor issues but they're not
really libtool's fault or concern. One interesting quirk was that the shell
script optimisation changes made between 2.4.6 and 2.4.7 resulted in very long
(6,000+ character) pathnames being passed to the C library functions. This upset
our fakeroot emulation but we've fixed that to workaround the issue.

Yocto Project is carrying a few patches. I did clean them up and shared many of
them in October:

https://lists.gnu.org/archive/html/libtool-patches/2021-10/msg00012.html

Some are more important than others and there what I believe are good bug fixes
in there. My questions:

a) Is there a possibility these could be considered for merging?

b) Should I rebase and repost them? I'm happy to do it if it would help.

c) Am I using the right process for patch submission? I never did get a reply
despite asking publicly and privately. If I'm doing something wrong process wise
to submit them, I'd happily correct it.

Yocto Project is interesting as we can quickly test changes against the software
of a whole Linux stack across many architectures "quite quickly" (complete build
and tests in a few hours). I'm going to try and more closely track libtool
development so that we hopefully identify regressions from our perpective
quickly.

Thanks,

Richard








Re: new release?

2022-02-06 Thread Richard Purdie
On Sat, 2022-02-05 at 21:06 -0500, Mike Frysinger wrote:
> On 05 Feb 2022 15:15, Alex Ameen wrote:
> > This is a good question. I plan on making a new release this month.
> > 
> > When I first adopted the project I ambitiously thought I'd manage to 
> > create a new release after about a month; but the truth is when I 
> > started doing a deep dive into the internals there was a lot of history 
> > and complexity for me to unpack. Things that are easy to overlook like 
> > how change-logs get generated, quirks in the testing framework, and 
> > tracing down disparate areas to align documentation took quite a while 
> > to navigate.
> > 
> > The good news is that I think I've got the confidence to push a release 
> > soon. One area that I was reading up on this weekend was whether the 
> > "alpha"/private releases of `libtool' might be appropriate, or whether I 
> > should just push a release immediately. I'll admit I am leaning towards 
> > just making a release to avoid the entire alpha process for the time being.
> 
> i wouldn't sweat it too much.  the next release of libtool will be 2.6, and
> you can note its state in the announcement/NEWS.  distros will give it a run
> to find regressions, and as fixes are merged, just do 2.6.1, 2.6.2, etc...
> 

I'd like to second that. Getting a release out would be great even if it isn't
perfect, then go from there.

I know there are some Yocto Project patches for issues we've collected from
across the embedded ecosystem over the last few years that I rebased and posted
in the hope they could be merged. I'd rather we got to those in due course and
had a release though! :)

Cheers,

Richard






Re: New libtool maintainer

2021-11-21 Thread Richard Purdie
On Sun, 2021-11-21 at 13:14 -0600, Alex Ameen wrote:
> I just took a look at those. Good catches on the typos, I probably would 
> not have noticed them just reading the script myself. Same thing with 
> the M4 `[]' quoting issue ( classic pitfall ). I'll get these merged ASAP.
> 
> For the non-Linux patches I can merge them, but I don't have personally 
> have OSX, powerpc, or Solaris boxes, and while I do have a Windows 
> partition I don't currently have it set up to run these kinds of tests. 
> Nonetheless I can merge these - if you have access to any of those 
> platforms let me know if you would be open to running `make check' and 
> posting the logs so I can sanity check the new behavior.
> 
> Thank you so much for bringing these to my attention. There's a long 
> list of old patches and mailing list archives and as a practical matter 
> it's hard to know which of them are still relevant - so I appreciate 
> your help.

I did recently post the better bits of the OpenEmbedded/Yocto Project patchset
for libtool:

https://lists.gnu.org/archive/html/libtool-patches/2021-10/msg00012.html

Those are at up to date and in regular use in OE/YP which is widely used for
cross compiling for Linux/mingw and others. We tend to find sysroot and cross
compile issues in particular and we also find reproducibility and parallel make
race issues.

If you have any questions or concerns on any of those I'm happy to try and help.

Cheers,

Richard






Re: transitive shared library dependencies and installation

2020-01-03 Thread Richard Purdie
On Thu, 2020-01-02 at 16:24 -0600, Bob Friesenhahn wrote:
> On Thu, 2 Jan 2020, wf...@niif.hu wrote:
> > > If Libtool were to depend on non-portable features, [...] then it
> > > could not longer be described as a portability tool.
> > 
> > In my understanding, Libtool is a portability shim, providing a
> > regular
> > interface for developers across systems with wildly varying
> > features.
> > It already uses non-portable features, just different ones on
> > different
> > architectures.  I don't say it should use -rpath-link generally, I
> > only
> > suggested that it might be an efficient solution on systems
> > supporting
> > it.  But I expect all systems supporting shared objects to allow
> > using
> > and installing them some way, irrespective of their
> > interdependencies.
> > Am I overly naive?
> 
> Certainly, libtool could use -rpath-link where it is supported but 
> libtool provides a common feature set and if it is only possible to 
> support a feature where -rpath-link is available, then offering the 
> feature would defeat the purpose of being a portability tool.
> 
> Sometimes it is better to force the using software to conform to the 
> limitations.
> 
> Libtool must also work for static linking.  It seems to me that your 
> issue also impacts static linking.

I think the challenge that libtool has here is that many of these older
systems that libtool supports aren't so prevalent anymore.

I work on the Yocto Project where we do a ton of cross compiling and
have to work with libtool but its mostly Linux with a small amount of
mingw/baremetal/darwin bits.

We have a number of libtool patches to sort cross compile issues which
really need discussion with upstream libtool. With the lack of
releases, our incentive to do that diminishes :(.

By reducing everything to the lowest common denominator, it means
libtool struggles to take advantage of any new linker technology too.

Finding new maintainers with the amount of knowledge of weird older
systems needed is going to be a struggle which only gets worse over
time :(

I do worry about the future here as libtool is a key part of the
autotools fabric but its most likely to be wholesale replaced given how
things are trending.

Cheers,

Richard






Re: Trailing slash in directory spec confuses libtool

2016-08-12 Thread Richard Purdie
On Fri, 2016-08-12 at 22:06 +0200, Jan Engelhardt wrote:
> Given certain circumstances, libtool 2.4.2 fails to install a
> library. 
> (a) The target directory spec contains a trailing slash
> (b) The library to install is linking to another just-built one in a 
> different path.

FWIW we have this patch:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/recipes-devtools/libtool/libtool/trailingslash.patch

which seems to help. We've never been able to clean it up enough to
submit upstream or have a reliable test case for it though :(

Cheers,

Richard

[patch follows below]

A command like /bin/sh ../../i586-poky-linux-libtool   --mode=install 
/usr/bin/install -c   gck-roots-store-standalone.la 
'/media/data1/builds/poky1/tmp/work/core2-poky-linux/gnome-keyring-2.26.1-r1/image/usr/lib/gnome-keyring/standalone/'
 fails (e.g. gnome-keyring or pulseaudio)

This is because libdir has a trailing slash which breaks the comparison.

RP 2/1/10

Merged a patch received from Gary Thomas 

Date: 2010/07/12
Nitin A Kamble 

Updated by: Robert Yang 

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -2356,8 +2356,15 @@ func_mode_install ()
func_append dir "$objdir"
 
if test -n "$relink_command"; then
+  # Strip any trailing slash from the destination.
+  func_stripname '' '/' "$libdir"
+  destlibdir=$func_stripname_result
+
+  func_stripname '' '/' "$destdir"
+  s_destdir=$func_stripname_result
+
  # Determine the prefix the user has applied to our future dir.
- inst_prefix_dir=`$ECHO "$destdir" | $SED -e "s%$libdir\$%%"`
+ inst_prefix_dir=`$ECHO "X$s_destdir" | $Xsed -e "s%$destlibdir\$%%"`
 
  # Don't allow the user to place us outside of our expected
  # location b/c this prevents finding dependent libraries that

___
https://lists.gnu.org/mailman/listinfo/libtool


Re: Bash-specific performance by avoiding sed

2015-03-09 Thread Richard Purdie
On Mon, 2015-03-09 at 18:28 +, Mike Gran wrote:
> I don't know if y'all saw this blogpost where a guy pushed
> the sed regular expression handling into bash-specific
> regular expressions when bash was available.  He claims
> there's a significant performance improvement because of
> reduced forking.
> 
> http://harald.hoyer.xyz/2015/03/05/libtool-getting-rid-of-18-sed-forks/

Please see my reply about this. The issue is the amount of looping
through the options parsing code which I did already report here as an
issue.

I do have a "fix" for it in the build systems I look after:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cc5f804483a9a1a60be24475baee0eed5d44bef5

which basically unrolls some of the internal code looping and I believe
that with that patch, the speedup to the specific sed invocation you're
looking at will be much less relevant.

The issue is that in its current form, it can't really be merged and my
shell knowledge and lack of contribution agreement to GNU are likely
blocking issues along with a lack of time :(

Cheers,

Richard




___
https://lists.gnu.org/mailman/listinfo/libtool


Re: Performance issue of libtool-2.4.4

2015-02-10 Thread Richard Purdie
On Tue, 2015-02-10 at 10:53 +, Gary V. Vaughan wrote:
> On Feb 10, 2015, at 10:35 AM, Richard Purdie 
>  wrote:
> > On Mon, 2015-02-09 at 23:36 +0000, Richard Purdie wrote:
> >> On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
> >>> In an effort to get to the bottom of this I made a git bisection, timing
> >>> the performance of building xz with make -j1 using each different
> >>> libtool.
> >>> 
> >>> The issues come down to this commit:
> >>> 
> >>> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> >>> 
> >>> libtool: rewritten over funclib.sh instead of general.m4sh.
> >>> 
> >>> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> >>> in the fix in master mentioned above, I get 27s.
> >>> 
> >>> So whilst things are better (thanks!), the above change is still causing
> >>> a regression in the performance somewhere else. Any ideas what else in
> >>> that rather large change may be causing this?
> >> 
> >> To further narrow this down, of the changes in the above commit, the
> >> problem appears to be in the changes to the option parsing code. I've
> >> included the diff below which if I apply on top of the above, I get the
> >> speed back. I've left the func_split_short_opt/func_split_long_opt code
> >> in there but that is worth a tiny part of the speed, the issues are
> >> around the addition of the func_options call.
> >> 
> >> As yet I don't know enough about the code in question to know why this
> >> is an issue but traces of libtool show a lot more looping in code to do
> >> with argument parsing and quoting.
> > 
> > To be more specific, if I take my "good" libtool and add:
> > 
> > func_options_prep ${1+"$@"}
> > 
> > it slows the build down by 0.5s on a 21s build. If I look at
> > func_options_prep and comment out the line:
> > 
> >func_run_hooks func_options_prep ${1+"$@"}
> > 
> > I get the 0.5s back.
> > 
> > In func_run_hooks, if I comment:
> > 
> >func_quote_for_eval ${1+"$@"}
> >func_run_hooks_result=$func_quote_for_eval_result
> > 
> > I get the 0.5s back. The issue is all the quoting of the various return
> > values through all this looping. It doesn't appear to be hitting the
> > printf/sed in func_quote_for_eval which would be an obvious slow path,
> > its just the shear number of loops run through with the commandline
> > arguments. The change adds a number of calls to func_run_hooks, not just
> > the single test case I have above and all combined, it slows things down
> > significantly.
> > 
> > So is there a way we can change things so its not calling
> > func_quote_for_eval all the time with all the looping that entails?
> 
> One possibility would be to add a post-processing script that rewrites
> the hook functions used by func_options into a a single top-level blob of
> sequential shell code.  I imagine that adding some carefully chosen comment
> strings would make extracting the right parts in the right order relatively
> straight forward... it might even be that inlining func_options_prep and
> any hook functions it calls would be enough?

Thanks for the background info on this. I understand the need to change
and improve the software so I'm not proposing reverting, I do think
there has to be some way to get some of the speed here back though.

I have a bit of a pressing need to have a things performing as they were
and I'd prefer to stay on libtool 2.4.5 than revert back to 2.4.2 so I
cooked up the patch I've included below. This basically manually unrolls
the key problematic parts, cut and paste from options-parser. With this
applied to master, I have a build time of 22s compared to 20s or 21s
(need to go and retest as I'm forgetting numbers) with 2.4.2.

For now I'll probably merge something like this into the Yocto
Project/OpenEmbedded, the exact way to solve this longer term is TBD.

Cheers,

Richard




diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index d5cf07a..0f54303 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -342,11 +342,15 @@ _LT_EOF
 # libtool_options_prep [ARG]...
 # -
 # Preparation for options parsed by libtool.
-libtool_options_prep ()
-{
+#libtool_options_prep ()
+#{
 $debug_mode
 
 # Option defaults:
+opt_verbose=false
+opt_warning_types=
+
+# Option defaults:
 opt_config=false
 opt_dlopen=
 opt_dry_run=false
@

Re: Performance issue of libtool-2.4.4

2015-02-10 Thread Richard Purdie
On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:
> On Mon, 2015-02-09 at 13:05 +0000, Richard Purdie wrote:
> > In an effort to get to the bottom of this I made a git bisection, timing
> > the performance of building xz with make -j1 using each different
> > libtool.
> > 
> > The issues come down to this commit:
> > 
> > http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> > 
> > libtool: rewritten over funclib.sh instead of general.m4sh.
> > 
> > Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> > in the fix in master mentioned above, I get 27s.
> > 
> > So whilst things are better (thanks!), the above change is still causing
> > a regression in the performance somewhere else. Any ideas what else in
> > that rather large change may be causing this?
> 
> To further narrow this down, of the changes in the above commit, the
> problem appears to be in the changes to the option parsing code. I've
> included the diff below which if I apply on top of the above, I get the
> speed back. I've left the func_split_short_opt/func_split_long_opt code
> in there but that is worth a tiny part of the speed, the issues are
> around the addition of the func_options call.
> 
> As yet I don't know enough about the code in question to know why this
> is an issue but traces of libtool show a lot more looping in code to do
> with argument parsing and quoting.

To be more specific, if I take my "good" libtool and add:

func_options_prep ${1+"$@"}

it slows the build down by 0.5s on a 21s build. If I look at
func_options_prep and comment out the line:

func_run_hooks func_options_prep ${1+"$@"}

I get the 0.5s back.

In func_run_hooks, if I comment:

func_quote_for_eval ${1+"$@"}
func_run_hooks_result=$func_quote_for_eval_result

I get the 0.5s back. The issue is all the quoting of the various return
values through all this looping. It doesn't appear to be hitting the
printf/sed in func_quote_for_eval which would be an obvious slow path,
its just the shear number of loops run through with the commandline
arguments. The change adds a number of calls to func_run_hooks, not just
the single test case I have above and all combined, it slows things down
significantly.

So is there a way we can change things so its not calling
func_quote_for_eval all the time with all the looping that entails?

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool


Re: Performance issue of libtool-2.4.4

2015-02-09 Thread Richard Purdie
On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
> In an effort to get to the bottom of this I made a git bisection, timing
> the performance of building xz with make -j1 using each different
> libtool.
> 
> The issues come down to this commit:
> 
> http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
> 
> libtool: rewritten over funclib.sh instead of general.m4sh.
> 
> Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
> in the fix in master mentioned above, I get 27s.
> 
> So whilst things are better (thanks!), the above change is still causing
> a regression in the performance somewhere else. Any ideas what else in
> that rather large change may be causing this?

To further narrow this down, of the changes in the above commit, the
problem appears to be in the changes to the option parsing code. I've
included the diff below which if I apply on top of the above, I get the
speed back. I've left the func_split_short_opt/func_split_long_opt code
in there but that is worth a tiny part of the speed, the issues are
around the addition of the func_options call.

As yet I don't know enough about the code in question to know why this
is an issue but traces of libtool show a lot more looping in code to do
with argument parsing and quoting.

Cheers,

Richard



--- libtool-bad 2015-02-09 16:34:30.702068736 +
+++ libtool 2015-02-09 23:31:37.238750764 +
@@ -791,13 +791,77 @@
 fi
 }
 
+exit_status=$EXIT_SUCCESS
 
-# libtool_options_prep [ARG]...
-# -
-# Preparation for options parsed by libtool.
-libtool_options_prep ()
-{
-$debug_mode
+# We should try to minimise forks, especially on Windows where they are
+# unreasonably slow, so skip the feature probes when bash or zsh are
+# being used:
+if test set = "${BASH_VERSION+set}$ZSH_VERSION"; then
+: ${lt_HAVE_ARITH_OP="yes"}
+: ${lt_HAVE_XSI_OPS="yes"}
+fi
+
+
+
+# lt_HAVE_XSI_OPS
+# Can be empty, in which case the shell is probed, "yes" if XSI length
+# and matching operators are useable or anything else if they do not work.
+test -z "$lt_HAVE_XSI_OPS" \
+&& (eval 'x=a/b/c;
+  test 5aa/bb/cc = "${#x}${x%%/*}${x%/*}${x#*/}${x##*/}"') 2>/dev/null \
+&& lt_HAVE_XSI_OPS=yes
+
+
+
+# If this shell supports prefix and suffix pattern removal, then
+# use them to avoid forking. Hide the definition in an eval in case
+# the shell chokes on unsupported syntax...
+if test yes = "$lt_HAVE_XSI_OPS"; then
+  # func_split_short_opt shortopt
+  # Set func_split_short_opt_name and func_split_short_opt_arg shell
+  # variables after splitting SHORTOPT after the 2nd character.
+  eval 'func_split_short_opt ()
+  {
+$debug_cmd
+
+func_split_short_opt_arg=${1#??}
+func_split_short_opt_name=${1%"$func_split_short_opt_arg"}
+  }'
+
+  # func_split_long_opt longopt
+  # Set func_split_long_opt_name and func_split_long_opt_arg shell
+  # variables after splitting LONGOPT at the `=' sign.
+  eval 'func_split_long_opt ()
+  {
+func_split_long_opt_name=${1%%=*}
+func_split_long_opt_arg=${1#*=}
+  }'
+else
+  # ...otherwise fall back to using sed.
+  func_split_short_opt ()
+  {
+my_sed_short_opt='1s/^\(..\).*$/\1/;q'
+my_sed_short_rest='1s/^..\(.*\)$/\1/;q'
+
+func_split_short_opt_name=`$ECHO "$1" | $SED "$my_sed_short_opt"`
+func_split_short_opt_arg=`$ECHO "$1" | $SED "$my_sed_short_rest"`
+  }
+
+  func_split_long_opt ()
+  {
+my_sed_long_opt='1s/^\(--[^=]*\)=.*/\1/;q'
+my_sed_long_arg='1s/^--[^=]*=//'
+
+func_split_long_opt_name=`$ECHO "$1" | $SED "$my_sed_long_opt"`
+func_split_long_opt_arg=`$ECHO "$1" | $SED "$my_sed_long_arg"`
+  }
+fi
+
+debug_cmd=${debug_cmd-':'}
+opt_warning=:
+opt_verbose=:
+opt_verbose=false
+exit_cmd=:
 
 # Option defaults:
 opt_config=false
@@ -836,26 +900,23 @@
   ;;
 esac
 
-# Pass back the list of options.
-func_quote_for_eval ${1+"$@"}
-libtool_options_prep_result=$func_quote_for_eval_result
-}
-func_add_hook func_options_prep libtool_options_prep
 
 
-# libtool_parse_options [ARG]...
-# -
-# Provide handling for libtool specific options.
-libtool_parse_options ()
+# Parse options once, thoroughly.  This comes as soon as possible in the
+# script to make things like `--version' happen as quickly as we can.
 {
-$debug_cmd
-
 # Perform our own loop to consume as many options as possible in
 # each iteration.
 while test $# -gt 0; do
   _G_opt=$1
   shift
   case $_G_opt in
+
+--debug|-x)debug_cmd='set -x'
+   fu

Re: Performance issue of libtool-2.4.4

2015-02-09 Thread Richard Purdie
On Mon, 2015-02-09 at 10:45 +0800, Robert Yang wrote:
> On 02/06/2015 10:46 PM, Bob Friesenhahn wrote:
> > On Fri, 6 Feb 2015, Robert Yang wrote:
> >
> >>
> >>
> >> On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:
> >>> I am not seeing quite the difference between libtool releases that you are
> >>> although I see a big slowdown starting with 2.4.3.  These timings are for
> >>> optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 
> >>> 12:
> >>
> >> I think that we can't see obviously slowdown by "make -jN",
> >> but "make -j1" will. And bash is much slower than dash, I'm trying to
> >> figure out why.
> >
> > It seems like this issue is already corrected in the source tree but you are
> 
> Yes, I think that the git repo has fixed the problem:
> 
> commit 408cfb9c5fa8a666917167ffb806cb19deded429
> Author: Gary V. Vaughan 
> Date:   Fri Feb 6 12:58:34 2015 +
> 
>  libtool: don't execute automake and autoconf on every invocation.

In an effort to get to the bottom of this I made a git bisection, timing
the performance of building xz with make -j1 using each different
libtool.

The issues come down to this commit:

http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430

libtool: rewritten over funclib.sh instead of general.m4sh.

Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
in the fix in master mentioned above, I get 27s.

So whilst things are better (thanks!), the above change is still causing
a regression in the performance somewhere else. Any ideas what else in
that rather large change may be causing this?

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool


Re: Performance issue of libtool-2.4.4

2015-02-02 Thread Richard Purdie
On Mon, 2015-02-02 at 08:19 -0600, Bob Friesenhahn wrote:
> On Mon, 2 Feb 2015, Robert Yang wrote:
> > It seems that libtool (2.4.4) has increased the packages build time, after
> > a rough investigation, it maybe because new libtoolize needs run a m4
> > command:
> 
> Did you time 'libtoolize' and compare the timings with 2.4.4 to the 
> release you were using before?
> 
> Libtoolize is not normally considered to be part of package build time 
> since packages usually already come 'libtoolized' (using some random 
> version of libtool).

I work with Robert as part of the Yocto Project (powered by OpenEmbedded
underneath). The system can build Linux from scratch for a variety of
different architectures. As part of the process we autoreconf the
software so that any changes we make to support new architectures and so
on are applied as standard everywhere. We do patch libtool to deal with
some sysroot issues I've mentioned here in the past for example.

With 2.4.3, we saw a build time of 1 hour 12 minutes for one of our
standard test targets. With 2.4.4, this increased to 1 hour 23 minutes
and we've confirmed it is this specific change. We're in the process of
testing 2.4.5 but looking at the changes, I'm not sure this will help.

As yet we've not gotten to the point of isolating the problem to
libtoolize (that is the next step) but there is a noticeable regression
in the speed of things :(. 

Put in other terms why would libtool need 15% of the time needed to
build a whole Linux OS?! :)

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool


Re: Libtool 2.4.3 release

2014-03-20 Thread Richard Purdie
On Thu, 2014-03-20 at 18:07 +1300, Gary V. Vaughan wrote:
> On Mar 20, 2014, at 5:37 AM, Arnout Vandecappelle
>  wrote:
> > [Please keep me in CC, I'm not on the list]
> > Dear libtool maintainers,
>>
> > Is there a possibility for a new libtool release in the foreseeable
> > future?
>
> Yes, absolutely. In fact there are only 2 things ahead of it on my
> TODO list:
>
>   1. Figure out why a4ffcdb5e is a regression for test 57
>   2. fix test 120 race condition
>
> Unfortunately, Libtool is a complex beast, and we are woefully
> undermanned here.
> While everything rests on my shoulders, it will be at least another
> month before I can start work (I'm in the process of emigrating and
> all that entails).
>
> Patches for those 2 items, or any other as yet unknown issues with git
> master (or http://vaughan.pe/libtool/libtool-2.4.2.458.tar.gz if a
> bootstrapped tarball is easier to work with) are extremely welcome,
> and could lead to an immediate release...

FWIW I'm another build system maintainer who sees email here. We
currently have 12 patches against libtool:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/recipes-devtools/libtool/libtool

Some of these are inappropriate for upstream, one or two may have been
merged into libtool, some others may highlight issues, particularly
around the sysroot support.

Unfortunately whilst I have the best intentions (and am still on the
list), I haven't found the time to look into the issues as yet and
figure out if we could get some into a form that they'd be accepted
upstream.

Cheers,

Richard




___
https://lists.gnu.org/mailman/listinfo/libtool


Re: libtool - warnings when installing GMP using DESTDIR

2013-09-20 Thread Richard Purdie
On Thu, 2013-09-19 at 15:58 -0400, Michael Young wrote:
> I'm trying to do a staged build/install of libgmp(xx) release 5.1.2.
>  This is
> part of an effort to develop a "general" build system (mostly bash
> scripts, make
> files, etc.) for toolchains I will be using.  I intend to use these
> scripts to
> build both native and cross-builds, depending on configuration /
> arguments, so
> using DESTDIR for prefixing the install tree is important.  Right now,
> I'm 
> working on native builds on an Ubuntu 12.04 32-bit x86 machine;
> libtool 
> version = 2.4.2.)
> 
I can't help with the libtool issue but there are several build systems
out there that already have these issues solves such as the one I work
on:

http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html

which runs on Linux and can build cross compilers targeting the common
architectures and the cross compiler itself can run on linux, windows
and osx.

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool


Re: no .pc file

2012-10-26 Thread Richard Purdie
On Fri, 2012-10-26 at 13:27 -0500, Bob Friesenhahn wrote:
> On Fri, 26 Oct 2012, Yaroslav Bulatov wrote:
> 
> > Oops my badthat was a bad paste from some auto-generated code.
> >
> > This is basically a modified version of .pc file I get when building
> > zlib. Not sure how useful this is because you need to update "prefix"
> > in this file manually each time you rebuild libtool. Ideally the .pc
> > file would be generated automatically by configure/make
> 
> If it is not clear, package config files are operating system and/or 
> operating system distribution and/or operating system release version 
> specific.  One reason for this is that each operating system 
> distribution uses its own names for pkg-config package definitions. 
> Using zlib as an example is not useful since zlib does not depend on 
> any other packages.  Most packages depend on other packages and so 
> there is an OS-distribution (or even site-specific) list of packages 
> that this package depends on.
> 
> While many packages do produce sample pkg-config files (based on 
> guess-work or assumption of a partiticular OS distribuiton), it is 
> common practice for the default offering to be modified by the OS 
> package distribution maintainer because the OS uses a different name 
> for a similar thing.
> 
> Being intended for portable software, libtool does not concern itself 
> with a hand-edited/non-portable framework like pkg-config.

As I understand it, the .pc files use their own namespace so once a
given piece of software has chosen its naming, other things can depend
on it using that name space and it doesn't matter about the OS
distribution or OS version used. This is a clear incentive to maintain
the .pc file with the software so that there is one common naming used,
at least in pkg-config space. There is no connection to the package
management namespace which is totally separate.

This assumes that people use some kind of common sense when choosing
namespace but other than that it seems to work well.

As one of the people looking after the Yocto Project (which includes an
build system targeted at embedded devices), I have to say I see less
problems with pkg-config than with libtool and I'm once again being
asked to remove all .la files as a policy decision due to them causing
more problems than they seem to solve in a cross environment :(. I'm
running out of arguments against this, not least as I couldn't get any
response to the libtool sysroot problems I reported a while back.

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool


Re: Libtool sysroot RPATH problems

2011-01-13 Thread Richard Purdie
Hi Ralf,

Thanks for the reply.

On Thu, 2011-01-13 at 08:24 +0100, Ralf Wildenhues wrote:
> * Richard Purdie wrote on Wed, Jan 12, 2011 at 12:06:13AM CET:
> > Firstly, for the first time ever for us, it appears libtool is no longer
> > relinking our libraries at install time.
> 
> That's weird, I don't think the sysroot feature should've caused this
> change.  Not quite sure though.
> 
> > This is welcome as we're cross
> > compiling to a sysroot and we'd never want to actually run them in situ
> > so this could probably save us some build time. It does however mean we
> > never go for a relink step so we're hitting codepaths and issues that
> > I've never seen. For reference, we always used to set installed="no" in
> > our "staged" libraries prior to sysroot support since this let us hack
> > around the lack of sysroot support easier.
> > 
> > Anyhow, the problem I'm seeing is that the final library has an RPATH
> > including the sysroot prefix when make is calling libtool with
> > -rpath /usr/lib at link time. I looked at the code in ltmain.m4sh
> > starting at line 7240 (see the end of the email for a quotation) and
> > firstly, I don't understand why the func_replace_sysroot call is inside
> > the $hardcode_libdir_separator test? Surely it should be outside it and
> > happen whenever $hardcode_libdir_flag_spec is set?

Firstly let me be upfront and clear, I do have some patches applied on
top of libtool 2.4 to address some problems. These are:

http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/rename-with-sysroot.patch?h=sgarman/libtool-sysroot
(to stop conflicts with gcc/binutils configure options)
http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/prefix.patch?h=sgarman/libtool-sysroot
(renamed "libtool" to -libtool, which is mainly useful to
detect when our libtool patches aren't being noticed)
http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/trailingslash.patch?h=sgarman/libtool-sysroot
(a path comparison problem I never did get to the bottom of to report
properly but the fix appears harmless)
http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/use-sysroot-in-libpath.patch?h=sgarman/libtool-sysroot
(a sysroot fix I believe is in libtool git now)

I don't believe any of these are contributing to the issues I'm seeing
though.

http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/fix-final-rpath.patch?h=sgarman/libtool-sysroot
is a WIP patch which addresses some of the problems I refer to in this
email thread. Its not applied for the tests below.

> Please show the --mode={link,install} commands plus all of their output,
> also './libtool --config'.  Please also show how exactly you invoke
> configure.  Thanks.

A link command and output showing the problem:

/bin/sh ./i586-poky-linux-libtool --tag=CC   --mode=link i586-poky-linux-gcc 
-march=i586 
--sysroot=/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux  
-fexpensive-optimizations -fomit-frame-pointer -frename-registers -O2 -ggdb 
-feliminate-unused-debug-types -no-undefined -export-dynamic -version-number 
0:44:0 -Wl,--version-script=libpng.vers  -Wl,-O1 -Wl,--hash-style=gnu 
-Wl,--as-needed -o libpng12.la -rpath /usr/lib libpng12_la-png.lo 
libpng12_la-pngset.lo libpng12_la-pngget.lo libpng12_la-pngrutil.lo 
libpng12_la-pngtrans.lo libpng12_la-pngwutil.lo libpng12_la-pngread.lo 
libpng12_la-pngrio.lo libpng12_la-pngwio.lo libpng12_la-pngwrite.lo 
libpng12_la-pngrtran.lo libpng12_la-pngwtran.lo libpng12_la-pngmem.lo 
libpng12_la-pngerror.lo libpng12_la-pngpread.lo  -lz -lm 
i586-poky-linux-libtool: link: i586-poky-linux-gcc -march=i586 
--sysroot=/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux -shared  
-fPIC -DPIC  .libs/libpng12_la-png.o .libs/libpng12_la-pngset.o 
.libs/libpng12_la-pngget.o .libs/libpng12_la-pngrutil.o 
.libs/libpng12_la-pngtrans.o .libs/libpng12_la-pngwutil.o 
.libs/libpng12_la-pngread.o .libs/libpng12_la-pngrio.o 
.libs/libpng12_la-pngwio.o .libs/libpng12_la-pngwrite.o 
.libs/libpng12_la-pngrtran.o .libs/libpng12_la-pngwtran.o 
.libs/libpng12_la-pngmem.o .libs/libpng12_la-pngerror.o 
.libs/libpng12_la-pngpread.o   -Wl,-rpath 
-Wl,/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux/usr/lib 
/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux/usr/lib/libz.so -lm 
 -march=i586 
--sysroot=/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux -O2 
-Wl,--version-script=libpng.vers -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed   
-Wl,-soname -Wl,libpng12.so.0 -o .libs/libpng12.so.0.44.0
i586-poky-linux-libtool: link: (cd ".libs" && rm -f "libpng12.so.0

Libtool sysroot RPATH problems

2011-01-11 Thread Richard Purdie
Hi,

I work on OpenEmbedded/Poky and we're been experimenting with the recent
libtool sysroot support. To say I'm pleased to have this is an
understatement! We're having a few problems around incorrect RPATHs
being encoded into libraries however.

Firstly, for the first time ever for us, it appears libtool is no longer
relinking our libraries at install time. This is welcome as we're cross
compiling to a sysroot and we'd never want to actually run them in situ
so this could probably save us some build time. It does however mean we
never go for a relink step so we're hitting codepaths and issues that
I've never seen. For reference, we always used to set installed="no" in
our "staged" libraries prior to sysroot support since this let us hack
around the lack of sysroot support easier.

Anyhow, the problem I'm seeing is that the final library has an RPATH
including the sysroot prefix when make is calling libtool with
-rpath /usr/lib at link time. I looked at the code in ltmain.m4sh
starting at line 7240 (see the end of the email for a quotation) and
firstly, I don't understand why the func_replace_sysroot call is inside
the $hardcode_libdir_separator test? Surely it should be outside it and
happen whenever $hardcode_libdir_flag_spec is set?

Changing that helps a bit and I end up with an RPATH of "=/usr/lib" so
the sysroot prefix is gone but the "=" is there. I suspect that isn't
valid so I added the following code after the func_replace_sysroot call

func_stripname '=' '' "$libdir"
libdir=$func_stripname_result

and then I get an RPATH of "/usr/lib" which is ok to a point.

Of course this is listed in $sys_lib_dlsearch_path_spec so shouldn't be
getting added at all. Its coming to this function via $compile_rpath
which is being set around line 5939 which in turn is coming from absdir.
absdir is being compared against $sys_lib_dlsearch_path_spec but without
adding/removing any sysroot prefixes.

I'd really therefore like to ask what behaviour to be expecting from
libtool before I try and write patches to fix any of this. To summarise
some of my questions:

a) If I'm using a sysroot can I expect to install without a relink step 
   (I hope so!)
b) I couldn't see a function to add/prepend a sysroot without a = in 
   there. I assume given these usecases we should add one?
c) An RPATH starting with an "=" is invalid, correct?
d) Should the absdir comparisions with sys_lib_dlsearch_path_spec have 
   the sysroot stripped? Are there other rpath variables that need this 
   treatment when comparing to sys_lib_dlsearch_path_spec?
e) The sysroot treatment should apply whenever 
   $hardcode_libdir_flag_spec is set and not on 
   $hardcode_libdir_separator?

If someone could explain the correct behaviour I might be able to come
up with some patches to help fix things! :)

Cheers,

Richard

http://git.savannah.gnu.org/cgit/libtool.git/tree/libltdl/config/ltmain.m4sh?id=9167aecabd12c5afe7a65d45dc73f8c92ab42f05
line 7240:
  # Test again, we may have decided not to build it any more
  if test "$build_libtool_libs" = yes; then
# Remove ${wl} instances when linking with ld.
# FIXME: should test the right _cmds variable.
case $archive_cmds in
  *\$LD\ *) wl= ;;
esac
if test "$hardcode_into_libs" = yes; then
  # Hardcode the library paths
  hardcode_libdirs=
  dep_rpath=
  rpath="$finalize_rpath"
  test "$opt_mode" != relink && rpath="$compile_rpath$rpath"
  for libdir in $rpath; do
if test -n "$hardcode_libdir_flag_spec"; then
  if test -n "$hardcode_libdir_separator"; then
func_replace_sysroot "$libdir"
libdir=$func_replace_sysroot_result
if test -z "$hardcode_libdirs"; then
  hardcode_libdirs="$libdir"
line 5929:
  if test "$linkmode" = lib &&
 test "$hardcode_into_libs" = yes; then
# Hardcode the library path.
# Skip directories that are in the system default run-time
# search path.
case " $sys_lib_dlsearch_path " in
*" $absdir "*) ;;
*)
  case "$compile_rpath " in
  *" $absdir "*) ;;
  *) func_append compile_rpath " $absdir" ;;
  esac
  ;;
esac


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Issues w/ "relink" and cross-compilation

2010-07-12 Thread Richard Purdie
Hi,

I've spoken about this in the past but it looks like I should mention it
again. I do extensive cross compiling of the entire Linux software stack
using OpenEmbedded and Poky (I maintain the latter). We have cross
compiling working with libtool with a variety of workarounds. This
mainly consists of the following simple patch to libtool itself:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool/cross_compile.patch

and then manipulation of the .la files with sed if we ever want to move
the sysroot directory, or use the .la files on the target device.

There are a couple of other patches we use. One is to allow multiple
libtool scripts to live in the same place by adding a cross prefix to
the tool name:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool/prefix.patch

This patch also allows us to tell when the "wrong" (unpatched) libtool
is being used. 

The final patch we're currently applying is probably a bug in libtool:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool/trailingslash.patch

where a directory comparison was failing as one variable ended in a
slash and the other did not. We're fixing this by stripping any slash
off both options (the patch is pending an update to do this). I haven't
isolated a proper test case to be able to submit this one as a proper
bug yet though.

Longer term I'm still hoping we'll see sysroot support in libtool and I
may even find some time to start the ball rolling on patches eventually
but I'm not going to get to that soon, much as I wish it were otherwise.

Cheers,

Richard


___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Issue with libtool when cross-compiling

2010-02-12 Thread Richard Purdie
On Fri, 2010-02-12 at 19:34 +0100, Thomas Petazzoni wrote:
> I suppose you apply these modifications to a libtool that you are
> compiling. However, libtool is typically already embedded in each
> upstream tarball. Do you run autoreconf on all packages, so that a
> newer libtool script gets generated (autoreconf-ing each package takes a
> lot of time) ? Do you somehow replace the provided libtool by a fixed
> one ?
> 
> Moreover, some packages use libtool 1.5.x and some newer packages use
> libtool 2.2.x. How do you handle this version difference ?

Due to exactly this kind of problem OpenEmbedded generally autoreconfs
all sources.

> > There are other workarounds we have to apply after installing
> > packages, particularly modifying the .la files to be uninstalled to
> > get the sysroot to work properly.
> 
> « to be uninstalled » ?

Set installed=no in the .la files.

> In Buildroot, we already modify the libdir='' in all the .la files so
> that they refer to the sysroot instead of /usr/lib.

We change the dependency_libs line. The libdir line refers to the final
install location and /usr/lib is in fact valid there.

> If possible, could you point me to the location in OpenEmbedded where
> libtool is handled ? I already had a look at your patches, but don't
> understand how you re-generate the libtool script inside the different
> upstream packages.

meta/classes/autoconf.bbclass has the autoreconf code

meta/classes/base.bbclass has the .la file manipulation.

What I find frustrating is that buildroot is playing catchup to OE these
days. By the time you make all the improvements to buildroot, you end up
with OE.

What would it take to get the buildroot people using OE and focusing
effort to new problems, not ones already at least partially solved?

I'd love to see a kconfig wrapper to OE for example...

Cheers,

Richard





___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Issue with libtool when cross-compiling

2010-02-12 Thread Richard Purdie
On Fri, 2010-02-12 at 16:34 +0100, Thomas Petazzoni wrote:
> I'm a contributor to the Buildroot project (http://www.buildroot.org),
> a build system for embedded Linux systems. We integrate many packages
> in order to ease their cross-compilation.
> 
> I'm facing an issue with several packages where libtool wants to link
> against libraries of the host (in /usr/lib), and I don't know what
> is happening. It is very likely a problem in the environment variables
> or the options we pass to the configure script, but I'm unable to find
> where the problem is, and thought that libtool experts might have an
> idea on what's going on.

libtool has known cross compile issues and doesn't get on well with
sysroots. 

I maintain the OpenEmbedded libtool patch set which at least lets us
work around the worst libtool issues. The main patch is:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool-2.2.6/cross_compile.patch

and there is a bugfix I've been meaning to look at in more detail which
became neccessary with recent autoconf/automake versions:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool-2.2.6/trailingslash.patch

where a trailing slash on paths causes a string comparison to fail. We
also apply: 

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool-2.2.6/prefix.patch

which renames libtool and allows us to easily tell when the wrong
(unpatched) libtool is being used.

There are other workarounds we have to apply after installing packages,
particularly modifying the .la files to be uninstalled to get the
sysroot to work properly. I'd love to teach libtool about sysroots and
make it work properly but as yet have never had the time to look into
it.

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool


Re: removal of .la files from Debian and a possible solution to the libtool shared libs problem

2009-08-25 Thread Richard Purdie
On Tue, 2009-08-25 at 20:44 +0200, Ralf Wildenhues wrote:
> * Bob Friesenhahn wrote on Tue, Aug 25, 2009 at 05:17:49PM CEST:
> > On Tue, 25 Aug 2009, Anssi Hannula wrote:
> > >
> > >I think the proper way to solve this is to not link to dependency_libs
> > >when linking dynamically on systems where it is not needed to link to
> > >those. I haven't seen any correctly working patches that implement this.
> > 
> > Relying on the OS's implicit dependency features seems to be an
> > approach which is fraught with peril.
> 
> With GNU/Linux, and libraries all being in directories searched by
> default by both the link editor and the runtime linker, the problems
> are fairly limited.  IIRC Debian requires that you link directly against
> all libraries that you require directly.
> 
> The problems start as soon as you link (directly or indirectly) against
> libraries in directories not searched by default.  IOW: typically
> anything not provided by a properly packaged Debian package, installed
> by the user or the system maintainer.

Surely at least on Linux the -rpath linker option would be a much better
way to solve this?

Linux does seem to have good dynamic linker support and its a shame
libtool effectively drags it down to a lower common denominator of other
platforms with worse support.

Cheers,

Richard

-- 
Richard Purdie
Intel Open Source Technology Centre



___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Crosscompiling fails since Gstreamer moved to new libtool version

2009-02-26 Thread Richard Purdie
On Thu, 2009-02-26 at 14:16 +0100, Andreas Frisch wrote:
> the guys from gstreamer went through and moved to a newer libtool
> version in their plugin packages. since then i can't crosscompile
> anymore them with openembedded like with the previous releases. i
> described the problem here:
> http://bugzilla.gnome.org/show_bug.cgi?id=572532
> unfortunately, none of the gstreamer guys as able to help me with this
> libtool-specific issue. i found out that the main libtool code moved
> out of aclocal.m4 into a new file libtool.m4 in the m4 subdir of the
> source directory. i couldn't get configure to create the
> mipsel-linux-libtool instead of regular "libtool" though, not even by
> changing the line default_ofile=${host_alias}-libtool.
> maybe any of you can give me hint how i could solve my issue - thanks
> in advance!

As a warning the creation of "mipsel-linux-libtool" is an OpenEmbedded
specific patch and I'd suspect is not supported by libtool upstream.
Which version of libtool are you using in OpenEmbedded?

For reference in Poky, the gstreamer recipes have:

"""
do_configure_prepend() {
# This m4 file contains nastiness which conflicts with libtool 2.2.2
rm ${S}/m4/lib-link.m4 || true
}
"""

suggesting there was something in that m4 file that caused problems.

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool


Re: how to link with libtool?

2009-02-03 Thread Richard Purdie
On Tue, 2009-02-03 at 11:05 +0100, Matěj Týč wrote:
> > Yes, it may be a good idea if somebody wrote this.
> > It should probably depend on LT_OUTPUT.
> >
> > OTOH, the havelib module from gnulib already provides quite a bit of
> > functionality in this area.
> 
> OK, but what should I tell to the library users? Using gnulib is quite
> troublesome since
> it does not have proper documentation and usage of the library would become 
> too
> complicated for a casual programmer.
> And I don't like pkg-config since it breaks cross-compilation...

Just for reference, pkg-config should not break cross-compilation after
the addition of sysroot support to it. I keep meaning to look into
sysroot support for libtool too, I just haven't had the time yet :(.

Cheers,

Richard

-- 
Richard Purdie
Intel Open Source Technology Centre



___
http://lists.gnu.org/mailman/listinfo/libtool


Re: [PATCH] Add 64 bit directories to sys_lib_dlsearch_path_spec for Linux ELF

2009-01-28 Thread Richard Purdie

On Wed, 2009-01-28 at 12:01 -0800, Dan Nicholson wrote:
> If they follow the LSB, then they should be using lib64. However, you
> can configure the linker any number of ways. I tried to get a patch to
> glibc so this path could be determined, but it was rejected.
> 
> http://sourceware.org/ml/libc-alpha/2008-12/msg00052.html
> 
> I think this is the only reasonable way to proceed, but I suppose that
> an existence test would be safe, too:
> 
> sys_lib_dlsearch_path_spec="/lib /usr/lib"
> case `/usr/bin/file conftest.o` in
> *64-bit*)
> for dir in /lib64 /usr/lib64; do
> test -d $dir &&
> sys_lib_dlsearch_path_spec="$sys_lib_dlsearch_path_spec $dir"
> done
> ;;
> esac

Just as a datapoint, my standard ubuntu 64 bit desktop has /lib64 as a
symlink to /lib which has 64 bit libraries in it.

Cheers,

Richard

-- 
Richard Purdie
Intel Open Source Technology Centre



___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Compiling into chroot

2008-06-12 Thread Richard Purdie

On Thu, 2008-06-12 at 22:03 +0300, Alon Bar-Lev wrote:
> On 6/12/08, Roumen Petrov <[EMAIL PROTECTED]> wrote:
> >  It look like an enhancement request. libtool to obey as example
> > FAKEROOT=/tmp/device-root and to look first in $FAKEROOT/path1
> >  , ... and $FAKEROOT/pathN and later in /path1,... and /pathN .
> 
> This what I had in mind.
> It also should append the FAKEROOT to pathes read from .la files.

You mean prepend :)

This is the same issue as the sysroot one I've previously mentioned.
OpenEmbedded currently hacks the .la files in its staging area to
include the sysroot prefix but this means you can't use staging as a
chroot. I've not had time to look into doing anything about this feature
but I would love to see it!

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool


Re: libtool 2.2.2, darwin and target prefixes

2008-05-02 Thread Richard Purdie
On Fri, 2008-05-02 at 01:03 -0500, Peter O'Gorman wrote:
> Peter O'Gorman wrote:
> > Richard Purdie wrote:
> >> Hi,
> >>
> >> As previously mentioned, I've been stress testing libtool 2.2.2 with
> >> Poky a bit.
> >>
> >> I've found one issue when I tested the darwin builds, specifically that
> >> it tried to use "otool" and "otool64" and not the versions with the host
> >> prefix which my setup has. I fixed this with the patch below which is
> >> fine for Poky since its always cross compiling but its a sign a better
> >> fix is probably needed in general.
> > 
> > Thanks,
> > I have pushed this, it also cleans up the sed sed echo ickyness.
> 
> It is always better if quotes get closed. I had tested, honest!
> 
> Pushed this.

I've updated Poky to use the proper fixes, thanks for looking at this!

Cheers,

Richard





___
http://lists.gnu.org/mailman/listinfo/libtool


libtool 2.2.2, darwin and target prefixes

2008-05-01 Thread Richard Purdie
Hi,

As previously mentioned, I've been stress testing libtool 2.2.2 with
Poky a bit.

I've found one issue when I tested the darwin builds, specifically that
it tried to use "otool" and "otool64" and not the versions with the host
prefix which my setup has. I fixed this with the patch below which is
fine for Poky since its always cross compiling but its a sign a better
fix is probably needed in general.

Regards,

Richard

Index: libtool-2.2.2/libltdl/config/ltmain.m4sh
===
--- libtool-2.2.2.orig/libltdl/config/ltmain.m4sh   2008-05-01 
12:19:37.0 +0100
+++ libtool-2.2.2/libltdl/config/ltmain.m4sh2008-05-01 12:29:05.0 
+0100
@@ -4965,10 +4965,10 @@
done
if test -f "$absdir/$objdir/$depdepl" ; then
  depdepl="$absdir/$objdir/$depdepl"
- darwin_install_name=`otool -L $depdepl | $SED -n -e 
'3q;2,2p' | $SED -e 's/(.*//'`
+ darwin_install_name=`$host-otool -L $depdepl | $SED -n -e 
'3q;2,2p' | $SED -e 's/(.*//'`
  darwin_install_name=`$ECHO $darwin_install_name`
   if test -z "$darwin_install_name"; then
-  darwin_install_name=`otool64 -L $depdepl | $SED -n 
-e '3q;2,2p' | $SED -e 's/(.*//'`
+  darwin_install_name=`$host-otool64 -L $depdepl | 
$SED -n -e '3q;2,2p' | $SED -e 's/(.*//'`
   darwin_install_name=`$ECHO $darwin_install_name`
   fi
  compiler_flags="$compiler_flags ${wl}-dylib_file 
${wl}${darwin_install_name}:${depdepl}"



___
http://lists.gnu.org/mailman/listinfo/libtool


Re: libtool 2.2.2, ccache and -all-static

2008-04-22 Thread Richard Purdie
Hi Ralf,

On Tue, 2008-04-22 at 21:43 +0200, Ralf Wildenhues wrote:
> Thanks for the bug report, and especially for providing an example to
> reproduce it!
> 
> > libtool: link: ccache -static gcc -O20 -ffast-math -D_REENTRANT 
> > -fsigned-char -DUSE_MEMORY_H -o decoder_example decoder_example.o  
> > ../lib/.libs/libvorbis.a -lm /usr/lib/libogg.a
> 
> Confirmed.  Fixed as below, committed, put you in THANKS.

I've confirmed the fix, added it to Poky and dropped the hacks I had,
thanks!

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool


libtool 2.2.2, ccache and -all-static

2008-04-22 Thread Richard Purdie
Hi,

I've noticed another problem with two packages in poky, prelink and
libvorbis. Both packages have areas where LDFLAGS="-all-static" is used.

The problem comes about since Poky sets CC to "ccache gcc", then libtool
puts the -static flag between ccache and gcc.

To reproduce:

wget http://www.vorbis.com/files/1.0.1/unix/libvorbis-1.0.1.tar.gz
tar -xvzf /usr/oe/sources/libvorbis-1.0.1.tar.gz
cd libvorbis-1.0.1
autoreconf -i
CC="ccache gcc" ./configure
make

which results in:

make[1]: Entering directory `/usr/src/libvorbis-1.0.1/examples'
ccache gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" 
-DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"libvorbis\" 
-DVERSION=\"1.0.1\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
-DLT_OBJDIR=\".libs/\" -DHAVE_ALLOCA_H=1 -DHAVE_ALLOCA=1 -I. -I../include 
-O20 -ffast-math -D_REENTRANT -fsigned-char  -DUSE_MEMORY_H -MT 
decoder_example.o -MD -MP -MF .deps/decoder_example.Tpo -c -o decoder_example.o 
decoder_example.c
mv -f .deps/decoder_example.Tpo .deps/decoder_example.Po
/bin/sh ../libtool --tag=CC   --mode=link ccache gcc  -O20 -ffast-math 
-D_REENTRANT -fsigned-char  -DUSE_MEMORY_H  -all-static -o decoder_example 
decoder_example.o ../lib/libvorbis.la -lm  -logg
libtool: link: ccache -static gcc -O20 -ffast-math -D_REENTRANT -fsigned-char 
-DUSE_MEMORY_H -o decoder_example decoder_example.o  ../lib/.libs/libvorbis.a 
-lm /usr/lib/libogg.a

which then fails with the ccache help message since it doesn't support a
"-static" option. Is it possible to fix this so the flag is properly
handled?

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool


Problems with libtool 2.2.2 and /bin/sh pointing to dash

2008-04-21 Thread Richard Purdie
Hi,

I've had reports of Poky breaking with libtool 2.2.2 and have isolated
this to people using dash as their /bin/sh provider (Poky runs the
configure script with /bin/sh). When used in this combination, the
global_symbol_pipe expression becomes corrupted in the generated libtool
file amongst other things and I've included a diff of the corruption
below. I noticed this with gtk+ 2.12.7. 

gtk+ also has the issues that it tries to run libtool before its been
generated and I've had to patch this to run a previously generated
version of libtool poky has around to solve cases like this. I'm not
sure if there is a neater way to address that problem?

Regards,

Richard

--- libtool-bash2008-04-21 17:57:40.0 +0100
+++ libtool-dash2008-04-21 23:36:38.0 +0100
@@ -1,4 +1,4 @@
-#! /bin/sh
+#! /bin/bash
 
 # arm-poky-linux-gnueabi-libtool - Provide generalized library-building 
support services.
 # Generated automatically by config.status (gtk+) 2.12.7
@@ -107,10 +107,10 @@
 lt_unset=unset
 
 # turn spaces into newlines.
-SP2NL="tr \\040 \\012"
+SP2NL="tr \040 \012"
 
 # turn newlines into spaces.
-NL2SP="tr \\015\\012 \\040\\040"
+NL2SP="tr \015\012 \040\040"
 
 # How to create reloadable object files.
 reload_flag=" -r"
@@ -141,22 +141,22 @@
 LTCFLAGS="-fexpensive-optimizations -fomit-frame-pointer -frename-registers 
-O2 -Wall"
 
 # Take the output of nm and produce a listing of raw symbols and C names.
-global_symbol_pipe="sed -n -e 's/^.*[   ]\\([ABCDGIRSTW][ABCDGIRSTW]*\\)[  
 ][  ]*\\([_A-Za-z][_A-Za-z0-9]*\\)\$/\\1 \\2 \\2/p'"
+global_symbol_pipe="sed -n -e 's/^.*[   ]\\([ABCDGIRSTW][ABCDGIRSTW]*\\)[  
 ][  ]*\\([_A-Za-z][_A-Za-z0-9]*\\)\$/  /p'"
 
 # Transform the output of nm in a proper C declaration.
-global_symbol_to_cdecl="sed -n -e 's/^T .* \\(.*\\)\$/extern int \\1();/p' -e 
's/^[ABCDGIRSTW]* .* \\(.*\\)\$/extern char \\1;/p'"
+global_symbol_to_cdecl="sed -n -e 's/^T .* \\(.*\\)\$/extern int ();/p' -e 
's/^[ABCDGIRSTW]* .* \\(.*\\)\$/extern char ;/p'"
 
 # Transform the output of nm in a C name address pair.
-global_symbol_to_c_name_address="sed -n -e 's/^: \\([^ ]*\\) \$/  
{\\\"\\1\\\", (void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ ]*\\)\$/  
{\"\\2\", (void *) \\&\\2},/p'"
+global_symbol_to_c_name_address="sed -n -e 's/^: \\([^ ]*\\) \$/  {\\\"\\\", 
(void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ ]*\\)\$/  {\"\", (void 
*) \\&},/p'"
 
 # Transform the output of nm in a C name address pair when lib prefix is 
needed.
-global_symbol_to_c_name_address_lib_prefix="sed -n -e 's/^: \\([^ ]*\\) \$/  
{\\\"\\1\\\", (void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\(lib[^ 
]*\\)\$/  {\"\\2\", (void *) \\&\\2},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ 
]*\\)\$/  {\"lib\\2\", (void *) \\&\\2},/p'"
+global_symbol_to_c_name_address_lib_prefix="sed -n -e 's/^: \\([^ ]*\\) \$/  
{\\\"\\\", (void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\(lib[^ ]*\\)\$/  
{\"\", (void *) \\&},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ ]*\\)\$/  
{\"lib\", (void *) \\&},/p'"
 
 # The name of the directory that contains temporary libtool files.
 objdir=.libs
 
 # Shell to use when invoking shell scripts.
-SHELL="/bin/sh"
+SHELL="/bin/bash"
 
 # An echo program that does not interpret backslashes.
 ECHO="echo"
@@ -301,7 +301,7 @@
 # Commands used to build a shared archive.
 archive_cmds="\$CC -shared \$libobjs \$deplibs \$compiler_flags \${wl}-soname 
\$wl\$soname -o \$lib"
 archive_expsym_cmds="echo \\\"{ global:\\\" > \$output_objdir/\$libname.ver~
-   cat \$export_symbols | sed -e \\\"s/(.*)/1;/\\\" >> 
\$output_objdir/\$libname.ver~
+   cat \$export_symbols | sed -e \\\"s/(.*)/;/\\\" >> 
\$output_objdir/\$libname.ver~
echo \\\"local: *; };\\\" >> \$output_objdir/\$libname.ver~
\$CC -shared \$libobjs \$deplibs \$compiler_flags \${wl}-soname 
\$wl\$soname \${wl}-version-script \${wl}\$output_objdir/\$libname.ver -o \$lib"
 



___
http://lists.gnu.org/mailman/listinfo/libtool


Re: libtool.m4 not always copied

2008-04-16 Thread Richard Purdie
On Wed, 2008-04-16 at 23:56 +0200, Ralf Wildenhues wrote:
> * Richard Purdie wrote on Wed, Apr 16, 2008 at 07:10:54PM CEST:
> > To 'fix' the echo issue I added:
> > 
> > http://svn.o-hand.com/view/poky/trunk/meta/packages/libpng/libpng-1.2.16/makefile_fix.patch?rev=4267&view=markup
> > 
> > although I suspect echo = @ECHO@ might be better. I think libpng has
> > relied on old libtool behaviour which happened to export that and no
> > longer does so its a libpng bug but I'm open to more informed comments!
> 
> Sorry, but your patch just hides the real bug that you're using macro
> files from Libtool 1.5.x.  You will see other, more obscure bugs (but
> then again, with their nature of being obscure, you may not see them,
> but curse a lot about how broken Libtool is).
> 
> Please drop the patch and get updated macros into your tree.

I am not using 1.5 macros. libtool 2.2.2 is being used, as I can see if
I grep aclocal.m4 or ltmain.sh for the libtool version.

Let me try again to explain the problem. Makefile.am contains:

$(ECHO) [EMAIL PROTECTED]@@[EMAIL PROTECTED] '{global:' > [EMAIL 
PROTECTED]
$(SED) s/$$/\;/ libpng.sym >> [EMAIL PROTECTED]
$(ECHO) 'local: *; };' >> [EMAIL PROTECTED]

automake expands ECHO_C, ECHO_N, ECHO_T and lt_ECHO but not ECHO
anymore. libtool 1.5 macros happened to let ECHO be expanded but in
2.2.2 this is lt_ECHO.

Since nothing expands ECHO anymore this breaks with libtool 2.2.2.

I think that libpng is in the wrong here though and it therefore needs
to add something to the Makefile.am to cause ECHO to expand.

Cheers,

Richard





___
http://lists.gnu.org/mailman/listinfo/libtool


Re: libtool.m4 not always copied

2008-04-16 Thread Richard Purdie
Hi,

On Wed, 2008-04-16 at 19:03 +0200, Vincent Torri wrote:
> I'm using libtool 2.2.3a (cvs, actually) and i'm trying to find out why 
> some lib are not configured correctly, while other are.
> 
> My problem is that ECHO and OBJDUMP are not defined when I'm configuring 
> libpng 1.2.26, while it's perfectly defined with another lib. I'm doing 
> cross-compiling for the cegcc compiler (i've improved the patch i've sent 
> to the libtool patch ML, but i want to solve all the problems i encounter 
> before sending it again).
> 
> I've grep'ed OBJDUMP and i've remarked that with libpng, it's never 
> defined, and in addition libtool.m4 and other m4 files that lib should 
> (must ?) copy are not in the libpng directory. I don't know if it's 
> related, but i prefer asking anyway :)
> 
> I have the same problem with ECHO, which is not defined too.
> 
> With the other lib, there is no problem at all. The m4 files are correctly 
> copied and used.
> 
> So I would like to know why libtoolize does not copy those files with the 
> libpng library. And if someone knows why OBJDUMP is not defined, i would 
> be glad to know :)

I'm in the process of upgrading Poky to use libtool 2.2.2 and noticed
the echo problem. libtool.m4 was correctly added to aclocal.m4 though
and I didn't see the OBJDUMP problem.

To 'fix' the echo issue I added:

http://svn.o-hand.com/view/poky/trunk/meta/packages/libpng/libpng-1.2.16/makefile_fix.patch?rev=4267&view=markup

although I suspect echo = @ECHO@ might be better. I think libpng has
relied on old libtool behaviour which happened to export that and no
longer does so its a libpng bug but I'm open to more informed comments!

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool


Re: sysroot support in libtool

2008-04-13 Thread Richard Purdie
On Sun, 2008-04-13 at 22:26 +0200, Ralf Wildenhues wrote:
> * Richard Purdie wrote on Sun, Apr 13, 2008 at 08:53:10PM CEST:
> > 
> > The patches we're using are publicly available as:
> > http://svn.o-hand.com/view/poky/trunk/meta/packages/libtool/libtool-1.5.10/
> 
> Let's take a look at the simple ones:
> 
> > nmedit_fix.patch - Call host-triplet-nmedit, not just "nmedit"
> 
> This patch shouldn't be needed any more with 2.2.2.

Agreed, the nmedit code looks much neater now!

> > sedvar.patch - Fixes some issue we saw when $SED wasn't set
> 
> Shouldn't be needed any more either.

Agreed.

> > tag.patch - The tag errors were breaking things for no good reason so we
> > turned the error into a warning
> 
> Well, don't come crying for bad performance to us if you do things like
> this.  Letting libtool infer the tag is rather expensive, at least
> relatively, for 2.2.2 on modern systems where there are few other forks
> needed.

Going totally from memory, I think this exists since we have CC set to
things like:

CC="gcc -march=armv5te -mtune=xscale"

and then this might change to

CC="gcc -march=armv5te -mtune=arm926ejs"

which is 100% binary compatible, just optimised differently. libtool
upon seeing CC change gets "upset". Sadly I can't remember how libtool
notices this change, it much be through some kind of state information
libtool saves into our staging area.

Thanks for the pointer on this though, I will look into it and find some
way to avoid it. 

> > uclibc.patch - Tweaks to libtool.m4 to support uclibc
> 
> If this happens to be needed for 2.2.2 as well, I'd like this one
> submitted to libtool-patches, ideally including a ChangeLog entry
> and testsuite results for this system, thanks.

Ok, I'll keep this in mind when updating and submit it if its still
needed.

Just for reference,
http://svn.o-hand.com/view/poky/trunk/meta/packages/libtool/libtool-2.2.2/
are the needed patches updated for 2.2.2, albeit rather untested at the
moment.

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool


Re: sysroot support in libtool

2008-04-13 Thread Richard Purdie
Hi Ralf,

On Sun, 2008-04-13 at 17:09 +0200, Ralf Wildenhues wrote:
> * Richard Purdie wrote on Sun, Apr 13, 2008 at 04:59:26PM CEST:
> > 
> > * the person who integrated libtool into OE has moved onto other things
> >   and the knowledge for a lot of the "magic" was lost
> > * we're stuck on an old version of libtool (1.5.10) which we know you 
> >   won't be interested in bugs against
> 
> > * this old version "works" [...]
> 
> No it doesn't.  If it did, you wouldn't be adopting dolt.
> OK, you put "works" in quotes, so I guess that's what you meant.

"works" means that with our patched 1.5.10 libtool OE/Poky can generate
working cross compiled binaries and images. Its in quotes since I don't
like the way we do it but having working builds with some ugliness
behind the scenes is better than ones that don't work!

dolt comes in purely as a speedup, it doesn't make the .la file and
sysroot situation better or worse.

> > In the meantime I will try and get Poky/OE using a much more recent
> > libtool, patched enough so builds continue to work as described above.
> > Once we get a modern version working it will be much easier to test
> > patches and improvements in libtool and from what I've read, we should
> > see some performance improvements which would be most welcome. We don't
> > have patches to add sysroot support, just the sed hacks I've mentioned
> > but if you'd be interested in proper sysroot support that is something
> > we'd be interested in collaborating on.
> 
> Even posting the sed hacks or the diffs against 1.5.10 that you were
> using (both to libtool-patches, please) are better than nothing.

I'll post here first, since I don't class any of these are ready to
apply and really need discussion. If you still feel libtool-patches is
more appropriate for that tell me and I'll switch lists though.

The patches we're using are publicly available as:
http://svn.o-hand.com/view/poky/trunk/meta/packages/libtool/libtool-1.5.10/

I've run through each with a quick description of what it does

3figures.patch - a better version was merged, not needed with 2.2.2

add_dolt.patch - hacks dolt into libtool.m4 instead of patching every
configure.ac. Inappropriate for upstream, mentioned for reference only.

autotools.patch - Make the old libtool work with more modern autotools.
Not needed with 2.2.2.

install-path-check.patch - OE used to have a staging layout which didn't
match the target system. Since we use sysroot this shouldn't be needed
anymore.

libdir-la.patch - See comments in the patch. Doesn't add libdir to the
search path, don't complain about 'moved' files, don't replace
uninstalled with installed libraries

libdir-la2.patch - Improves on the above to stop libdir leaking into
search paths and checking more locations in the installed=no case.

nmedit_fix.patch - Call host-triplet-nmedit, not just "nmedit"

nousrlib.patch - Stop libdir leaking into linker flags

prefix.patch - Rename libtool to be prefixed by the host triplet which
makes it much more obvious when the wrong libtool is being used and is
in line with autotools wanting cross tool names prefixed with the
triplet.

sedvar.patch - Fixes some issue we saw when $SED wasn't set

tag.patch - The tag errors were breaking things for no good reason so we
turned the error into a warning

uclibc.patch - Tweaks to libtool.m4 to support uclibc


I don't have the full background on all of these and have only checked
whether some are needed against 2.2.2, I'll aim to do the rest soon and
try to get 2.2.2 working. The core patches are the libdir-la.patch,
lidir-la2.patch and prefix.patch, all of which I know are still needed.

The sed magic I mentioned is in
http://svn.o-hand.com/view/poky/trunk/meta/classes/base.bbclass?rev=4064&view=markup

specifically:

sed -e 's/^installed=yes$/installed=no/' \
 -e 
'/^dependency_libs=/s,${WORKDIR}[[:alnum:]/\._+-]*/\([[:alnum:]\._+-]*\),${STAGING_LIBDIR}/\1,g'
 \
 -e "/^dependency_libs=/s,\([[:space:]']+\)${libdir},\1${STAGING_LIBDIR},g" 
\
 $dotlai >$destpath/$libname.la

which changes installed=yes to installed=no and cleans up
dependency_libs so it only refers to 'staging' (the sysroot), not the
work directories (which are temporary in nature) or libdir (which are
binaries from the wrong architecture).

> Proper sysroot support in Libtool would be very welcome, but
> unfortunately it won't be easy.  Of course there are more constraints:
> it should work right in most practical cases, not just the ones your
> package cares about.  Since it cannot be made to work on some systems
> due to linker and binary file format limitations,

Re: sysroot support in libtool

2008-04-13 Thread Richard Purdie
Hi,

On Sun, 2008-04-13 at 08:21 -0400, Gary V. Vaughan wrote:
> On 13 Apr 2008, at 07:55, Richard Purdie wrote:
> > [1] Are there any plans to support sysroots with libtool?
> 
> No one is sending us bug reports or patches... so we don't even know
> there is a problem!

Ok, there are some fairly long running issues with OpenEmbedded and
libtool. I appreciate there has not been feedback to upstream about the
problems and this is entirely our fault. There are various reasons for
this:

* the person who integrated libtool into OE has moved onto other things
  and the knowledge for a lot of the "magic" was lost
* we're stuck on an old version of libtool (1.5.10) which we know you 
  won't be interested in bugs against
* this old version "works" and nobody has found the time for the 
  learning curve and debugging involved in upgrading

More recently a lot of things have changed for the better in OE and
we're striving for a clean logical build structure rather than hacks,
I've been one of the people trying to achieve this. One way we've done
this is embracing the sysroot options in gcc/binutils and getting
sysroot support into pkg-config.

So this leaves us with the problem of libtool. Why a problem?

Lets give an example of compiling some library which uses libtool and
autotools:

* We configure it with libdir=/usr/lib which is where the library will
be on the target system ultimately.
* Compile it using a gcc cross compiler and binutils which have been
configured with /somewhere/sysroot as the sysroot. This means they don't
look in /usr/lib and /usr/include, they look
in /somewhere/sysroot/usr/lib and /somewhere/sysroot/usr/include.
* We "make install DESTDIR=/someplace" and then build packages out
of /someplace
* We "make install DESTDIR=/somewhere/sysroot" which adds the library to
our sysroot environment so something using this lib can compile against
it.

The problem is this last step which installs .la files into our sysroot
which don't work. The issues we can have:

* The .la files say installed=yes but they're in the sysroot, not in
libdir.
* dependency_libs= can contain an expanded version of libdir. This is
wrong and we have to prepend the sysroot to this.
* libdir itself can be added to the search path for libraries which
finds things from /usr/lib on the host system - really bad when cross
compiling

To workaround these issues we sed the sysroot .la files to:

* change installed=yes to installed=no, it changes some of the logic
within libtool and breaks some assumptions about the files being in
libdir which helps.
* prepend sysroot to the appropriate parts of dependency_libs

and patch libtool to:

* remove the parts which add libdir into the search path
* cope with the installed=no logic and search both the sysroot and the
app being compiled

The above is the situation with 1.5.10. I've just tried a build with a
clean 2.2.2 and it broke with at least one of the problems mentioned
above, I don't know if all the above issues are still present but I
suspect they are meaning I need to forward port our hacks.

Whats the dream solution? We can set an environmental variable say
LIBTOOL_SYSROOT=/somewhere/sysroot and libtool would take this into
account exactly where needed and "just work". The .la files in our
sysroot would match those we install onto the target system.

Its possible things have changed and libtool has some mechanism to cope
with setups like this and if so please let me know what they are! If not
does the above illustrate the problem and is this something you'd be
prepared to help fix?

In the meantime I will try and get Poky/OE using a much more recent
libtool, patched enough so builds continue to work as described above.
Once we get a modern version working it will be much easier to test
patches and improvements in libtool and from what I've read, we should
see some performance improvements which would be most welcome. We don't
have patches to add sysroot support, just the sed hacks I've mentioned
but if you'd be interested in proper sysroot support that is something
we'd be interested in collaborating on.

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool


Re: Announcing Dolt, a drop-in Libtool replacement which cuts build times in half

2008-04-13 Thread Richard Purdie
Josh Triplett  freedesktop.org> writes:
> Thus, I wrote Dolt, a drop-in replacement for libtool's compilation
> mode.  Dolt runs any necessary system-specific or
> configuration-specific logic as part of configure, writes out a simple
> shell script "doltcompile"[1], and substitutes it for libtool in the
> automake variables LTCOMPILE and LTCXXCOMPILE.  If you use automake,
> autoconf, and libtool, then using Dolt just requires two steps:
> 
> 1) add "DOLT" after the call to LT_INIT, AC_PATH_LIBTOOL, or
>AM_PATH_LIBTOOL in your configure.ac or configure.in script, and
> 2) append dolt.m4 to your project's acinclude.m4.
> 
> For any system Dolt does not support, it will transparently fall back
> to libtool.

I work with Poky (pokylinux.org) and OpenEmbedded (openembedded.org) which are
cross compiling build systems and speed improvements like this are extremely
useful to us. Those systems also provide an interesting testing ground
where we can expose changes like this to a wide variety of source code.

I found this last part doesn't hold true, dolt does not fall back to
libtool transparently, the reason being the AC_SUBST causes LTCOMPILE to
become "", even if that section of the if block isn't called. I couldn't
find a nice way to fix this since AC_SUBST operates at reautoconf time,
not at configure and you only know if the system is compatible at
configure time.

Poky/OE use an old libtool (1.5.10) since we have various hacks we had
to make to get libtool to support cross compiling sanely[1] and
everytime we've tried to upgrade, something goes wrong and we've never
had time to debug the newer versions.

[1] Are there any plans to support sysroots with libtool?

In the interests of experimentation I hacked dolt into our libtool
recipe, inserting it into libtool.m4 in AC_PROG_LIBTOOL so I didn't have
to change any apps. I enabled it for arm linux targets and removed the
broken fallback code. The test build I used as a benchmark builds a
cross compiling toolchain and then from that a complete PDA style system
including X and some GTK+ apps with all the parts in between, many of
which use libtool.

Before the dolt change this image took 108 minutes, afterwards it took
96 minutes so an 11% reduction in time which is certainly beneficial!

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool