[PATCH 2/3] ltmain.in: Parse additional clang options

[Richard Purdie]

clang uses -rtlib and --unwindlib to select proper compiler runtime in
some cases. Therefore pass these options to linker when found in
ldflags

* build-aux/ltmain.in: Handle clang linker options
---
 build-aux/ltmain.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 5b59d959..8a832a47 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -5413,12 +5413,14 @@ func_mode_link ()
   # -shared-libsan   Link with shared sanitizer runtimes (Clang)
   # -static-libsan   Link with static sanitizer runtimes (Clang)
   # -fuse-ld=*   Linker select flags for GCC
+  # -rtlib=* select c runtime lib with clang
+  # --unwindlib=*select unwinder library with clang
   # -f{file|debug|macro|profile}-prefix-map* needed for lto linking
   # -Wa,*Pass flags directly to the assembler
   # -Werror, -Werror=*   Report (specified) warnings as errors
   -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
   
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
-  
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \
+  
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*|-rtlib=*|--unwindlib=*|
 \
   -specs=*|-fsanitize=*|-fno-sanitize*|-shared-libsan|-static-libsan| \
   
-ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*|-fprofile-prefix-map=*|
 \
   -fdiagnostics-color*|-frecord-gcc-switches| \
-- 
2.39.2

[PATCH 1/3] ltmain.in: Handle prefix-map compiler options correctly

[Richard Purdie]

If lto is enabled, we need the prefix-map variables to be passed to the linker
to correctly link the objects using correctly mapped paths.

Add these to the list of options libtool passes through.

* build-aux/ltmain.in: Handle prefix-map compiler options
---
 build-aux/ltmain.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 830b3982..5b59d959 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -5413,12 +5413,14 @@ func_mode_link ()
   # -shared-libsan   Link with shared sanitizer runtimes (Clang)
   # -static-libsan   Link with static sanitizer runtimes (Clang)
   # -fuse-ld=*   Linker select flags for GCC
+  # -f{file|debug|macro|profile}-prefix-map* needed for lto linking
   # -Wa,*Pass flags directly to the assembler
   # -Werror, -Werror=*   Report (specified) warnings as errors
   -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
   
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
   
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \
   -specs=*|-fsanitize=*|-fno-sanitize*|-shared-libsan|-static-libsan| \
+  
-ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*|-fprofile-prefix-map=*|
 \
   -fdiagnostics-color*|-frecord-gcc-switches| \
   -fuse-ld=*|-Wa,*|-Werror|-Werror=*)
 func_quote_arg pretty "$arg"
-- 
2.39.2

[PATCH 3/3] libtool.m4: Cleanup sysroot trailing "/"

[Richard Purdie]

If $CC has --sysroot=/, it is a valid configuration however libtool will
then set lt_sysroot to "/".

This means references like $lt_sysroot$libdir become //usr/lib instead
of the more normally expected /usr/lib. This may or may not break something
but certainly is confusing to the user and gives confusing output. Making
"/" simply unset lt_sysroot is much cleaner.

Whilst here, trim any trailing '/' from sysroot paths to drop the duplication
and result in cleaner/consistent output.

* m4/libtool.m4: Cleanup sysroot trailing '/' handling
---
 m4/libtool.m4 | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index ed572bb3..f2532f0e 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -1255,7 +1255,9 @@ lt_sysroot=
 case $with_sysroot in #(
  yes)
if test yes = "$GCC"; then
- lt_sysroot=`$CC --print-sysroot 2>/dev/null`
+ # Trim trailing / since we'll always append absolute paths and we want
+ # to avoid //, if only for less confusing output for the user.
+ lt_sysroot=`$CC --print-sysroot 2>/dev/null | $SED 's:/\+$::'`
fi
;; #(
  /*)
-- 
2.39.2

Re: [PATCH 1/3] ltmain.in: Handle prefix-map compiler options correctly

[Richard Purdie]

On Tue, 2024-01-16 at 20:47 -0500, Mike Frysinger wrote:
> On 16 Jan 2024 15:14, Richard Purdie wrote:
> > --- a/build-aux/ltmain.in
> > +++ b/build-aux/ltmain.in
> > @@ -7,7 +7,6 @@
> >  # Copyright (C) 1996-2019, 2021-2024 Free Software Foundation, Inc.
> >  # This is free software; see the source for copying conditions.  There is 
> > NO
> >  # warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR 
> > PURPOSE.
> > -
> >  # GNU Libtool is free software; you can redistribute it and/or modify
> >  # it under the terms of the GNU General Public License as published by
> >  # the Free Software Foundation; either version 2 of the License, or
> 
> let's leave unrelated changes out please

Sorry.

> 
> > @@ -5413,12 +5412,14 @@ func_mode_link ()
> ># -shared-libsan   Link with shared sanitizer runtimes (Clang)
> ># -static-libsan   Link with static sanitizer runtimes (Clang)
> ># -fuse-ld=*   Linker select flags for GCC
> > +  # -f{file|debug|macro}-prefix-map* needed for lto linking
> 
> these take the form of -ffile-prefix-map=xxx right ?

Yes. There was talk of a 'read from file' version but that doesn't look
like it was merged anywhere.

> so we'd want to put a = before the glob at the end ?

Sure. I'll also add profile-prefix-map for completeness since it looks
like the naming debate was settled on that one and it was merged (I
personally don't need it).

Cheers,

Richard

Re: [PATCH 06/12] libtool.m4: Handle "/" as a sysroot correctly

[Richard Purdie]

On Mon, 2024-01-15 at 17:00 -0500, Mike Frysinger wrote:
> On 25 Oct 2021 15:33, Richard Purdie wrote:
> > Update libtool.m4 to resolve a problem with lt_sysroot not being properly
> > updated if the option '--with[-libtool]-sysroot' is not provided when
> > running the 'configure' script for a package so that "/" as a sysroot
> > is handled correctly by libtool.
> 
> this is a long sentence, and there's no concrete examples, so i'm having a
> bit of trouble extrapolating the bug.  can you tweak this a bit to make it
> more clear ?

I've resent with part of that patch and a clearer explanation of what
the issue is. I need to look into what the other bit of the patch is
doing, it looks a bit like two different changes became intertwined.

Cheers,

Richard

[PATCH] libtool.m4: Handle "/" as a sysroot more correctly

[Richard Purdie]

If $CC has --sysroot=/, it is a valid configuration however libtool will
then set lt_sysroot to "/".

This means references like $lt_sysroot$libdir become //usr/lib instead
of the more normally expected /usr/lib. This may or may not break something
but certainly is confusing to the user and gives confusing output. Making
"/" simply unset lt_sysroot is much cleaner.

* m4/libtool.m4: Improve '/' sysroot handling
---
 m4/libtool.m4 | 4 
 1 file changed, 4 insertions(+)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index 695ccac4..1c8ea182 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -1256,6 +1256,10 @@ case $with_sysroot in #(
  yes)
if test yes = "$GCC"; then
  lt_sysroot=`$CC --print-sysroot 2>/dev/null`
+ # Treat "/" the same a an unset sysroot to avoid ugly // expansion for 
$lt_sysroot$libdir
+ if test "$lt_sysroot" = /; then
+   lt_sysroot=
+ fi
fi
;; #(
  /*)
-- 
2.39.2

Re: [PATCH 05/12] ltmain.in: Don't encode RATHS which match default linker paths

[Richard Purdie]

On Mon, 2024-01-15 at 20:10 -0500, Mike Frysinger wrote:
> On 25 Oct 2021 15:33, Richard Purdie wrote:
> > We don't want to add RPATHS which match default linker search paths, they're
> > a waste of space. This patch filters libtools list of paths to encoode and
> > removes the ones we don't need.
> 
> how are you defining "linker" ?  are you talking about the runtime linker
> (i.e. ld.so) or the build time linker (i.e. `ld`) ?

I meant the runtime linker/loader ld.so.

> the commit message also doesn't really explain in concrete details the
> scenario you're running into.

We were seeing binaries with RPATHS like /usr/lib in them which
basically doesn't do anything useful since it is a default for ld.so.
We were therefore trying to remove those to improve the efficiency of
the binaries slightly.

Cheers,

Richard

[PATCH 1/3] ltmain.in: Handle prefix-map compiler options correctly

[Richard Purdie]

If lto is enabled, we need the prefix-map variables to be passed to the linker
to correctly link the objects using correctly mapped paths.

Add these to the list of options libtool passes through.

* build-aux/ltmain.in: Handle prefix-map compiler options
---
 build-aux/ltmain.in | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index d5157a8d..34784c8c 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -7,7 +7,6 @@
 # Copyright (C) 1996-2019, 2021-2024 Free Software Foundation, Inc.
 # This is free software; see the source for copying conditions.  There is NO
 # warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-
 # GNU Libtool is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation; either version 2 of the License, or
@@ -5413,12 +5412,14 @@ func_mode_link ()
   # -shared-libsan   Link with shared sanitizer runtimes (Clang)
   # -static-libsan   Link with static sanitizer runtimes (Clang)
   # -fuse-ld=*   Linker select flags for GCC
+  # -f{file|debug|macro}-prefix-map* needed for lto linking
   # -Wa,*Pass flags directly to the assembler
   # -Werror, -Werror=*   Report (specified) warnings as errors
   -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
   
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
   
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \
   -specs=*|-fsanitize=*|-fno-sanitize*|-shared-libsan|-static-libsan| \
+  -ffile-prefix-map*|-fdebug-prefix-map*|-fmacro-prefix-map*| \
   -fdiagnostics-color*|-frecord-gcc-switches| \
   -fuse-ld=*|-Wa,*|-Werror|-Werror=*)
 func_quote_arg pretty "$arg"
-- 
2.39.2

[PATCH 2/3] ltmain.in: Parse additional clang options

[Richard Purdie]

clang uses -rtlib and --unwindlib to select proper compiler runtime in
some cases. Therefore pass these options to linker when found in
ldflags

* build-aux/ltmain.in: Handle clang linker options
---
 build-aux/ltmain.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 34784c8c..cf6dc13a 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -5412,12 +5412,14 @@ func_mode_link ()
   # -shared-libsan   Link with shared sanitizer runtimes (Clang)
   # -static-libsan   Link with static sanitizer runtimes (Clang)
   # -fuse-ld=*   Linker select flags for GCC
+  # -rtlib=* select c runtime lib with clang
+  # --unwindlib=*select unwinder library with clang
   # -f{file|debug|macro}-prefix-map* needed for lto linking
   # -Wa,*Pass flags directly to the assembler
   # -Werror, -Werror=*   Report (specified) warnings as errors
   -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
   
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
-  
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \
+  
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*|-rtlib=*|--unwindlib=*|
 \
   -specs=*|-fsanitize=*|-fno-sanitize*|-shared-libsan|-static-libsan| \
   -ffile-prefix-map*|-fdebug-prefix-map*|-fmacro-prefix-map*| \
   -fdiagnostics-color*|-frecord-gcc-switches| \
-- 
2.39.2

[PATCH 3/3] libtool: Fix support for NIOS2 processor

[Richard Purdie]

The name of the system contains the string "nios2". This string
is caught by the some of the greedy checks for OS/2 in libtool,
in particular the *os2* branches of switch statements match for
the nios2 string, which results in incorrect behavior of libtool.

Switch to use $host_os instead of $host and tweak the patterns to
match to avoid this problem for nios2.

* build-aux/ltmain.in: Fix NIOS2 support
---
 build-aux/ltmain.in | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index cf6dc13a..ac11b7ae 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -517,10 +517,10 @@ libtool_validate_options ()
 # preserve --debug
 test : = "$debug_cmd" || func_append preserve_args " --debug"
 
-case $host in
+case $host_os in
   # Solaris2 added to fix 
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16452
   # see also: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59788
-  *cygwin* | *mingw* | *windows* | *pw32* | *cegcc* | *solaris2* | *os2*)
+  cygwin* | mingw* | windows* | pw32* | cegcc* | solaris2* | os2*)
 # don't eliminate duplications in $postdeps and $predeps
 opt_duplicate_compiler_generated_deps=:
 ;;
@@ -6279,8 +6279,8 @@ func_mode_link ()
fi
if test -n "$library_names" &&
   { test no = "$use_static_libs" || test -z "$old_library"; }; then
- case $host in
- *cygwin* | *mingw* | *windows* | *cegcc* | *os2*)
+ case $host_os in
+ cygwin* | mingw* | windows* | cegcc* | os2*)
  # No point in relinking DLLs because paths are not encoded
  func_append notinst_deplibs " $lib"
  need_relink=no
@@ -6349,8 +6349,8 @@ func_mode_link ()
  soname=$dlname
elif test -n "$soname_spec"; then
  # bleh windows
- case $host in
- *cygwin* | mingw* | *windows* | *cegcc* | *os2*)
+ case $host_os in
+ cygwin* | mingw* | windows* | cegcc* | os2*)
func_arith $current - $age
major=$func_arith_result
versuffix=-$major
-- 
2.39.2

Re: [PATCH 02/12] libtool.m4: Rename the --with-sysroot option to avoid conflict with gcc/binutils

[Richard Purdie]

Somehow my original reply ended up blank, sorry. I've retyped it from
memory as best I can.

On Mon, 2024-01-15 at 20:08 -0500, Mike Frysinger wrote:
> On 25 Oct 2021 15:33, Richard Purdie wrote:
> > This patch renames the --with-sysroot option to --with-libtool-sysroot
> > to avoid namespace conflict with binutils, gcc and other toolchain
> > components since these componets also add that option to configure
> > and this becomes confusing and conflicting otherwise.
> 
> we're talking like 3 projects ?  libtool is used & included in way more
> projects than that.  it seems like --with-sysroot as a standard name to
> propagate out into the wider ecosystem makes more sense.

You're saying binutils and gcc should change? I'm fairly sure they'll
decline to do that. Their options are encoded into quite a few cross
compiling documents and build systems.

> i agree there's confusion between "use this path as a sysroot to compile
> and link against" and "have the compile tools target this sysroot".  the
> same problem exists with host selection which is why we have the build,
> host, and target options.  sounds like we want names that can align with
> these somehow.

FWIW gcc and binutils have gone with --with-sysroot and --with-build-
sysroot to differentiate. Unfortunately that doesn't really help
libtool (see below).

> --with-libtool-sysroot doesn't make sense as it isn't an internal libtool
> setting, it's changing how executables are linked, and users (who run the
> configure scripts) don't know or care about this "libtool" thing.

I think it partly depends upon how you view it. Most of the time
libtool uses `$CC --print-sysroot` to set this and therefore most users
wouldn't use the commandline option. In general you do want the
compiler/linker and libtool to be working the same way.

The challenge is that binutils/gcc and libtool have adopted different
meanings for the same option and if/as/where the commandline options
get passed around, things can become quite broken when cross compiling.

My ask is to try and find a way to avoid the confusion (and the need
for Yocto Project to have to patch one set of components to make things
work).

The options as I see it are:

a) Use a libtool specific option (as per the patch under review)

b) Stop providing a commandline option for libtool and rely on the
compiler query.

c) Switch to --with-build-sysroot

d) Switch to --sysroot

e) Ask gcc/binutils to change


I can't see gcc/binutils changing. Using c/d would likely have a
different set of issues. For example in some cases you do want the
build sysroot and sometimes the target with libtool so c swaps one set
of problems for another. 

Since most of the time users never need to care about setting this
option and we couldn't find an alternative, changing it as per the
patch seemed like the least worst solution and  was why we end up with
a).

Would:

--with-sysroot-libtool

be any better?

I'm open to other ideas.

Cheers,

Richard

Re: [PATCH 02/12] libtool.m4: Rename the --with-sysroot option to avoid conflict with gcc/binutils

[Richard Purdie]

Re: [PATCH v2 00/12] Rebased version of Yocto patches

[Richard Purdie]

On Sat, 2022-11-19 at 18:32 +, Sam James wrote:
> 
> > On 16 Apr 2022, at 18:58, Sam James  wrote:
> > 
> > Done after discussion w/ Alex (thanks!). Rebased on master after
> > 2.4.7 release.
> 
> Ping. Could we maybe get the easy ones in and then revisit
> the bits which received comments (unclear to me what we need
> to do for those)?

I'd second that, I'd love to get the easy ones sorted and am happy to
try and help with the others, just not quite sure where we stand at the
moment.

Cheers,

Richard

Re: [PATCH v2 01/12] ltmain.sh: Fix sysroot paths being encoded into RPATHs

[Richard Purdie]

On Wed, 2022-04-20 at 01:12 +0100, Sam James wrote:
> 
> > On 17 Apr 2022, at 05:55, Alex Ameen  wrote:
> > 
> > This was all green down the line on the test suite on multiple systems (
> > don't get too excited yet ) until I found bugs in the testsuite...
> > 
> > I see how this flew under the radar previously though - currently there are
> > no tests which attempt to check RPATH or RUNPATH entries. I'll definitely be
> > working on that... I'm going to be working out some M4 macros to abstract
> > some functions like `lt_read_pheader([BIN], [ENTRY])',
> > `lt_read_rpath([BIN])', and `lt_read_runpath([BIN])', so that those can be
> > abstracted for handling non-ELF binaries.
> > 
> > I'll make a test case to the effect of `readelf -d -W BIN|grep -v
> > "$sysroot/";', if you have any additional input on new test cases let me
> > know.
> > 
> > You also helped me catch some bad regex in the existing sysroot tests that
> > would cause them to never be run on a system which used '/' as their GCC
> > sysroot ( all of dpkg's  cross compilers seem to... ).
> 
> Nice! I've found a *lot* of things don't respect this case, actually.
> 
> > 
> > So a big thank you for helping to catch all of these places that the tool
> > can be improved.
> > 
> > Naturally now that test cases aren't skipped they're red, so once I sanity
> > check that they fail on the mainline branch I can move forward. I'm ~99%
> > sure this patch will have no effect on those results.
> > 
> > 
> 
> FWIW, given the comments on the main libtool ML, I at least am happy to drop
> this one for now, and revisit later. Richard might feel differently though.

I think this one is ok, it would be "ltmain.in: Don't encode RATHS which match
default linker paths" which is being questioned. We see a awful lot of it and I
don't think we need them but I can see the concern others are raising even if it
what I think should be a more unusual use case. It may be something that could
be configured ultimately?

> I like incremental progress so the more easy stuff in, the better, even if it
> means we have to come back to some of the harder ones.

Definitely agreed. I'd rather we make progress on the things we can agree on
that block on that.

Cheers,

Richard

Re: rpath stripping

[Richard Purdie]

On Mon, 2022-04-18 at 07:39 -0400, Carlos O'Donell wrote:
> On 4/17/22 10:06, Bob Friesenhahn wrote:
> > The libtool I was using (originating from Ubuntu Linux) stripped the
> > rpath (which was provided like '-Wl,rpath=/usr/lib') so I was unable
> > to embed an rpath in the libcurl I built so that applications linked
> > with that libcurl would find it.
> 
> I agree with our position.
> 
> The behaviour of stripping '-Wl,-rpath' is incorrect.
> 
> With new DT_RUNPATH semantics (DT_RPATH being deprecated and binutils having
> switched defaults), each shared object, including the binary, must correctly
> specify the search path for the immediate needed objects. Stripping this off
> will result in incorrectly built shared objects and binaries that don't
> operate correctly.
> 
> I'm curious what justification is given for this behaviour.

As I understand it the dynamic loader has a set of search paths it falls back to
(sys_lib_dlsearch_path in libtool). An RPATH or RUNPATH entry matching a system
loader path isn't usually of much use, it just takes up space.

I can imagine some cases where that might not be true such as linking with "-z
nodeflib" or some fairly convoluted setups but I suspect those would have other
issues too.

Cheers,

Richard

Submission process for libtool?

[Richard Purdie]

Hi,

It was great to see a libtool release, thanks for that!

I upgraded Yocto Project to it in time for our LTS release:

https://git.yoctoproject.org/poky/commit/?id=ff7b41573842a403c81f58bee41fc8163a9d7754

so far things seem reasonable, we've had a few minor issues but they're not
really libtool's fault or concern. One interesting quirk was that the shell
script optimisation changes made between 2.4.6 and 2.4.7 resulted in very long
(6,000+ character) pathnames being passed to the C library functions. This upset
our fakeroot emulation but we've fixed that to workaround the issue.

Yocto Project is carrying a few patches. I did clean them up and shared many of
them in October:

https://lists.gnu.org/archive/html/libtool-patches/2021-10/msg00012.html

Some are more important than others and there what I believe are good bug fixes
in there. My questions:

a) Is there a possibility these could be considered for merging?

b) Should I rebase and repost them? I'm happy to do it if it would help.

c) Am I using the right process for patch submission? I never did get a reply
despite asking publicly and privately. If I'm doing something wrong process wise
to submit them, I'd happily correct it.

Yocto Project is interesting as we can quickly test changes against the software
of a whole Linux stack across many architectures "quite quickly" (complete build
and tests in a few hours). I'm going to try and more closely track libtool
development so that we hopefully identify regressions from our perpective
quickly.

Thanks,

Richard

Re: new release?

[Richard Purdie]

On Sat, 2022-02-05 at 21:06 -0500, Mike Frysinger wrote:
> On 05 Feb 2022 15:15, Alex Ameen wrote:
> > This is a good question. I plan on making a new release this month.
> > 
> > When I first adopted the project I ambitiously thought I'd manage to 
> > create a new release after about a month; but the truth is when I 
> > started doing a deep dive into the internals there was a lot of history 
> > and complexity for me to unpack. Things that are easy to overlook like 
> > how change-logs get generated, quirks in the testing framework, and 
> > tracing down disparate areas to align documentation took quite a while 
> > to navigate.
> > 
> > The good news is that I think I've got the confidence to push a release 
> > soon. One area that I was reading up on this weekend was whether the 
> > "alpha"/private releases of `libtool' might be appropriate, or whether I 
> > should just push a release immediately. I'll admit I am leaning towards 
> > just making a release to avoid the entire alpha process for the time being.
> 
> i wouldn't sweat it too much.  the next release of libtool will be 2.6, and
> you can note its state in the announcement/NEWS.  distros will give it a run
> to find regressions, and as fixes are merged, just do 2.6.1, 2.6.2, etc...
> 

I'd like to second that. Getting a release out would be great even if it isn't
perfect, then go from there.

I know there are some Yocto Project patches for issues we've collected from
across the embedded ecosystem over the last few years that I rebased and posted
in the hope they could be merged. I'd rather we got to those in due course and
had a release though! :)

Cheers,

Richard

Re: New libtool maintainer

[Richard Purdie]

On Sun, 2021-11-21 at 13:14 -0600, Alex Ameen wrote:
> I just took a look at those. Good catches on the typos, I probably would 
> not have noticed them just reading the script myself. Same thing with 
> the M4 `[]' quoting issue ( classic pitfall ). I'll get these merged ASAP.
> 
> For the non-Linux patches I can merge them, but I don't have personally 
> have OSX, powerpc, or Solaris boxes, and while I do have a Windows 
> partition I don't currently have it set up to run these kinds of tests. 
> Nonetheless I can merge these - if you have access to any of those 
> platforms let me know if you would be open to running `make check' and 
> posting the logs so I can sanity check the new behavior.
> 
> Thank you so much for bringing these to my attention. There's a long 
> list of old patches and mailing list archives and as a practical matter 
> it's hard to know which of them are still relevant - so I appreciate 
> your help.

I did recently post the better bits of the OpenEmbedded/Yocto Project patchset
for libtool:

https://lists.gnu.org/archive/html/libtool-patches/2021-10/msg00012.html

Those are at up to date and in regular use in OE/YP which is widely used for
cross compiling for Linux/mingw and others. We tend to find sysroot and cross
compile issues in particular and we also find reproducibility and parallel make
race issues.

If you have any questions or concerns on any of those I'm happy to try and help.

Cheers,

Richard

[PATCH 01/12] ltmain.in: Handle trailing slashes on install commands correctly

[Richard Purdie]

A command like:

libtool --mode=install /usr/bin/install -c gck-roots-store-standalone.la 
'/image/usr/lib/gnome-keyring/standalone/'

where the path ends with a trailing slash currently fails. This occurs in
software like gnome-keyring or pulseaudio and is because the comparision
code doesn't see the paths as equal. Strip both paths to ensure this works
reliably.

Signed-off-by: Richard Purdie 
---
 build-aux/ltmain.in | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 96b37003..3d5dcd0a 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -2378,8 +2378,14 @@ func_mode_install ()
func_append dir "$objdir"
 
if test -n "$relink_command"; then
+ # Strip any trailing slash from the destination.
+ func_stripname '' '/' "$libdir"
+ destlibdir=$func_stripname_result
+ func_stripname '' '/' "$destdir"
+ s_destdir=$func_stripname_result
+
  # Determine the prefix the user has applied to our future dir.
- inst_prefix_dir=`$ECHO "$destdir" | $SED -e "s%$libdir\$%%"`
+ inst_prefix_dir=`$ECHO "X$s_destdir" | $Xsed -e "s%$destlibdir\$%%"`
 
  # Don't allow the user to place us outside of our expected
  # location b/c this prevents finding dependent libraries that
-- 
2.25.1

[PATCH 04/12] ltmain.sh: Fix sysroot paths being encoded into RPATHs

[Richard Purdie]

There is a bug where RPATHs could end up containing sysroot values when
cross compiling which is obviously incorrect. Strip out sysroot components
from libdir when building RPATH values to avoid this.

Signed-off-by: Richard Purdie 
---
 build-aux/ltmain.in | 14 --
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 2c994612..96238350 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -7654,9 +7654,11 @@ EOF
  test relink = "$opt_mode" || rpath=$compile_rpath$rpath
  for libdir in $rpath; do
if test -n "$hardcode_libdir_flag_spec"; then
+ func_replace_sysroot "$libdir"
+ libdir=$func_replace_sysroot_result
+ func_stripname '=' '' "$libdir"
+ libdir=$func_stripname_result
  if test -n "$hardcode_libdir_separator"; then
-   func_replace_sysroot "$libdir"
-   libdir=$func_replace_sysroot_result
if test -z "$hardcode_libdirs"; then
  hardcode_libdirs=$libdir
else
@@ -8386,6 +8388,10 @@ EOF
   hardcode_libdirs=
   for libdir in $compile_rpath $finalize_rpath; do
if test -n "$hardcode_libdir_flag_spec"; then
+ func_replace_sysroot "$libdir"
+ libdir=$func_replace_sysroot_result
+ func_stripname '=' '' "$libdir"
+ libdir=$func_stripname_result
  if test -n "$hardcode_libdir_separator"; then
if test -z "$hardcode_libdirs"; then
  hardcode_libdirs=$libdir
@@ -8437,6 +8443,10 @@ EOF
   hardcode_libdirs=
   for libdir in $finalize_rpath; do
if test -n "$hardcode_libdir_flag_spec"; then
+ func_replace_sysroot "$libdir"
+ libdir=$func_replace_sysroot_result
+ func_stripname '=' '' "$libdir"
+ libdir=$func_stripname_result
  if test -n "$hardcode_libdir_separator"; then
if test -z "$hardcode_libdirs"; then
  hardcode_libdirs=$libdir
-- 
2.25.1

[PATCH 09/12] Makefile.am: make sure autoheader run before autoconf

[Richard Purdie]

From: Mingli Yu 

autoheader will update ../libtool-2.4.6/libltdl/config-h.in which
autoconf needs, so there comes a race sometimes as below:
 | configure.ac:45: error: required file 'config-h.in' not found
 | touch '../libtool-2.4.6/libltdl/config-h.in'

So make sure autoheader run before autoconf to avoid this race.

Signed-off-by: Mingli Yu 
Signed-off-by: Richard Purdie 
---
 Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile.am b/Makefile.am
index 6b546092..84795d87 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -370,7 +370,7 @@ lt_configure_deps = $(lt_aclocal_m4) $(lt_aclocal_m4_deps)
 $(lt_aclocal_m4): $(lt_aclocal_m4_deps)
$(AM_V_GEN)cd '$(srcdir)/$(ltdl_dir)' && $(ACLOCAL) -I ../m4
 
-$(lt_configure): $(lt_configure_deps)
+$(lt_configure): $(lt_configure_deps) $(lt_config_h_in)
$(AM_V_GEN)cd '$(srcdir)/$(ltdl_dir)' && $(AUTOCONF)
 
 $(lt_config_h_in): $(lt_configure_deps)
-- 
2.25.1

[PATCH 06/12] libtool.m4: Handle "/" as a sysroot correctly

[Richard Purdie]

Update libtool.m4 to resolve a problem with lt_sysroot not being properly
updated if the option '--with[-libtool]-sysroot' is not provided when
running the 'configure' script for a package so that "/" as a sysroot
is handled correctly by libtool.

Signed-off-by: Richard Purdie 
---
 m4/libtool.m4 | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index de2f1ebf..180dd9d1 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -1256,16 +1256,20 @@ dnl lt_sysroot will always be passed unquoted.  We 
quote it here
 dnl in case the user passed a directory name.
 lt_sysroot=
 case $with_libtool_sysroot in #(
- yes)
+ no)
if test yes = "$GCC"; then
  lt_sysroot=`$CC --print-sysroot 2>/dev/null`
+ # Treat "/" the same a an unset sysroot.
+ if test "$lt_sysroot" = /; then
+   lt_sysroot=
+ fi
fi
;; #(
+ yes|''|/)
+   ;; #(
  /*)
lt_sysroot=`echo "$with_libtool_sysroot" | sed -e "$sed_quote_subst"`
;; #(
- no|'')
-   ;; #(
  *)
AC_MSG_RESULT([$with_libtool_sysroot])
AC_MSG_ERROR([The sysroot must be an absolute path.])
-- 
2.25.1

[PATCH 05/12] ltmain.in: Don't encode RATHS which match default linker paths

[Richard Purdie]

We don't want to add RPATHS which match default linker search paths, they're
a waste of space. This patch filters libtools list of paths to encoode and
removes the ones we don't need.

Libtool may be passed link paths of the form "/usr/lib/../lib" so normalize
the paths before comparision.

Signed-off-by: Richard Purdie 
---
 build-aux/ltmain.in | 34 --
 1 file changed, 28 insertions(+), 6 deletions(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 96238350..6fb58ed2 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -7672,8 +7672,16 @@ EOF
  esac
fi
  else
-   eval flag=\"$hardcode_libdir_flag_spec\"
-   func_append dep_rpath " $flag"
+# We only want to hardcode in an rpath if it isn't in the
+# default dlsearch path.
+func_normal_abspath "$libdir"
+libdir_norm=$func_normal_abspath_result
+   case " $sys_lib_dlsearch_path " in
+   *" $libdir_norm "*) ;;
+   *) eval flag=\"$hardcode_libdir_flag_spec\"
+   func_append dep_rpath " $flag"
+   ;;
+   esac
  fi
elif test -n "$runpath_var"; then
  case "$perm_rpath " in
@@ -8406,8 +8414,16 @@ EOF
  esac
fi
  else
-   eval flag=\"$hardcode_libdir_flag_spec\"
-   func_append rpath " $flag"
+# We only want to hardcode in an rpath if it isn't in the
+# default dlsearch path.
+func_normal_abspath "$libdir"
+libdir_norm=$func_normal_abspath_result
+   case " $sys_lib_dlsearch_path " in
+   *" $libdir_norm "*) ;;
+   *) eval flag=\"$hardcode_libdir_flag_spec\"
+   rpath+=" $flag"
+   ;;
+   esac
  fi
elif test -n "$runpath_var"; then
  case "$perm_rpath " in
@@ -8461,8 +8477,14 @@ EOF
  esac
fi
  else
-   eval flag=\"$hardcode_libdir_flag_spec\"
-   func_append rpath " $flag"
+# We only want to hardcode in an rpath if it isn't in the
+# default dlsearch path.
+   case " $sys_lib_dlsearch_path " in
+   *" $libdir "*) ;;
+   *) eval flag=\"$hardcode_libdir_flag_spec\"
+   func_append rpath " $flag"
+   ;;
+   esac
  fi
elif test -n "$runpath_var"; then
  case "$finalize_perm_rpath " in
-- 
2.25.1

[PATCH 03/12] ltmain.in: Add missing sysroot to library path

[Richard Purdie]

From: Khem Raj 

When using a sysroot we should append it to libdir, which is helpful in
cross builds as the system is staged in the sysroot. For normal builds,
i.e. when lt_sysroot is not set, it will still behave the same and add
-L/usr/lib to the relink command.

Signed-off-by: Richard Purdie 
---
 build-aux/ltmain.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 3d5dcd0a..2c994612 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -6475,7 +6475,7 @@ func_mode_link ()
  fi
else
  # We cannot seem to hardcode it, guess we'll fake it.
- add_dir=-L$libdir
+ add_dir="-L$lt_sysroot$libdir"
  # Try looking first in the location we're being installed to.
  if test -n "$inst_prefix_dir"; then
case $libdir in
-- 
2.25.1

[PATCH 10/12] Makefile.am: make sure autoheader run before automake

[Richard Purdie]

From: Mingli Yu 

When use automake to generate Makefile.in from Makefile.am, there
comes below race:
 | configure.ac:45: error: required file 'config-h.in' not found

It is because the file config-h.in in updating process by autoheader,
so make automake run after autoheader to avoid the above race.

Signed-off-by: Mingli Yu 
Signed-off-by: Richard Purdie 
---
 Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile.am b/Makefile.am
index 84795d87..8c9949ed 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -333,7 +333,7 @@ EXTRA_DIST += $(lt_aclocal_m4) \
  $(lt_obsolete_m4) \
  $(stamp_mk)
 
-$(lt_Makefile_in): $(lt_Makefile_am) $(lt_aclocal_m4)
+$(lt_Makefile_in): $(lt_Makefile_am) $(lt_aclocal_m4) $(lt_config_h_in)
$(AM_V_GEN)cd '$(srcdir)/$(ltdl_dir)' && $(AUTOMAKE) Makefile
 
 # Don't let unused scripts leak into the libltdl Makefile
-- 
2.25.1

[PATCH 11/12] ltmain.in: Handle prefix-map compiler options correctly

[Richard Purdie]

If lto is enabled, we need the prefix-map variables to be passed to the linker.
Add these to the list of options libtool passes through.

Signed-off-by: Richard Purdie 
---
 build-aux/ltmain.in | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 606f17be..7dc2e897 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -5392,10 +5392,11 @@ func_mode_link ()
   # -stdlib=*select c++ std lib with clang
   # -fsanitize=* Clang/GCC memory and address sanitizer
   # -fuse-ld=*   Linker select flags for GCC
+  # -f*-prefix-map*  needed for lto linking
   -64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
   
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|@*|-tp=*|--sysroot=*| \
   
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \
-  -specs=*|-fsanitize=*|-fuse-ld=*)
+  -specs=*|-fsanitize=*|-fuse-ld=*|-f*-prefix-map*)
 func_quote_arg pretty "$arg"
arg=$func_quote_arg_result
 func_append compile_command " $arg"
-- 
2.25.1

[PATCH 07/12] libtool: Fix support for NIOS2 processor

[Richard Purdie]

From: Marek Vasut 

The name of the system contains the string "nios2". This string
is caught by the some of the greedy checks for OS/2 in libtool,
in particular the *os2* branches of switch statements match for
the nios2 string, which results in incorrect behavior of libtool.

This patch adds an explicit check for *nios2* before the *os2*
checks to prevent the OS/2 check incorrectly trapping the nios2
as well.

Signed-off-by: Marek Vasut 
Signed-off-by: Richard Purdie 
---
 build-aux/ltmain.in | 20 
 1 file changed, 20 insertions(+)

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 6fb58ed2..606f17be 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -519,6 +519,12 @@ libtool_validate_options ()
 test : = "$debug_cmd" || func_append preserve_args " --debug"
 
 case $host in
+  # For NIOS2, we want to make sure that it's not caught by the
+  # more general OS/2 check below. Otherwise, NIOS2 is the same
+  # as the default option.
+  *nios2*)
+opt_duplicate_compiler_generated_deps=$opt_preserve_dup_deps
+;;
   # Solaris2 added to fix 
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16452
   # see also: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59788
   *cygwin* | *mingw* | *pw32* | *cegcc* | *solaris2* | *os2*)
@@ -6246,6 +6252,15 @@ func_mode_link ()
if test -n "$library_names" &&
   { test no = "$use_static_libs" || test -z "$old_library"; }; then
  case $host in
+ *nios2*)
+   # For NIOS2, we want to make sure that it's not caught by the
+   # more general OS/2 check below. Otherwise, NIOS2 is the same
+   # as the default option.
+   if test no = "$installed"; then
+ func_append notinst_deplibs " $lib"
+ need_relink=yes
+   fi
+   ;;
  *cygwin* | *mingw* | *cegcc* | *os2*)
  # No point in relinking DLLs because paths are not encoded
  func_append notinst_deplibs " $lib"
@@ -6316,6 +6331,11 @@ func_mode_link ()
elif test -n "$soname_spec"; then
  # bleh windows
  case $host in
+ *nios2*)
+   # For NIOS2, we want to make sure that it's not caught by the
+   # more general OS/2 check below. Otherwise, NIOS2 is the same
+   # as the default option.
+   ;;
  *cygwin* | mingw* | *cegcc* | *os2*)
func_arith $current - $age
major=$func_arith_result
-- 
2.25.1

[PATCH 08/12] libtool: Check for static libs for internal compiler libraries

[Richard Purdie]

From: Khem Raj 

Libtool checks only for libraries linked as -l* when trying to
find internal compiler libraries. Clang, however uses the absolute
path to link its internal libraries e.g. compiler_rt. This patch
handles clang's statically linked libraries when finding internal
compiler libraries.

Signed-off-by: Khem Raj 
Signed-off-by: Richard Purdie 
---
 m4/libtool.m4 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index 180dd9d1..022c1292 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -7560,7 +7560,7 @@ if AC_TRY_EVAL(ac_compile); then
   for p in `eval "$output_verbose_link_cmd"`; do
 case $prev$p in
 
--L* | -R* | -l*)
+-L* | -R* | -l* | */libclang_rt.*.a)
# Some compilers place space between "-{L,R}" and the path.
# Remove the space.
if test x-L = "$p" ||
-- 
2.25.1

[PATCH 12/12] libtool.m4: For reproducibility stop encoding hostname in libtool script

[Richard Purdie]

For reproducibilty, stop encoding the hostname into the libtool script, this 
isn't
really adding much to debugging and most distros are carrying such a patch now 
as
reproducibility is important.

Signed-off-by: Richard Purdie 
---
 m4/libtool.m4 | 1 -
 1 file changed, 1 deletion(-)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index 022c1292..1a8a2998 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -728,7 +728,6 @@ _LT_CONFIG_SAVE_COMMANDS([
 cat <<_LT_EOF >> "$cfgfile"
 #! $SHELL
 # Generated automatically by $as_me ($PACKAGE) $VERSION
-# Libtool was configured on host `(hostname || uname -n) 2>/dev/null | sed 1q`:
 # NOTE: Changes made to this file will be lost: look at ltmain.sh.
 
 # Provide generalized library-building support services.
-- 
2.25.1

[PATCH 02/12] libtool.m4: Rename the --with-sysroot option to avoid conflict with gcc/binutils

[Richard Purdie]

From: Khem Raj 

This patch renames the --with-sysroot option to --with-libtool-sysroot
to avoid namespace conflict with binutils, gcc and other toolchain
components since these componets also add that option to configure
and this becomes confusing and conflicting otherwise.

Signed-off-by: Richard Purdie 
---
 m4/libtool.m4| 12 ++--
 tests/sysroot.at |  6 +++---
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/m4/libtool.m4 b/m4/libtool.m4
index f2d1f398..de2f1ebf 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -1246,28 +1246,28 @@ _LT_DECL([], [ECHO], [1], [An echo program that 
protects backslashes])
 # 
 AC_DEFUN([_LT_WITH_SYSROOT],
 [AC_MSG_CHECKING([for sysroot])
-AC_ARG_WITH([sysroot],
-[AS_HELP_STRING([--with-sysroot@<:@=DIR@:>@],
+AC_ARG_WITH([libtool-sysroot],
+[AS_HELP_STRING([--with-libtool-sysroot@<:@=DIR@:>@],
   [Search for dependent libraries within DIR (or the compiler's sysroot
if not specified).])],
-[], [with_sysroot=no])
+[], [with_libtool_sysroot=no])
 
 dnl lt_sysroot will always be passed unquoted.  We quote it here
 dnl in case the user passed a directory name.
 lt_sysroot=
-case $with_sysroot in #(
+case $with_libtool_sysroot in #(
  yes)
if test yes = "$GCC"; then
  lt_sysroot=`$CC --print-sysroot 2>/dev/null`
fi
;; #(
  /*)
-   lt_sysroot=`echo "$with_sysroot" | sed -e "$sed_quote_subst"`
+   lt_sysroot=`echo "$with_libtool_sysroot" | sed -e "$sed_quote_subst"`
;; #(
  no|'')
;; #(
  *)
-   AC_MSG_RESULT([$with_sysroot])
+   AC_MSG_RESULT([$with_libtool_sysroot])
AC_MSG_ERROR([The sysroot must be an absolute path.])
;;
 esac
diff --git a/tests/sysroot.at b/tests/sysroot.at
index 05faa13d..7a1f9567 100644
--- a/tests/sysroot.at
+++ b/tests/sysroot.at
@@ -64,7 +64,7 @@ while read file; do
 done])
 
 LDFLAGS="$LDFLAGS --sysroot=$sysroot -no-undefined"
-configure_options="$configure_options --with-sysroot=$sysroot --prefix=$prefix"
+configure_options="$configure_options --with-libtool-sysroot=$sysroot 
--prefix=$prefix"
 
 #???
 if test PATH = "$shlibpath_var"; then
@@ -114,7 +114,7 @@ AM_INIT_AUTOMAKE([foreign])
 AC_PROG_CC
 AC_CONFIG_SRCDIR([lib2.c])
 LT_INIT
-sysroot=$with_sysroot
+sysroot=$with_libtool_sysroot
 AC_SUBST([sysroot])
 AC_OUTPUT(Makefile)
 ]])
@@ -155,7 +155,7 @@ AM_INIT_AUTOMAKE([foreign])
 AC_PROG_CC
 AC_CONFIG_SRCDIR([prog.c])
 LT_INIT
-sysroot=$with_sysroot
+sysroot=$with_libtool_sysroot
 AC_SUBST([sysroot])
 AC_OUTPUT(Makefile)
 ]])
-- 
2.25.1

[PATCH 00/12] Yocto Project libtool patch queue

[Richard Purdie]

Hi,

I saw comments about a possible pending release and decided to take the 
opportunity to send out the bulk of Yocto Project's patch queue
for libtool. We use it in a cross compiling environment and make
extensive use of the sysroot support in the toolchain so most of our
issues are in that area but there are also some reproducubility and
reduction of unnecessary rpaths tweaks here.

We've been using these patches for many years in some cases so it
would be create to reduce our patchset and share the fixes with others.

Some have been on the libtool mailing lists before but I thought it
simpler to send the set out for review together as a set.

Cheers,

Richard

Khem Raj (3):
  libtool.m4: Rename the --with-sysroot option to avoid conflict with
gcc/binutils
  ltmain.in: Add missing sysroot to library path
  libtool: Check for static libs for internal compiler libraries

Marek Vasut (1):
  libtool: Fix support for NIOS2 processor

Mingli Yu (2):
  Makefile.am: make sure autoheader run before autoconf
  Makefile.am: make sure autoheader run before automake

Richard Purdie (6):
  ltmain.in: Handle trailing slashes on install commands correctly
  ltmain.sh: Fix sysroot paths being encoded into RPATHs
  ltmain.in: Don't encode RATHS which match default linker paths
  libtool.m4: Handle "/" as a sysroot correctly
  ltmain.in: Handle prefix-map compiler options correctly
  libtool.m4: For reproducibility stop encoding hostname in libtool
script

 Makefile.am |  4 +--
 build-aux/ltmain.in | 81 +++--
 m4/libtool.m4   | 25 --
 tests/sysroot.at|  6 ++--
 4 files changed, 89 insertions(+), 27 deletions(-)

-- 
2.25.1

Re: transitive shared library dependencies and installation

[Richard Purdie]

On Thu, 2020-01-02 at 16:24 -0600, Bob Friesenhahn wrote:
> On Thu, 2 Jan 2020, wf...@niif.hu wrote:
> > > If Libtool were to depend on non-portable features, [...] then it
> > > could not longer be described as a portability tool.
> > 
> > In my understanding, Libtool is a portability shim, providing a
> > regular
> > interface for developers across systems with wildly varying
> > features.
> > It already uses non-portable features, just different ones on
> > different
> > architectures.  I don't say it should use -rpath-link generally, I
> > only
> > suggested that it might be an efficient solution on systems
> > supporting
> > it.  But I expect all systems supporting shared objects to allow
> > using
> > and installing them some way, irrespective of their
> > interdependencies.
> > Am I overly naive?
> 
> Certainly, libtool could use -rpath-link where it is supported but 
> libtool provides a common feature set and if it is only possible to 
> support a feature where -rpath-link is available, then offering the 
> feature would defeat the purpose of being a portability tool.
> 
> Sometimes it is better to force the using software to conform to the 
> limitations.
> 
> Libtool must also work for static linking.  It seems to me that your 
> issue also impacts static linking.

I think the challenge that libtool has here is that many of these older
systems that libtool supports aren't so prevalent anymore.

I work on the Yocto Project where we do a ton of cross compiling and
have to work with libtool but its mostly Linux with a small amount of
mingw/baremetal/darwin bits.

We have a number of libtool patches to sort cross compile issues which
really need discussion with upstream libtool. With the lack of
releases, our incentive to do that diminishes :(.

By reducing everything to the lowest common denominator, it means
libtool struggles to take advantage of any new linker technology too.

Finding new maintainers with the amount of knowledge of weird older
systems needed is going to be a struggle which only gets worse over
time :(

I do worry about the future here as libtool is a key part of the
autotools fabric but its most likely to be wholesale replaced given how
things are trending.

Cheers,

Richard

Re: Trailing slash in directory spec confuses libtool

[Richard Purdie]

On Fri, 2016-08-12 at 22:06 +0200, Jan Engelhardt wrote:
> Given certain circumstances, libtool 2.4.2 fails to install a
> library. 
> (a) The target directory spec contains a trailing slash
> (b) The library to install is linking to another just-built one in a 
> different path.

FWIW we have this patch:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/recipes-devtools/libtool/libtool/trailingslash.patch

which seems to help. We've never been able to clean it up enough to
submit upstream or have a reliable test case for it though :(

Cheers,

Richard

[patch follows below]

A command like /bin/sh ../../i586-poky-linux-libtool   --mode=install 
/usr/bin/install -c   gck-roots-store-standalone.la 
'/media/data1/builds/poky1/tmp/work/core2-poky-linux/gnome-keyring-2.26.1-r1/image/usr/lib/gnome-keyring/standalone/'
 fails (e.g. gnome-keyring or pulseaudio)

This is because libdir has a trailing slash which breaks the comparison.

RP 2/1/10

Merged a patch received from Gary Thomas 

Date: 2010/07/12
Nitin A Kamble 

Updated by: Robert Yang 

diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -2356,8 +2356,15 @@ func_mode_install ()
func_append dir "$objdir"
 
if test -n "$relink_command"; then
+  # Strip any trailing slash from the destination.
+  func_stripname '' '/' "$libdir"
+  destlibdir=$func_stripname_result
+
+  func_stripname '' '/' "$destdir"
+  s_destdir=$func_stripname_result
+
  # Determine the prefix the user has applied to our future dir.
- inst_prefix_dir=`$ECHO "$destdir" | $SED -e "s%$libdir\$%%"`
+ inst_prefix_dir=`$ECHO "X$s_destdir" | $Xsed -e "s%$destlibdir\$%%"`
 
  # Don't allow the user to place us outside of our expected
  # location b/c this prevents finding dependent libraries that

___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Bash-specific performance by avoiding sed

[Richard Purdie]

On Mon, 2015-03-09 at 18:28 +, Mike Gran wrote:
 I don't know if y'all saw this blogpost where a guy pushed
 the sed regular expression handling into bash-specific
 regular expressions when bash was available.  He claims
 there's a significant performance improvement because of
 reduced forking.
 
 http://harald.hoyer.xyz/2015/03/05/libtool-getting-rid-of-18-sed-forks/

Please see my reply about this. The issue is the amount of looping
through the options parsing code which I did already report here as an
issue.

I do have a fix for it in the build systems I look after:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=cc5f804483a9a1a60be24475baee0eed5d44bef5

which basically unrolls some of the internal code looping and I believe
that with that patch, the speedup to the specific sed invocation you're
looking at will be much less relevant.

The issue is that in its current form, it can't really be merged and my
shell knowledge and lack of contribution agreement to GNU are likely
blocking issues along with a lack of time :(

Cheers,

Richard




___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Tue, 2015-02-10 at 10:53 +, Gary V. Vaughan wrote:
 On Feb 10, 2015, at 10:35 AM, Richard Purdie 
 richard.pur...@linuxfoundation.org wrote:
  On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:
  On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
  In an effort to get to the bottom of this I made a git bisection, timing
  the performance of building xz with make -j1 using each different
  libtool.
  
  The issues come down to this commit:
  
  http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
  
  libtool: rewritten over funclib.sh instead of general.m4sh.
  
  Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
  in the fix in master mentioned above, I get 27s.
  
  So whilst things are better (thanks!), the above change is still causing
  a regression in the performance somewhere else. Any ideas what else in
  that rather large change may be causing this?
  
  To further narrow this down, of the changes in the above commit, the
  problem appears to be in the changes to the option parsing code. I've
  included the diff below which if I apply on top of the above, I get the
  speed back. I've left the func_split_short_opt/func_split_long_opt code
  in there but that is worth a tiny part of the speed, the issues are
  around the addition of the func_options call.
  
  As yet I don't know enough about the code in question to know why this
  is an issue but traces of libtool show a lot more looping in code to do
  with argument parsing and quoting.
  
  To be more specific, if I take my good libtool and add:
  
  func_options_prep ${1+$@}
  
  it slows the build down by 0.5s on a 21s build. If I look at
  func_options_prep and comment out the line:
  
 func_run_hooks func_options_prep ${1+$@}
  
  I get the 0.5s back.
  
  In func_run_hooks, if I comment:
  
 func_quote_for_eval ${1+$@}
 func_run_hooks_result=$func_quote_for_eval_result
  
  I get the 0.5s back. The issue is all the quoting of the various return
  values through all this looping. It doesn't appear to be hitting the
  printf/sed in func_quote_for_eval which would be an obvious slow path,
  its just the shear number of loops run through with the commandline
  arguments. The change adds a number of calls to func_run_hooks, not just
  the single test case I have above and all combined, it slows things down
  significantly.
  
  So is there a way we can change things so its not calling
  func_quote_for_eval all the time with all the looping that entails?
 
 One possibility would be to add a post-processing script that rewrites
 the hook functions used by func_options into a a single top-level blob of
 sequential shell code.  I imagine that adding some carefully chosen comment
 strings would make extracting the right parts in the right order relatively
 straight forward... it might even be that inlining func_options_prep and
 any hook functions it calls would be enough?

Thanks for the background info on this. I understand the need to change
and improve the software so I'm not proposing reverting, I do think
there has to be some way to get some of the speed here back though.

I have a bit of a pressing need to have a things performing as they were
and I'd prefer to stay on libtool 2.4.5 than revert back to 2.4.2 so I
cooked up the patch I've included below. This basically manually unrolls
the key problematic parts, cut and paste from options-parser. With this
applied to master, I have a build time of 22s compared to 20s or 21s
(need to go and retest as I'm forgetting numbers) with 2.4.2.

For now I'll probably merge something like this into the Yocto
Project/OpenEmbedded, the exact way to solve this longer term is TBD.

Cheers,

Richard




diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index d5cf07a..0f54303 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
@@ -342,11 +342,15 @@ _LT_EOF
 # libtool_options_prep [ARG]...
 # -
 # Preparation for options parsed by libtool.
-libtool_options_prep ()
-{
+#libtool_options_prep ()
+#{
 $debug_mode
 
 # Option defaults:
+opt_verbose=false
+opt_warning_types=
+
+# Option defaults:
 opt_config=false
 opt_dlopen=
 opt_dry_run=false
@@ -382,19 +386,14 @@ libtool_options_prep ()
   shift; set dummy --mode uninstall ${1+$@}; shift
   ;;
 esac
-
-# Pass back the list of options.
-func_quote_for_eval ${1+$@}
-libtool_options_prep_result=$func_quote_for_eval_result
-}
-func_add_hook func_options_prep libtool_options_prep
+#}
 
 
 # libtool_parse_options [ARG]...
 # -
 # Provide handling for libtool specific options.
-libtool_parse_options ()
-{
+#libtool_parse_options ()
+#{
 $debug_cmd
 
 # Perform our own loop to consume as many options as possible in
@@ -474,29 +473,90 @@ libtool_parse_options ()
 func_append preserve_args  $_G_opt

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-09 at 23:36 +, Richard Purdie wrote:
 On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
  In an effort to get to the bottom of this I made a git bisection, timing
  the performance of building xz with make -j1 using each different
  libtool.
  
  The issues come down to this commit:
  
  http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
  
  libtool: rewritten over funclib.sh instead of general.m4sh.
  
  Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
  in the fix in master mentioned above, I get 27s.
  
  So whilst things are better (thanks!), the above change is still causing
  a regression in the performance somewhere else. Any ideas what else in
  that rather large change may be causing this?
 
 To further narrow this down, of the changes in the above commit, the
 problem appears to be in the changes to the option parsing code. I've
 included the diff below which if I apply on top of the above, I get the
 speed back. I've left the func_split_short_opt/func_split_long_opt code
 in there but that is worth a tiny part of the speed, the issues are
 around the addition of the func_options call.
 
 As yet I don't know enough about the code in question to know why this
 is an issue but traces of libtool show a lot more looping in code to do
 with argument parsing and quoting.

To be more specific, if I take my good libtool and add:

func_options_prep ${1+$@}

it slows the build down by 0.5s on a 21s build. If I look at
func_options_prep and comment out the line:

func_run_hooks func_options_prep ${1+$@}

I get the 0.5s back.

In func_run_hooks, if I comment:

func_quote_for_eval ${1+$@}
func_run_hooks_result=$func_quote_for_eval_result

I get the 0.5s back. The issue is all the quoting of the various return
values through all this looping. It doesn't appear to be hitting the
printf/sed in func_quote_for_eval which would be an obvious slow path,
its just the shear number of loops run through with the commandline
arguments. The change adds a number of calls to func_run_hooks, not just
the single test case I have above and all combined, it slows things down
significantly.

So is there a way we can change things so its not calling
func_quote_for_eval all the time with all the looping that entails?

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-09 at 10:45 +0800, Robert Yang wrote:
 On 02/06/2015 10:46 PM, Bob Friesenhahn wrote:
  On Fri, 6 Feb 2015, Robert Yang wrote:
 
 
 
  On 02/06/2015 12:12 PM, Bob Friesenhahn wrote:
  I am not seeing quite the difference between libtool releases that you are
  although I see a big slowdown starting with 2.4.3.  These timings are for
  optimized builds of GraphicsMagick on a 12-core GNU/Linux system using -j 
  12:
 
  I think that we can't see obviously slowdown by make -jN,
  but make -j1 will. And bash is much slower than dash, I'm trying to
  figure out why.
 
  It seems like this issue is already corrected in the source tree but you are
 
 Yes, I think that the git repo has fixed the problem:
 
 commit 408cfb9c5fa8a666917167ffb806cb19deded429
 Author: Gary V. Vaughan g...@gnu.org
 Date:   Fri Feb 6 12:58:34 2015 +
 
  libtool: don't execute automake and autoconf on every invocation.

In an effort to get to the bottom of this I made a git bisection, timing
the performance of building xz with make -j1 using each different
libtool.

The issues come down to this commit:

http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430

libtool: rewritten over funclib.sh instead of general.m4sh.

Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
in the fix in master mentioned above, I get 27s.

So whilst things are better (thanks!), the above change is still causing
a regression in the performance somewhere else. Any ideas what else in
that rather large change may be causing this?

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Performance issue of libtool-2.4.4

[Richard Purdie]

On Mon, 2015-02-09 at 13:05 +, Richard Purdie wrote:
 In an effort to get to the bottom of this I made a git bisection, timing
 the performance of building xz with make -j1 using each different
 libtool.
 
 The issues come down to this commit:
 
 http://git.savannah.gnu.org/cgit/libtool.git/commit/?id=0a42997c6032b9550a009a271552e811bfbcc430
 
 libtool: rewritten over funclib.sh instead of general.m4sh.
 
 Before that, I get a time of about 20s, after it, 39s. If I cherry-pick
 in the fix in master mentioned above, I get 27s.
 
 So whilst things are better (thanks!), the above change is still causing
 a regression in the performance somewhere else. Any ideas what else in
 that rather large change may be causing this?

To further narrow this down, of the changes in the above commit, the
problem appears to be in the changes to the option parsing code. I've
included the diff below which if I apply on top of the above, I get the
speed back. I've left the func_split_short_opt/func_split_long_opt code
in there but that is worth a tiny part of the speed, the issues are
around the addition of the func_options call.

As yet I don't know enough about the code in question to know why this
is an issue but traces of libtool show a lot more looping in code to do
with argument parsing and quoting.

Cheers,

Richard



--- libtool-bad 2015-02-09 16:34:30.702068736 +
+++ libtool 2015-02-09 23:31:37.238750764 +
@@ -791,13 +791,77 @@
 fi
 }
 
+exit_status=$EXIT_SUCCESS
 
-# libtool_options_prep [ARG]...
-# -
-# Preparation for options parsed by libtool.
-libtool_options_prep ()
-{
-$debug_mode
+# We should try to minimise forks, especially on Windows where they are
+# unreasonably slow, so skip the feature probes when bash or zsh are
+# being used:
+if test set = ${BASH_VERSION+set}$ZSH_VERSION; then
+: ${lt_HAVE_ARITH_OP=yes}
+: ${lt_HAVE_XSI_OPS=yes}
+fi
+
+
+
+# lt_HAVE_XSI_OPS
+# Can be empty, in which case the shell is probed, yes if XSI length
+# and matching operators are useable or anything else if they do not work.
+test -z $lt_HAVE_XSI_OPS \
+ (eval 'x=a/b/c;
+  test 5aa/bb/cc = ${#x}${x%%/*}${x%/*}${x#*/}${x##*/}') 2/dev/null \
+ lt_HAVE_XSI_OPS=yes
+
+
+
+# If this shell supports prefix and suffix pattern removal, then
+# use them to avoid forking. Hide the definition in an eval in case
+# the shell chokes on unsupported syntax...
+if test yes = $lt_HAVE_XSI_OPS; then
+  # func_split_short_opt shortopt
+  # Set func_split_short_opt_name and func_split_short_opt_arg shell
+  # variables after splitting SHORTOPT after the 2nd character.
+  eval 'func_split_short_opt ()
+  {
+$debug_cmd
+
+func_split_short_opt_arg=${1#??}
+func_split_short_opt_name=${1%$func_split_short_opt_arg}
+  }'
+
+  # func_split_long_opt longopt
+  # Set func_split_long_opt_name and func_split_long_opt_arg shell
+  # variables after splitting LONGOPT at the `=' sign.
+  eval 'func_split_long_opt ()
+  {
+func_split_long_opt_name=${1%%=*}
+func_split_long_opt_arg=${1#*=}
+  }'
+else
+  # ...otherwise fall back to using sed.
+  func_split_short_opt ()
+  {
+my_sed_short_opt='1s/^\(..\).*$/\1/;q'
+my_sed_short_rest='1s/^..\(.*\)$/\1/;q'
+
+func_split_short_opt_name=`$ECHO $1 | $SED $my_sed_short_opt`
+func_split_short_opt_arg=`$ECHO $1 | $SED $my_sed_short_rest`
+  }
+
+  func_split_long_opt ()
+  {
+my_sed_long_opt='1s/^\(--[^=]*\)=.*/\1/;q'
+my_sed_long_arg='1s/^--[^=]*=//'
+
+func_split_long_opt_name=`$ECHO $1 | $SED $my_sed_long_opt`
+func_split_long_opt_arg=`$ECHO $1 | $SED $my_sed_long_arg`
+  }
+fi
+
+debug_cmd=${debug_cmd-':'}
+opt_warning=:
+opt_verbose=:
+opt_verbose=false
+exit_cmd=:
 
 # Option defaults:
 opt_config=false
@@ -836,26 +900,23 @@
   ;;
 esac
 
-# Pass back the list of options.
-func_quote_for_eval ${1+$@}
-libtool_options_prep_result=$func_quote_for_eval_result
-}
-func_add_hook func_options_prep libtool_options_prep
 
 
-# libtool_parse_options [ARG]...
-# -
-# Provide handling for libtool specific options.
-libtool_parse_options ()
+# Parse options once, thoroughly.  This comes as soon as possible in the
+# script to make things like `--version' happen as quickly as we can.
 {
-$debug_cmd
-
 # Perform our own loop to consume as many options as possible in
 # each iteration.
 while test $# -gt 0; do
   _G_opt=$1
   shift
   case $_G_opt in
+
+--debug|-x)debug_cmd='set -x'
+   func_echo enabling shell trace mode
+   $debug_cmd
+   ;;
+
 --dry-run|--dryrun|-n)
 opt_dry_run=:
 ;;
@@ -927,29 +988,39 @@
 func_append preserve_args  $_G_opt
 ;;
 
-   # An option not handled by this hook function:
-*) set dummy $_G_opt

Re: Libtool 2.4.3 release

[Richard Purdie]

On Thu, 2014-03-20 at 18:07 +1300, Gary V. Vaughan wrote:
 On Mar 20, 2014, at 5:37 AM, Arnout Vandecappelle
 arnout.vandecappe...@essensium.com wrote:
  [Please keep me in CC, I'm not on the list]
  Dear libtool maintainers,

  Is there a possibility for a new libtool release in the foreseeable
  future?

 Yes, absolutely. In fact there are only 2 things ahead of it on my
 TODO list:

   1. Figure out why a4ffcdb5e is a regression for test 57
   2. fix test 120 race condition

 Unfortunately, Libtool is a complex beast, and we are woefully
 undermanned here.
 While everything rests on my shoulders, it will be at least another
 month before I can start work (I'm in the process of emigrating and
 all that entails).

 Patches for those 2 items, or any other as yet unknown issues with git
 master (or http://vaughan.pe/libtool/libtool-2.4.2.458.tar.gz if a
 bootstrapped tarball is easier to work with) are extremely welcome,
 and could lead to an immediate release...

FWIW I'm another build system maintainer who sees email here. We
currently have 12 patches against libtool:

http://git.yoctoproject.org/cgit.cgi/poky/tree/meta/recipes-devtools/libtool/libtool

Some of these are inappropriate for upstream, one or two may have been
merged into libtool, some others may highlight issues, particularly
around the sysroot support.

Unfortunately whilst I have the best intentions (and am still on the
list), I haven't found the time to look into the issues as yet and
figure out if we could get some into a form that they'd be accepted
upstream.

Cheers,

Richard




___
https://lists.gnu.org/mailman/listinfo/libtool

Re: libtool - warnings when installing GMP using DESTDIR

[Richard Purdie]

On Thu, 2013-09-19 at 15:58 -0400, Michael Young wrote:
 I'm trying to do a staged build/install of libgmp(xx) release 5.1.2.
  This is
 part of an effort to develop a general build system (mostly bash
 scripts, make
 files, etc.) for toolchains I will be using.  I intend to use these
 scripts to
 build both native and cross-builds, depending on configuration /
 arguments, so
 using DESTDIR for prefixing the install tree is important.  Right now,
 I'm 
 working on native builds on an Ubuntu 12.04 32-bit x86 machine;
 libtool 
 version = 2.4.2.)
 
I can't help with the libtool issue but there are several build systems
out there that already have these issues solves such as the one I work
on:

http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html

which runs on Linux and can build cross compilers targeting the common
architectures and the cross compiler itself can run on linux, windows
and osx.

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: no .pc file

[Richard Purdie]

On Fri, 2012-10-26 at 13:27 -0500, Bob Friesenhahn wrote:
 On Fri, 26 Oct 2012, Yaroslav Bulatov wrote:
 
  Oops my badthat was a bad paste from some auto-generated code.
 
  This is basically a modified version of .pc file I get when building
  zlib. Not sure how useful this is because you need to update prefix
  in this file manually each time you rebuild libtool. Ideally the .pc
  file would be generated automatically by configure/make
 
 If it is not clear, package config files are operating system and/or 
 operating system distribution and/or operating system release version 
 specific.  One reason for this is that each operating system 
 distribution uses its own names for pkg-config package definitions. 
 Using zlib as an example is not useful since zlib does not depend on 
 any other packages.  Most packages depend on other packages and so 
 there is an OS-distribution (or even site-specific) list of packages 
 that this package depends on.
 
 While many packages do produce sample pkg-config files (based on 
 guess-work or assumption of a partiticular OS distribuiton), it is 
 common practice for the default offering to be modified by the OS 
 package distribution maintainer because the OS uses a different name 
 for a similar thing.
 
 Being intended for portable software, libtool does not concern itself 
 with a hand-edited/non-portable framework like pkg-config.

As I understand it, the .pc files use their own namespace so once a
given piece of software has chosen its naming, other things can depend
on it using that name space and it doesn't matter about the OS
distribution or OS version used. This is a clear incentive to maintain
the .pc file with the software so that there is one common naming used,
at least in pkg-config space. There is no connection to the package
management namespace which is totally separate.

This assumes that people use some kind of common sense when choosing
namespace but other than that it seems to work well.

As one of the people looking after the Yocto Project (which includes an
build system targeted at embedded devices), I have to say I see less
problems with pkg-config than with libtool and I'm once again being
asked to remove all .la files as a policy decision due to them causing
more problems than they seem to solve in a cross environment :(. I'm
running out of arguments against this, not least as I couldn't get any
response to the libtool sysroot problems I reported a while back.

Cheers,

Richard


___
https://lists.gnu.org/mailman/listinfo/libtool

Re: Libtool sysroot RPATH problems

[Richard Purdie]

Hi Ralf,

Thanks for the reply.

On Thu, 2011-01-13 at 08:24 +0100, Ralf Wildenhues wrote:
 * Richard Purdie wrote on Wed, Jan 12, 2011 at 12:06:13AM CET:
  Firstly, for the first time ever for us, it appears libtool is no longer
  relinking our libraries at install time.
 
 That's weird, I don't think the sysroot feature should've caused this
 change.  Not quite sure though.
 
  This is welcome as we're cross
  compiling to a sysroot and we'd never want to actually run them in situ
  so this could probably save us some build time. It does however mean we
  never go for a relink step so we're hitting codepaths and issues that
  I've never seen. For reference, we always used to set installed=no in
  our staged libraries prior to sysroot support since this let us hack
  around the lack of sysroot support easier.
  
  Anyhow, the problem I'm seeing is that the final library has an RPATH
  including the sysroot prefix when make is calling libtool with
  -rpath /usr/lib at link time. I looked at the code in ltmain.m4sh
  starting at line 7240 (see the end of the email for a quotation) and
  firstly, I don't understand why the func_replace_sysroot call is inside
  the $hardcode_libdir_separator test? Surely it should be outside it and
  happen whenever $hardcode_libdir_flag_spec is set?

Firstly let me be upfront and clear, I do have some patches applied on
top of libtool 2.4 to address some problems. These are:

http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/rename-with-sysroot.patch?h=sgarman/libtool-sysroot
(to stop conflicts with gcc/binutils configure options)
http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/prefix.patch?h=sgarman/libtool-sysroot
(renamed libtool to TARGET_PREFIX-libtool, which is mainly useful to
detect when our libtool patches aren't being noticed)
http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/trailingslash.patch?h=sgarman/libtool-sysroot
(a path comparison problem I never did get to the bottom of to report
properly but the fix appears harmless)
http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/use-sysroot-in-libpath.patch?h=sgarman/libtool-sysroot
(a sysroot fix I believe is in libtool git now)

I don't believe any of these are contributing to the issues I'm seeing
though.

http://git.pokylinux.org/cgit.cgi/poky-contrib/tree/meta/recipes-devtools/libtool/libtool/fix-final-rpath.patch?h=sgarman/libtool-sysroot
is a WIP patch which addresses some of the problems I refer to in this
email thread. Its not applied for the tests below.

 Please show the --mode={link,install} commands plus all of their output,
 also './libtool --config'.  Please also show how exactly you invoke
 configure.  Thanks.

A link command and output showing the problem:

/bin/sh ./i586-poky-linux-libtool --tag=CC   --mode=link i586-poky-linux-gcc 
-march=i586 
--sysroot=/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux  
-fexpensive-optimizations -fomit-frame-pointer -frename-registers -O2 -ggdb 
-feliminate-unused-debug-types -no-undefined -export-dynamic -version-number 
0:44:0 -Wl,--version-script=libpng.vers  -Wl,-O1 -Wl,--hash-style=gnu 
-Wl,--as-needed -o libpng12.la -rpath /usr/lib libpng12_la-png.lo 
libpng12_la-pngset.lo libpng12_la-pngget.lo libpng12_la-pngrutil.lo 
libpng12_la-pngtrans.lo libpng12_la-pngwutil.lo libpng12_la-pngread.lo 
libpng12_la-pngrio.lo libpng12_la-pngwio.lo libpng12_la-pngwrite.lo 
libpng12_la-pngrtran.lo libpng12_la-pngwtran.lo libpng12_la-pngmem.lo 
libpng12_la-pngerror.lo libpng12_la-pngpread.lo  -lz -lm 
i586-poky-linux-libtool: link: i586-poky-linux-gcc -march=i586 
--sysroot=/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux -shared  
-fPIC -DPIC  .libs/libpng12_la-png.o .libs/libpng12_la-pngset.o 
.libs/libpng12_la-pngget.o .libs/libpng12_la-pngrutil.o 
.libs/libpng12_la-pngtrans.o .libs/libpng12_la-pngwutil.o 
.libs/libpng12_la-pngread.o .libs/libpng12_la-pngrio.o 
.libs/libpng12_la-pngwio.o .libs/libpng12_la-pngwrite.o 
.libs/libpng12_la-pngrtran.o .libs/libpng12_la-pngwtran.o 
.libs/libpng12_la-pngmem.o .libs/libpng12_la-pngerror.o 
.libs/libpng12_la-pngpread.o   -Wl,-rpath 
-Wl,/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux/usr/lib 
/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux/usr/lib/libz.so -lm 
 -march=i586 
--sysroot=/media/build2/builds/rptest/b2/tmp/sysroots/i586-poky-linux -O2 
-Wl,--version-script=libpng.vers -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed   
-Wl,-soname -Wl,libpng12.so.0 -o .libs/libpng12.so.0.44.0
i586-poky-linux-libtool: link: (cd .libs  rm -f libpng12.so.0  ln -s 
libpng12.so.0.44.0 libpng12.so.0)
i586-poky-linux-libtool: link: (cd .libs  rm -f libpng12.so  ln -s 
libpng12.so.0.44.0 libpng12.so)
mv -f .deps/libpng_la-pngwtran.Tpo .deps/libpng_la-pngwtran.Plo
i586-poky-linux-libtool: link: i586-poky-linux-ar cru .libs/libpng12

Libtool sysroot RPATH problems

[Richard Purdie]

Hi,

I work on OpenEmbedded/Poky and we're been experimenting with the recent
libtool sysroot support. To say I'm pleased to have this is an
understatement! We're having a few problems around incorrect RPATHs
being encoded into libraries however.

Firstly, for the first time ever for us, it appears libtool is no longer
relinking our libraries at install time. This is welcome as we're cross
compiling to a sysroot and we'd never want to actually run them in situ
so this could probably save us some build time. It does however mean we
never go for a relink step so we're hitting codepaths and issues that
I've never seen. For reference, we always used to set installed=no in
our staged libraries prior to sysroot support since this let us hack
around the lack of sysroot support easier.

Anyhow, the problem I'm seeing is that the final library has an RPATH
including the sysroot prefix when make is calling libtool with
-rpath /usr/lib at link time. I looked at the code in ltmain.m4sh
starting at line 7240 (see the end of the email for a quotation) and
firstly, I don't understand why the func_replace_sysroot call is inside
the $hardcode_libdir_separator test? Surely it should be outside it and
happen whenever $hardcode_libdir_flag_spec is set?

Changing that helps a bit and I end up with an RPATH of =/usr/lib so
the sysroot prefix is gone but the = is there. I suspect that isn't
valid so I added the following code after the func_replace_sysroot call

func_stripname '=' '' $libdir
libdir=$func_stripname_result

and then I get an RPATH of /usr/lib which is ok to a point.

Of course this is listed in $sys_lib_dlsearch_path_spec so shouldn't be
getting added at all. Its coming to this function via $compile_rpath
which is being set around line 5939 which in turn is coming from absdir.
absdir is being compared against $sys_lib_dlsearch_path_spec but without
adding/removing any sysroot prefixes.

I'd really therefore like to ask what behaviour to be expecting from
libtool before I try and write patches to fix any of this. To summarise
some of my questions:

a) If I'm using a sysroot can I expect to install without a relink step 
   (I hope so!)
b) I couldn't see a function to add/prepend a sysroot without a = in 
   there. I assume given these usecases we should add one?
c) An RPATH starting with an = is invalid, correct?
d) Should the absdir comparisions with sys_lib_dlsearch_path_spec have 
   the sysroot stripped? Are there other rpath variables that need this 
   treatment when comparing to sys_lib_dlsearch_path_spec?
e) The sysroot treatment should apply whenever 
   $hardcode_libdir_flag_spec is set and not on 
   $hardcode_libdir_separator?

If someone could explain the correct behaviour I might be able to come
up with some patches to help fix things! :)

Cheers,

Richard

http://git.savannah.gnu.org/cgit/libtool.git/tree/libltdl/config/ltmain.m4sh?id=9167aecabd12c5afe7a65d45dc73f8c92ab42f05
line 7240:
  # Test again, we may have decided not to build it any more
  if test $build_libtool_libs = yes; then
# Remove ${wl} instances when linking with ld.
# FIXME: should test the right _cmds variable.
case $archive_cmds in
  *\$LD\ *) wl= ;;
esac
if test $hardcode_into_libs = yes; then
  # Hardcode the library paths
  hardcode_libdirs=
  dep_rpath=
  rpath=$finalize_rpath
  test $opt_mode != relink  rpath=$compile_rpath$rpath
  for libdir in $rpath; do
if test -n $hardcode_libdir_flag_spec; then
  if test -n $hardcode_libdir_separator; then
func_replace_sysroot $libdir
libdir=$func_replace_sysroot_result
if test -z $hardcode_libdirs; then
  hardcode_libdirs=$libdir
line 5929:
  if test $linkmode = lib 
 test $hardcode_into_libs = yes; then
# Hardcode the library path.
# Skip directories that are in the system default run-time
# search path.
case  $sys_lib_dlsearch_path  in
* $absdir *) ;;
*)
  case $compile_rpath  in
  * $absdir *) ;;
  *) func_append compile_rpath  $absdir ;;
  esac
  ;;
esac


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: Issues w/ relink and cross-compilation

[Richard Purdie]

Hi,

I've spoken about this in the past but it looks like I should mention it
again. I do extensive cross compiling of the entire Linux software stack
using OpenEmbedded and Poky (I maintain the latter). We have cross
compiling working with libtool with a variety of workarounds. This
mainly consists of the following simple patch to libtool itself:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool/cross_compile.patch

and then manipulation of the .la files with sed if we ever want to move
the sysroot directory, or use the .la files on the target device.

There are a couple of other patches we use. One is to allow multiple
libtool scripts to live in the same place by adding a cross prefix to
the tool name:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool/prefix.patch

This patch also allows us to tell when the wrong (unpatched) libtool
is being used. 

The final patch we're currently applying is probably a bug in libtool:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool/trailingslash.patch

where a directory comparison was failing as one variable ended in a
slash and the other did not. We're fixing this by stripping any slash
off both options (the patch is pending an update to do this). I haven't
isolated a proper test case to be able to submit this one as a proper
bug yet though.

Longer term I'm still hoping we'll see sysroot support in libtool and I
may even find some time to start the ball rolling on patches eventually
but I'm not going to get to that soon, much as I wish it were otherwise.

Cheers,

Richard


___
http://lists.gnu.org/mailman/listinfo/libtool

Re: Issue with libtool when cross-compiling

[Richard Purdie]

On Fri, 2010-02-12 at 16:34 +0100, Thomas Petazzoni wrote:
 I'm a contributor to the Buildroot project (http://www.buildroot.org),
 a build system for embedded Linux systems. We integrate many packages
 in order to ease their cross-compilation.
 
 I'm facing an issue with several packages where libtool wants to link
 against libraries of the host (in /usr/lib), and I don't know what
 is happening. It is very likely a problem in the environment variables
 or the options we pass to the configure script, but I'm unable to find
 where the problem is, and thought that libtool experts might have an
 idea on what's going on.

libtool has known cross compile issues and doesn't get on well with
sysroots. 

I maintain the OpenEmbedded libtool patch set which at least lets us
work around the worst libtool issues. The main patch is:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool-2.2.6/cross_compile.patch

and there is a bugfix I've been meaning to look at in more detail which
became neccessary with recent autoconf/automake versions:

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool-2.2.6/trailingslash.patch

where a trailing slash on paths causes a string comparison to fail. We
also apply: 

http://git.pokylinux.org/cgit.cgi/poky/tree/meta/packages/libtool/libtool-2.2.6/prefix.patch

which renames libtool and allows us to easily tell when the wrong
(unpatched) libtool is being used.

There are other workarounds we have to apply after installing packages,
particularly modifying the .la files to be uninstalled to get the
sysroot to work properly. I'd love to teach libtool about sysroots and
make it work properly but as yet have never had the time to look into
it.

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool

Re: Issue with libtool when cross-compiling

[Richard Purdie]

On Fri, 2010-02-12 at 19:34 +0100, Thomas Petazzoni wrote:
 I suppose you apply these modifications to a libtool that you are
 compiling. However, libtool is typically already embedded in each
 upstream tarball. Do you run autoreconf on all packages, so that a
 newer libtool script gets generated (autoreconf-ing each package takes a
 lot of time) ? Do you somehow replace the provided libtool by a fixed
 one ?
 
 Moreover, some packages use libtool 1.5.x and some newer packages use
 libtool 2.2.x. How do you handle this version difference ?

Due to exactly this kind of problem OpenEmbedded generally autoreconfs
all sources.

  There are other workarounds we have to apply after installing
  packages, particularly modifying the .la files to be uninstalled to
  get the sysroot to work properly.
 
 « to be uninstalled » ?

Set installed=no in the .la files.

 In Buildroot, we already modify the libdir='' in all the .la files so
 that they refer to the sysroot instead of /usr/lib.

We change the dependency_libs line. The libdir line refers to the final
install location and /usr/lib is in fact valid there.

 If possible, could you point me to the location in OpenEmbedded where
 libtool is handled ? I already had a look at your patches, but don't
 understand how you re-generate the libtool script inside the different
 upstream packages.

meta/classes/autoconf.bbclass has the autoreconf code

meta/classes/base.bbclass has the .la file manipulation.

What I find frustrating is that buildroot is playing catchup to OE these
days. By the time you make all the improvements to buildroot, you end up
with OE.

What would it take to get the buildroot people using OE and focusing
effort to new problems, not ones already at least partially solved?

I'd love to see a kconfig wrapper to OE for example...

Cheers,

Richard





___
http://lists.gnu.org/mailman/listinfo/libtool

Re: removal of .la files from Debian and a possible solution to the libtool shared libs problem

[Richard Purdie]

On Tue, 2009-08-25 at 20:44 +0200, Ralf Wildenhues wrote:
 * Bob Friesenhahn wrote on Tue, Aug 25, 2009 at 05:17:49PM CEST:
  On Tue, 25 Aug 2009, Anssi Hannula wrote:
  
  I think the proper way to solve this is to not link to dependency_libs
  when linking dynamically on systems where it is not needed to link to
  those. I haven't seen any correctly working patches that implement this.
  
  Relying on the OS's implicit dependency features seems to be an
  approach which is fraught with peril.
 
 With GNU/Linux, and libraries all being in directories searched by
 default by both the link editor and the runtime linker, the problems
 are fairly limited.  IIRC Debian requires that you link directly against
 all libraries that you require directly.
 
 The problems start as soon as you link (directly or indirectly) against
 libraries in directories not searched by default.  IOW: typically
 anything not provided by a properly packaged Debian package, installed
 by the user or the system maintainer.

Surely at least on Linux the -rpath linker option would be a much better
way to solve this?

Linux does seem to have good dynamic linker support and its a shame
libtool effectively drags it down to a lower common denominator of other
platforms with worse support.

Cheers,

Richard

-- 
Richard Purdie
Intel Open Source Technology Centre



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: how to link with libtool?

[Richard Purdie]

On Tue, 2009-02-03 at 11:05 +0100, Matěj Týč wrote:
  Yes, it may be a good idea if somebody wrote this.
  It should probably depend on LT_OUTPUT.
 
  OTOH, the havelib module from gnulib already provides quite a bit of
  functionality in this area.
 
 OK, but what should I tell to the library users? Using gnulib is quite
 troublesome since
 it does not have proper documentation and usage of the library would become 
 too
 complicated for a casual programmer.
 And I don't like pkg-config since it breaks cross-compilation...

Just for reference, pkg-config should not break cross-compilation after
the addition of sysroot support to it. I keep meaning to look into
sysroot support for libtool too, I just haven't had the time yet :(.

Cheers,

Richard

-- 
Richard Purdie
Intel Open Source Technology Centre



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: [PATCH] Add 64 bit directories to sys_lib_dlsearch_path_spec for Linux ELF

[Richard Purdie]

On Wed, 2009-01-28 at 12:01 -0800, Dan Nicholson wrote:
 If they follow the LSB, then they should be using lib64. However, you
 can configure the linker any number of ways. I tried to get a patch to
 glibc so this path could be determined, but it was rejected.
 
 http://sourceware.org/ml/libc-alpha/2008-12/msg00052.html
 
 I think this is the only reasonable way to proceed, but I suppose that
 an existence test would be safe, too:
 
 sys_lib_dlsearch_path_spec=/lib /usr/lib
 case `/usr/bin/file conftest.o` in
 *64-bit*)
 for dir in /lib64 /usr/lib64; do
 test -d $dir 
 sys_lib_dlsearch_path_spec=$sys_lib_dlsearch_path_spec $dir
 done
 ;;
 esac

Just as a datapoint, my standard ubuntu 64 bit desktop has /lib64 as a
symlink to /lib which has 64 bit libraries in it.

Cheers,

Richard

-- 
Richard Purdie
Intel Open Source Technology Centre



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: Compiling into chroot

[Richard Purdie]

On Thu, 2008-06-12 at 22:03 +0300, Alon Bar-Lev wrote:
 On 6/12/08, Roumen Petrov [EMAIL PROTECTED] wrote:
   It look like an enhancement request. libtool to obey as example
  FAKEROOT=/tmp/device-root and to look first in $FAKEROOT/path1
   , ... and $FAKEROOT/pathN and later in /path1,... and /pathN .
 
 This what I had in mind.
 It also should append the FAKEROOT to pathes read from .la files.

You mean prepend :)

This is the same issue as the sysroot one I've previously mentioned.
OpenEmbedded currently hacks the .la files in its staging area to
include the sysroot prefix but this means you can't use staging as a
chroot. I've not had time to look into doing anything about this feature
but I would love to see it!

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: libtool 2.2.2, darwin and target prefixes

[Richard Purdie]

On Fri, 2008-05-02 at 01:03 -0500, Peter O'Gorman wrote:
 Peter O'Gorman wrote:
  Richard Purdie wrote:
  Hi,
 
  As previously mentioned, I've been stress testing libtool 2.2.2 with
  Poky a bit.
 
  I've found one issue when I tested the darwin builds, specifically that
  it tried to use otool and otool64 and not the versions with the host
  prefix which my setup has. I fixed this with the patch below which is
  fine for Poky since its always cross compiling but its a sign a better
  fix is probably needed in general.
  
  Thanks,
  I have pushed this, it also cleans up the sed sed echo ickyness.
 
 It is always better if quotes get closed. I had tested, honest!
 
 Pushed this.

I've updated Poky to use the proper fixes, thanks for looking at this!

Cheers,

Richard

Re: libtool 2.2.2, darwin and target prefixes

[Richard Purdie]

On Fri, 2008-05-02 at 01:03 -0500, Peter O'Gorman wrote:
 Peter O'Gorman wrote:
  Richard Purdie wrote:
  Hi,
 
  As previously mentioned, I've been stress testing libtool 2.2.2 with
  Poky a bit.
 
  I've found one issue when I tested the darwin builds, specifically that
  it tried to use otool and otool64 and not the versions with the host
  prefix which my setup has. I fixed this with the patch below which is
  fine for Poky since its always cross compiling but its a sign a better
  fix is probably needed in general.
  
  Thanks,
  I have pushed this, it also cleans up the sed sed echo ickyness.
 
 It is always better if quotes get closed. I had tested, honest!
 
 Pushed this.

I've updated Poky to use the proper fixes, thanks for looking at this!

Cheers,

Richard





___
http://lists.gnu.org/mailman/listinfo/libtool

libtool 2.2.2, darwin and target prefixes

[Richard Purdie]

Hi,

As previously mentioned, I've been stress testing libtool 2.2.2 with
Poky a bit.

I've found one issue when I tested the darwin builds, specifically that
it tried to use otool and otool64 and not the versions with the host
prefix which my setup has. I fixed this with the patch below which is
fine for Poky since its always cross compiling but its a sign a better
fix is probably needed in general.

Regards,

Richard

Index: libtool-2.2.2/libltdl/config/ltmain.m4sh
===
--- libtool-2.2.2.orig/libltdl/config/ltmain.m4sh   2008-05-01 
12:19:37.0 +0100
+++ libtool-2.2.2/libltdl/config/ltmain.m4sh2008-05-01 12:29:05.0 
+0100
@@ -4965,10 +4965,10 @@
done
if test -f $absdir/$objdir/$depdepl ; then
  depdepl=$absdir/$objdir/$depdepl
- darwin_install_name=`otool -L $depdepl | $SED -n -e 
'3q;2,2p' | $SED -e 's/(.*//'`
+ darwin_install_name=`$host-otool -L $depdepl | $SED -n -e 
'3q;2,2p' | $SED -e 's/(.*//'`
  darwin_install_name=`$ECHO $darwin_install_name`
   if test -z $darwin_install_name; then
-  darwin_install_name=`otool64 -L $depdepl | $SED -n 
-e '3q;2,2p' | $SED -e 's/(.*//'`
+  darwin_install_name=`$host-otool64 -L $depdepl | 
$SED -n -e '3q;2,2p' | $SED -e 's/(.*//'`
   darwin_install_name=`$ECHO $darwin_install_name`
   fi
  compiler_flags=$compiler_flags ${wl}-dylib_file 
${wl}${darwin_install_name}:${depdepl}



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: libtool 2.2.2, ccache and -all-static

[Richard Purdie]

Hi Ralf,

On Tue, 2008-04-22 at 21:43 +0200, Ralf Wildenhues wrote:
 Thanks for the bug report, and especially for providing an example to
 reproduce it!
 
  libtool: link: ccache -static gcc -O20 -ffast-math -D_REENTRANT 
  -fsigned-char -DUSE_MEMORY_H -o decoder_example decoder_example.o  
  ../lib/.libs/libvorbis.a -lm /usr/lib/libogg.a
 
 Confirmed.  Fixed as below, committed, put you in THANKS.

I've confirmed the fix, added it to Poky and dropped the hacks I had,
thanks!

Cheers,

Richard

libtool 2.2.2, ccache and -all-static

[Richard Purdie]

Hi,

I've noticed another problem with two packages in poky, prelink and
libvorbis. Both packages have areas where LDFLAGS=-all-static is used.

The problem comes about since Poky sets CC to ccache gcc, then libtool
puts the -static flag between ccache and gcc.

To reproduce:

wget http://www.vorbis.com/files/1.0.1/unix/libvorbis-1.0.1.tar.gz
tar -xvzf /usr/oe/sources/libvorbis-1.0.1.tar.gz
cd libvorbis-1.0.1
autoreconf -i
CC=ccache gcc ./configure
make

which results in:

make[1]: Entering directory `/usr/src/libvorbis-1.0.1/examples'
ccache gcc -DPACKAGE_NAME=\\ -DPACKAGE_TARNAME=\\ -DPACKAGE_VERSION=\\ 
-DPACKAGE_STRING=\\ -DPACKAGE_BUGREPORT=\\ -DPACKAGE=\libvorbis\ 
-DVERSION=\1.0.1\ -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 
-DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 
-DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 
-DLT_OBJDIR=\.libs/\ -DHAVE_ALLOCA_H=1 -DHAVE_ALLOCA=1 -I. -I../include 
-O20 -ffast-math -D_REENTRANT -fsigned-char  -DUSE_MEMORY_H -MT 
decoder_example.o -MD -MP -MF .deps/decoder_example.Tpo -c -o decoder_example.o 
decoder_example.c
mv -f .deps/decoder_example.Tpo .deps/decoder_example.Po
/bin/sh ../libtool --tag=CC   --mode=link ccache gcc  -O20 -ffast-math 
-D_REENTRANT -fsigned-char  -DUSE_MEMORY_H  -all-static -o decoder_example 
decoder_example.o ../lib/libvorbis.la -lm  -logg
libtool: link: ccache -static gcc -O20 -ffast-math -D_REENTRANT -fsigned-char 
-DUSE_MEMORY_H -o decoder_example decoder_example.o  ../lib/.libs/libvorbis.a 
-lm /usr/lib/libogg.a

which then fails with the ccache help message since it doesn't support a
-static option. Is it possible to fix this so the flag is properly
handled?

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool

Problems with libtool 2.2.2 and /bin/sh pointing to dash

[Richard Purdie]

Hi,

I've had reports of Poky breaking with libtool 2.2.2 and have isolated
this to people using dash as their /bin/sh provider (Poky runs the
configure script with /bin/sh). When used in this combination, the
global_symbol_pipe expression becomes corrupted in the generated libtool
file amongst other things and I've included a diff of the corruption
below. I noticed this with gtk+ 2.12.7. 

gtk+ also has the issues that it tries to run libtool before its been
generated and I've had to patch this to run a previously generated
version of libtool poky has around to solve cases like this. I'm not
sure if there is a neater way to address that problem?

Regards,

Richard

--- libtool-bash2008-04-21 17:57:40.0 +0100
+++ libtool-dash2008-04-21 23:36:38.0 +0100
@@ -1,4 +1,4 @@
-#! /bin/sh
+#! /bin/bash
 
 # arm-poky-linux-gnueabi-libtool - Provide generalized library-building 
support services.
 # Generated automatically by config.status (gtk+) 2.12.7
@@ -107,10 +107,10 @@
 lt_unset=unset
 
 # turn spaces into newlines.
-SP2NL=tr \\040 \\012
+SP2NL=tr \040 \012
 
 # turn newlines into spaces.
-NL2SP=tr \\015\\012 \\040\\040
+NL2SP=tr \015\012 \040\040
 
 # How to create reloadable object files.
 reload_flag= -r
@@ -141,22 +141,22 @@
 LTCFLAGS=-fexpensive-optimizations -fomit-frame-pointer -frename-registers 
-O2 -Wall
 
 # Take the output of nm and produce a listing of raw symbols and C names.
-global_symbol_pipe=sed -n -e 's/^.*[   ]\\([ABCDGIRSTW][ABCDGIRSTW]*\\)[  
 ][  ]*\\([_A-Za-z][_A-Za-z0-9]*\\)\$/\\1 \\2 \\2/p'
+global_symbol_pipe=sed -n -e 's/^.*[   ]\\([ABCDGIRSTW][ABCDGIRSTW]*\\)[  
 ][  ]*\\([_A-Za-z][_A-Za-z0-9]*\\)\$/
  /p'
 
 # Transform the output of nm in a proper C declaration.
-global_symbol_to_cdecl=sed -n -e 's/^T .* \\(.*\\)\$/extern int \\1();/p' -e 
's/^[ABCDGIRSTW]* .* \\(.*\\)\$/extern char \\1;/p'
+global_symbol_to_cdecl=sed -n -e 's/^T .* \\(.*\\)\$/extern int 
();/p' -e 
's/^[ABCDGIRSTW]* .* \\(.*\\)\$/extern char 
;/p'
 
 # Transform the output of nm in a C name address pair.
-global_symbol_to_c_name_address=sed -n -e 's/^: \\([^ ]*\\) \$/  
{\1\\\, (void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ ]*\\)\$/  
{\\\2\, (void *) 2},/p'
+global_symbol_to_c_name_address=sed -n -e 's/^: \\([^ ]*\\) \$/  {\\\
\\\, 
(void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ ]*\\)\$/  {\\, (void 
*) \\},/p'
 
 # Transform the output of nm in a C name address pair when lib prefix is 
needed.
-global_symbol_to_c_name_address_lib_prefix=sed -n -e 's/^: \\([^ ]*\\) \$/  
{\1\\\, (void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\(lib[^ 
]*\\)\$/  {\\\2\, (void *) 2},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ 
]*\\)\$/  {\lib\\2\, (void *) 2},/p'
+global_symbol_to_c_name_address_lib_prefix=sed -n -e 's/^: \\([^ ]*\\) \$/  
{\\\
\\\, (void *) 0},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\(lib[^ ]*\\)\$/  
{\\, (void *) \\},/p' -e 's/^[ABCDGIRSTW]* \\([^ ]*\\) \\([^ ]*\\)\$/  
{\lib\, (void *) \\},/p'
 
 # The name of the directory that contains temporary libtool files.
 objdir=.libs
 
 # Shell to use when invoking shell scripts.
-SHELL=/bin/sh
+SHELL=/bin/bash
 
 # An echo program that does not interpret backslashes.
 ECHO=echo
@@ -301,7 +301,7 @@
 # Commands used to build a shared archive.
 archive_cmds=\$CC -shared \$libobjs \$deplibs \$compiler_flags \${wl}-soname 
\$wl\$soname -o \$lib
 archive_expsym_cmds=echo \\\{ global:\\\  \$output_objdir/\$libname.ver~
-   cat \$export_symbols | sed -e \\\s/(.*)/1;/\\\  
\$output_objdir/\$libname.ver~
+   cat \$export_symbols | sed -e \\\s/(.*)/
;/\\\  
\$output_objdir/\$libname.ver~
echo \\\local: *; };\\\  \$output_objdir/\$libname.ver~
\$CC -shared \$libobjs \$deplibs \$compiler_flags \${wl}-soname 
\$wl\$soname \${wl}-version-script \${wl}\$output_objdir/\$libname.ver -o \$lib
 



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: libtool.m4 not always copied

[Richard Purdie]

Hi,

On Wed, 2008-04-16 at 19:03 +0200, Vincent Torri wrote:
 I'm using libtool 2.2.3a (cvs, actually) and i'm trying to find out why 
 some lib are not configured correctly, while other are.
 
 My problem is that ECHO and OBJDUMP are not defined when I'm configuring 
 libpng 1.2.26, while it's perfectly defined with another lib. I'm doing 
 cross-compiling for the cegcc compiler (i've improved the patch i've sent 
 to the libtool patch ML, but i want to solve all the problems i encounter 
 before sending it again).
 
 I've grep'ed OBJDUMP and i've remarked that with libpng, it's never 
 defined, and in addition libtool.m4 and other m4 files that lib should 
 (must ?) copy are not in the libpng directory. I don't know if it's 
 related, but i prefer asking anyway :)
 
 I have the same problem with ECHO, which is not defined too.
 
 With the other lib, there is no problem at all. The m4 files are correctly 
 copied and used.
 
 So I would like to know why libtoolize does not copy those files with the 
 libpng library. And if someone knows why OBJDUMP is not defined, i would 
 be glad to know :)

I'm in the process of upgrading Poky to use libtool 2.2.2 and noticed
the echo problem. libtool.m4 was correctly added to aclocal.m4 though
and I didn't see the OBJDUMP problem.

To 'fix' the echo issue I added:

http://svn.o-hand.com/view/poky/trunk/meta/packages/libpng/libpng-1.2.16/makefile_fix.patch?rev=4267view=markup

although I suspect echo = @ECHO@ might be better. I think libpng has
relied on old libtool behaviour which happened to export that and no
longer does so its a libpng bug but I'm open to more informed comments!

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool

Re: libtool.m4 not always copied

[Richard Purdie]

On Wed, 2008-04-16 at 23:56 +0200, Ralf Wildenhues wrote:
 * Richard Purdie wrote on Wed, Apr 16, 2008 at 07:10:54PM CEST:
  To 'fix' the echo issue I added:
  
  http://svn.o-hand.com/view/poky/trunk/meta/packages/libpng/libpng-1.2.16/makefile_fix.patch?rev=4267view=markup
  
  although I suspect echo = @ECHO@ might be better. I think libpng has
  relied on old libtool behaviour which happened to export that and no
  longer does so its a libpng bug but I'm open to more informed comments!
 
 Sorry, but your patch just hides the real bug that you're using macro
 files from Libtool 1.5.x.  You will see other, more obscure bugs (but
 then again, with their nature of being obscure, you may not see them,
 but curse a lot about how broken Libtool is).
 
 Please drop the patch and get updated macros into your tree.

I am not using 1.5 macros. libtool 2.2.2 is being used, as I can see if
I grep aclocal.m4 or ltmain.sh for the libtool version.

Let me try again to explain the problem. Makefile.am contains:

$(ECHO) [EMAIL PROTECTED]@@[EMAIL PROTECTED] '{global:'  [EMAIL 
PROTECTED]
$(SED) s/$$/\;/ libpng.sym  [EMAIL PROTECTED]
$(ECHO) 'local: *; };'  [EMAIL PROTECTED]

automake expands ECHO_C, ECHO_N, ECHO_T and lt_ECHO but not ECHO
anymore. libtool 1.5 macros happened to let ECHO be expanded but in
2.2.2 this is lt_ECHO.

Since nothing expands ECHO anymore this breaks with libtool 2.2.2.

I think that libpng is in the wrong here though and it therefore needs
to add something to the Makefile.am to cause ECHO to expand.

Cheers,

Richard





___
http://lists.gnu.org/mailman/listinfo/libtool

Re: Announcing Dolt, a drop-in Libtool replacement which cuts build times in half

[Richard Purdie]

Josh Triplett josh at freedesktop.org writes:
 Thus, I wrote Dolt, a drop-in replacement for libtool's compilation
 mode.  Dolt runs any necessary system-specific or
 configuration-specific logic as part of configure, writes out a simple
 shell script doltcompile[1], and substitutes it for libtool in the
 automake variables LTCOMPILE and LTCXXCOMPILE.  If you use automake,
 autoconf, and libtool, then using Dolt just requires two steps:
 
 1) add DOLT after the call to LT_INIT, AC_PATH_LIBTOOL, or
AM_PATH_LIBTOOL in your configure.ac or configure.in script, and
 2) append dolt.m4 to your project's acinclude.m4.
 
 For any system Dolt does not support, it will transparently fall back
 to libtool.

I work with Poky (pokylinux.org) and OpenEmbedded (openembedded.org) which are
cross compiling build systems and speed improvements like this are extremely
useful to us. Those systems also provide an interesting testing ground
where we can expose changes like this to a wide variety of source code.

I found this last part doesn't hold true, dolt does not fall back to
libtool transparently, the reason being the AC_SUBST causes LTCOMPILE to
become , even if that section of the if block isn't called. I couldn't
find a nice way to fix this since AC_SUBST operates at reautoconf time,
not at configure and you only know if the system is compatible at
configure time.

Poky/OE use an old libtool (1.5.10) since we have various hacks we had
to make to get libtool to support cross compiling sanely[1] and
everytime we've tried to upgrade, something goes wrong and we've never
had time to debug the newer versions.

[1] Are there any plans to support sysroots with libtool?

In the interests of experimentation I hacked dolt into our libtool
recipe, inserting it into libtool.m4 in AC_PROG_LIBTOOL so I didn't have
to change any apps. I enabled it for arm linux targets and removed the
broken fallback code. The test build I used as a benchmark builds a
cross compiling toolchain and then from that a complete PDA style system
including X and some GTK+ apps with all the parts in between, many of
which use libtool.

Before the dolt change this image took 108 minutes, afterwards it took
96 minutes so an 11% reduction in time which is certainly beneficial!

Cheers,

Richard



___
http://lists.gnu.org/mailman/listinfo/libtool

Re: sysroot support in libtool

[Richard Purdie]

Hi,

On Sun, 2008-04-13 at 08:21 -0400, Gary V. Vaughan wrote:
 On 13 Apr 2008, at 07:55, Richard Purdie wrote:
  [1] Are there any plans to support sysroots with libtool?
 
 No one is sending us bug reports or patches... so we don't even know
 there is a problem!

Ok, there are some fairly long running issues with OpenEmbedded and
libtool. I appreciate there has not been feedback to upstream about the
problems and this is entirely our fault. There are various reasons for
this:

* the person who integrated libtool into OE has moved onto other things
  and the knowledge for a lot of the magic was lost
* we're stuck on an old version of libtool (1.5.10) which we know you 
  won't be interested in bugs against
* this old version works and nobody has found the time for the 
  learning curve and debugging involved in upgrading

More recently a lot of things have changed for the better in OE and
we're striving for a clean logical build structure rather than hacks,
I've been one of the people trying to achieve this. One way we've done
this is embracing the sysroot options in gcc/binutils and getting
sysroot support into pkg-config.

So this leaves us with the problem of libtool. Why a problem?

Lets give an example of compiling some library which uses libtool and
autotools:

* We configure it with libdir=/usr/lib which is where the library will
be on the target system ultimately.
* Compile it using a gcc cross compiler and binutils which have been
configured with /somewhere/sysroot as the sysroot. This means they don't
look in /usr/lib and /usr/include, they look
in /somewhere/sysroot/usr/lib and /somewhere/sysroot/usr/include.
* We make install DESTDIR=/someplace and then build packages out
of /someplace
* We make install DESTDIR=/somewhere/sysroot which adds the library to
our sysroot environment so something using this lib can compile against
it.

The problem is this last step which installs .la files into our sysroot
which don't work. The issues we can have:

* The .la files say installed=yes but they're in the sysroot, not in
libdir.
* dependency_libs= can contain an expanded version of libdir. This is
wrong and we have to prepend the sysroot to this.
* libdir itself can be added to the search path for libraries which
finds things from /usr/lib on the host system - really bad when cross
compiling

To workaround these issues we sed the sysroot .la files to:

* change installed=yes to installed=no, it changes some of the logic
within libtool and breaks some assumptions about the files being in
libdir which helps.
* prepend sysroot to the appropriate parts of dependency_libs

and patch libtool to:

* remove the parts which add libdir into the search path
* cope with the installed=no logic and search both the sysroot and the
app being compiled

The above is the situation with 1.5.10. I've just tried a build with a
clean 2.2.2 and it broke with at least one of the problems mentioned
above, I don't know if all the above issues are still present but I
suspect they are meaning I need to forward port our hacks.

Whats the dream solution? We can set an environmental variable say
LIBTOOL_SYSROOT=/somewhere/sysroot and libtool would take this into
account exactly where needed and just work. The .la files in our
sysroot would match those we install onto the target system.

Its possible things have changed and libtool has some mechanism to cope
with setups like this and if so please let me know what they are! If not
does the above illustrate the problem and is this something you'd be
prepared to help fix?

In the meantime I will try and get Poky/OE using a much more recent
libtool, patched enough so builds continue to work as described above.
Once we get a modern version working it will be much easier to test
patches and improvements in libtool and from what I've read, we should
see some performance improvements which would be most welcome. We don't
have patches to add sysroot support, just the sed hacks I've mentioned
but if you'd be interested in proper sysroot support that is something
we'd be interested in collaborating on.

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool

Re: sysroot support in libtool

[Richard Purdie]

Hi Ralf,

On Sun, 2008-04-13 at 17:09 +0200, Ralf Wildenhues wrote:
 * Richard Purdie wrote on Sun, Apr 13, 2008 at 04:59:26PM CEST:
  
  * the person who integrated libtool into OE has moved onto other things
and the knowledge for a lot of the magic was lost
  * we're stuck on an old version of libtool (1.5.10) which we know you 
won't be interested in bugs against
 
  * this old version works [...]
 
 No it doesn't.  If it did, you wouldn't be adopting dolt.
 OK, you put works in quotes, so I guess that's what you meant.

works means that with our patched 1.5.10 libtool OE/Poky can generate
working cross compiled binaries and images. Its in quotes since I don't
like the way we do it but having working builds with some ugliness
behind the scenes is better than ones that don't work!

dolt comes in purely as a speedup, it doesn't make the .la file and
sysroot situation better or worse.

  In the meantime I will try and get Poky/OE using a much more recent
  libtool, patched enough so builds continue to work as described above.
  Once we get a modern version working it will be much easier to test
  patches and improvements in libtool and from what I've read, we should
  see some performance improvements which would be most welcome. We don't
  have patches to add sysroot support, just the sed hacks I've mentioned
  but if you'd be interested in proper sysroot support that is something
  we'd be interested in collaborating on.
 
 Even posting the sed hacks or the diffs against 1.5.10 that you were
 using (both to libtool-patches, please) are better than nothing.

I'll post here first, since I don't class any of these are ready to
apply and really need discussion. If you still feel libtool-patches is
more appropriate for that tell me and I'll switch lists though.

The patches we're using are publicly available as:
http://svn.o-hand.com/view/poky/trunk/meta/packages/libtool/libtool-1.5.10/

I've run through each with a quick description of what it does

3figures.patch - a better version was merged, not needed with 2.2.2

add_dolt.patch - hacks dolt into libtool.m4 instead of patching every
configure.ac. Inappropriate for upstream, mentioned for reference only.

autotools.patch - Make the old libtool work with more modern autotools.
Not needed with 2.2.2.

install-path-check.patch - OE used to have a staging layout which didn't
match the target system. Since we use sysroot this shouldn't be needed
anymore.

libdir-la.patch - See comments in the patch. Doesn't add libdir to the
search path, don't complain about 'moved' files, don't replace
uninstalled with installed libraries

libdir-la2.patch - Improves on the above to stop libdir leaking into
search paths and checking more locations in the installed=no case.

nmedit_fix.patch - Call host-triplet-nmedit, not just nmedit

nousrlib.patch - Stop libdir leaking into linker flags

prefix.patch - Rename libtool to be prefixed by the host triplet which
makes it much more obvious when the wrong libtool is being used and is
in line with autotools wanting cross tool names prefixed with the
triplet.

sedvar.patch - Fixes some issue we saw when $SED wasn't set

tag.patch - The tag errors were breaking things for no good reason so we
turned the error into a warning

uclibc.patch - Tweaks to libtool.m4 to support uclibc


I don't have the full background on all of these and have only checked
whether some are needed against 2.2.2, I'll aim to do the rest soon and
try to get 2.2.2 working. The core patches are the libdir-la.patch,
lidir-la2.patch and prefix.patch, all of which I know are still needed.

The sed magic I mentioned is in
http://svn.o-hand.com/view/poky/trunk/meta/classes/base.bbclass?rev=4064view=markup

specifically:

sed -e 's/^installed=yes$/installed=no/' \
 -e 
'/^dependency_libs=/s,${WORKDIR}[[:alnum:]/\._+-]*/\([[:alnum:]\._+-]*\),${STAGING_LIBDIR}/\1,g'
 \
 -e /^dependency_libs=/s,\([[:space:]']+\)${libdir},\1${STAGING_LIBDIR},g 
\
 $dotlai $destpath/$libname.la

which changes installed=yes to installed=no and cleans up
dependency_libs so it only refers to 'staging' (the sysroot), not the
work directories (which are temporary in nature) or libdir (which are
binaries from the wrong architecture).

 Proper sysroot support in Libtool would be very welcome, but
 unfortunately it won't be easy.  Of course there are more constraints:
 it should work right in most practical cases, not just the ones your
 package cares about.  Since it cannot be made to work on some systems
 due to linker and binary file format limitations, it should degrade
 gracefully on those systems.
 
 I've tried it a couple of years ago, but gave up since I figured it
 would take me all remaining weekends of the year to do.

I agree it won't be easy. If I knew it stood a chance of being accepted
into libtool and/or that others were prepared to help with it, that
might encourage me to start writing some patches though. It has the
advantage of being opt in since if sysroot

Re: sysroot support in libtool

[Richard Purdie]

On Sun, 2008-04-13 at 22:26 +0200, Ralf Wildenhues wrote:
 * Richard Purdie wrote on Sun, Apr 13, 2008 at 08:53:10PM CEST:
  
  The patches we're using are publicly available as:
  http://svn.o-hand.com/view/poky/trunk/meta/packages/libtool/libtool-1.5.10/
 
 Let's take a look at the simple ones:
 
  nmedit_fix.patch - Call host-triplet-nmedit, not just nmedit
 
 This patch shouldn't be needed any more with 2.2.2.

Agreed, the nmedit code looks much neater now!

  sedvar.patch - Fixes some issue we saw when $SED wasn't set
 
 Shouldn't be needed any more either.

Agreed.

  tag.patch - The tag errors were breaking things for no good reason so we
  turned the error into a warning
 
 Well, don't come crying for bad performance to us if you do things like
 this.  Letting libtool infer the tag is rather expensive, at least
 relatively, for 2.2.2 on modern systems where there are few other forks
 needed.

Going totally from memory, I think this exists since we have CC set to
things like:

CC=gcc -march=armv5te -mtune=xscale

and then this might change to

CC=gcc -march=armv5te -mtune=arm926ejs

which is 100% binary compatible, just optimised differently. libtool
upon seeing CC change gets upset. Sadly I can't remember how libtool
notices this change, it much be through some kind of state information
libtool saves into our staging area.

Thanks for the pointer on this though, I will look into it and find some
way to avoid it. 

  uclibc.patch - Tweaks to libtool.m4 to support uclibc
 
 If this happens to be needed for 2.2.2 as well, I'd like this one
 submitted to libtool-patches, ideally including a ChangeLog entry
 and testsuite results for this system, thanks.

Ok, I'll keep this in mind when updating and submit it if its still
needed.

Just for reference,
http://svn.o-hand.com/view/poky/trunk/meta/packages/libtool/libtool-2.2.2/
are the needed patches updated for 2.2.2, albeit rather untested at the
moment.

Cheers,

Richard




___
http://lists.gnu.org/mailman/listinfo/libtool