Re: [PATCH] Build AIX shared libraries with binutils 2.19.50+

[Richard Sandiford]

Ralf Wildenhues  writes:
> sorry for the long delay.  Apparently the 4.3.3 system has flaky
> hardware; anyway I don't trust its results any more.
>
> I've now pushed your two proposed patches, added a small NEWS entry to
> the first, and added you to THANKS.

Great!  Thanks for all your help.  And sorry for the long time in
replying, have been away for the last 2 weeks.

Richard

Re: [PATCH] Build AIX shared libraries with binutils 2.19.50+

[Ralf Wildenhues]

Hello Richard,

sorry for the long delay.  Apparently the 4.3.3 system has flaky
hardware; anyway I don't trust its results any more.

I've now pushed your two proposed patches, added a small NEWS entry to
the first, and added you to THANKS.

Thanks again!
Ralf

2009-06-06  Richard Sandiford  

Build AIX shared libraries with binutils 2.19.50+.
* libltdl/m4/libtool.m4 (_LT_LINKER_SHLIBS, _LT_LANG_CXX_CONFIG)
: New variable to control whether the
GNU ld or native ld interface is used.  Set to "no" for GNU ld
2.19.50+ on AIX, otherwise mirror $with_gnu_ld.  Update the
warning message that is printed when using GNU ld on AIX.
Adjust the whole_archive_flag_spec value for GNU ld on AIX.
* NEWS, THANKS: Update.

diff --git a/NEWS b/NEWS
index 42d3c69..3656acd 100644
--- a/NEWS
+++ b/NEWS
@@ -20,6 +20,7 @@ New in 2.2.8 2009-??-??: git version 2.2.7a, Libtool team:
   - Support for GNU/kOpenSolaris (kopensolaris*-gnu).
   - Initial support for compilers on BlueGene BG/L.
   - Improved support for Atari FreeMiNT.
+  - With binutils 2.19.50+, shared libraries can be built on AIX.
 
 * Bug fixes:
 
diff --git a/libltdl/m4/libtool.m4 b/libltdl/m4/libtool.m4
index 3596384..21db0b0 100644
--- a/libltdl/m4/libtool.m4
+++ b/libltdl/m4/libtool.m4
@@ -4208,7 +4208,33 @@ dnl Note also adjust exclude_expsyms for C++ above.
   esac
 
   _LT_TAGVAR(ld_shlibs, $1)=yes
+
+  # On some targets, GNU ld is compatible enough with the native linker
+  # that we're better off using the native interface for both.
+  lt_use_gnu_ld_interface=no
   if test "$with_gnu_ld" = yes; then
+case $host_os in
+  aix*)
+   # The AIX port of GNU ld has always aspired to compatibility
+   # with the native linker.  However, as the warning in the GNU ld
+   # block says, versions before 2.19.5* couldn't really create working
+   # shared libraries, regardless of the interface used.
+   case `$LD -v 2>&1` in
+ *\ \(GNU\ Binutils\)\ 2.19.5*) ;;
+ *\ \(GNU\ Binutils\)\ 2.[[2-9]]*) ;;
+ *\ \(GNU\ Binutils\)\ [[3-9]]*) ;;
+ *)
+   lt_use_gnu_ld_interface=yes
+   ;;
+   esac
+   ;;
+  *)
+   lt_use_gnu_ld_interface=yes
+   ;;
+esac
+  fi
+
+  if test "$lt_use_gnu_ld_interface" = yes; then
 # If archive_cmds runs LD, not CC, wlarc should be empty
 wlarc='${wl}'
 
@@ -4241,11 +4267,12 @@ dnl Note also adjust exclude_expsyms for C++ above.
_LT_TAGVAR(ld_shlibs, $1)=no
cat <<_LT_EOF 1>&2
 
-*** Warning: the GNU linker, at least up to release 2.9.1, is reported
+*** Warning: the GNU linker, at least up to release 2.19, is reported
 *** to be unable to reliably create shared libraries on AIX.
 *** Therefore, libtool is disabling shared libraries support.  If you
-*** really care for shared libraries, you may want to modify your PATH
-*** so that a non-GNU linker is found, and then restart.
+*** really care for shared libraries, you may want to install binutils
+*** 2.20 or above, or modify your PATH so that a non-GNU linker is found.
+*** You will then need to restart the configuration process.
 
 _LT_EOF
   fi
@@ -4605,8 +4632,13 @@ _LT_EOF
  # -berok will link without error, but may produce a broken library.
  _LT_TAGVAR(no_undefined_flag, $1)=' ${wl}-bernotok'
  _LT_TAGVAR(allow_undefined_flag, $1)=' ${wl}-berok'
- # Exported symbols can be pulled into shared objects from archives
- _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience'
+ if test "$with_gnu_ld" = yes; then
+   # We only use this code for GNU lds that support --whole-archive.
+   _LT_TAGVAR(whole_archive_flag_spec, 
$1)='${wl}--whole-archive$convenience ${wl}--no-whole-archive'
+ else
+   # Exported symbols can be pulled into shared objects from archives
+   _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience'
+ fi
  _LT_TAGVAR(archive_cmds_need_lc, $1)=yes
  # This is similar to how AIX traditionally builds its shared 
libraries.
  _LT_TAGVAR(archive_expsym_cmds, $1)="\$CC $shared_flag"' -o 
$output_objdir/$soname $libobjs $deplibs ${wl}-bnoentry $compiler_flags 
${wl}-bE:$export_symbols${allow_undefined_flag}~$AR $AR_FLAGS 
$output_objdir/$libname$release.a $output_objdir/$soname'
@@ -5546,8 +5578,13 @@ if test "$_lt_caught_CXX_error" != yes; then
# -berok will link without error, but may produce a broken library.
_LT_TAGVAR(no_undefined_flag, $1)=' ${wl}-bernotok'
_LT_TAGVAR(allow_undefined_flag, $1)=' ${wl}-berok'
-   # Exported symbols can be pulled into shared objects from archives
-   _LT_TAGVAR(whole_archive_flag_spec, $1)='$convenience'
+   if test "$with_gnu_ld" = yes; then
+ # We only use this code for GNU lds that support --whole-archive.
+ _LT_TAGVAR(whole_archive_flag_spec, 
$1)='${wl}--whole

Re: [PATCH] Build AIX shared libraries with binutils 2.19.50+

[Richard Sandiford]

Hi Ralf,

Thanks for the review, and sorry for the long time it's taken to reply.

Ralf Wildenhues  writes:
> * Richard Sandiford wrote on Mon, Mar 16, 2009 at 10:49:17AM CET:
>> If you try to configure libtool to build shared libraries on AIX,
>> it prints the following message when using GNU ld:
>> 
>> *** Warning: the GNU linker, at least up to release 2.9.1, is reported
>> *** to be unable to reliably create shared libraries on AIX.
>> *** Therefore, libtool is disabling shared libraries support.  If you
>> *** really care for shared libraries, you may want to modify your PATH
>> *** so that a non-GNU linker is found, and then restart.
>> 
>> binutils 2.19 doesn't work either, so the patch below updates the
>> version number.  However, a series of AIX patches were applied to
>> binutils CVS this weekend:
>> 
>>   http://sourceware.org/ml/binutils/2009-03/msg00172.html
>
> For those interested in reading the whole thread, this is:
> 
>
> Do they target AIX 6 only, or earlier versions as well?
> Which versions were they tested with?

They effectively target all versions, in that almost all the patches
fix generic AIX problems rather than AIX-6-specific problems.  Only one
patch was actually specific to AIX 6, and it simply added support for
the target triplet.  Binutils doesn't implement any features that
are specific to AIX 6.

The patches were tested on AIX 6.  I don't have access to other versions.

> We have users going back to 4.3.3 IIRC.  If the new support doesn't go
> back all that far, we should only enable it for as far back as is safe.

Understood.  It's probably best if I leave that call up to you.  On the
one hand, I think there's a good chance that the binutils patches will
work on AIX 5.2+, since like I say, binutils doesn't really implement
any features specific to AIX 6 per se.  If that turns out not to be
true for any reason, I'd be happy to look at the bug reports.  It might
therefore seem overly hash to stop AIX 5.2 users from even trying to use
binutils to build shared libraries.

  (The same offer extends to 4.3.3 btw, but the impact of the change
  is harder to predict there, since the OS doesn't have weak support.)

On the other hand, I suppose it will take a long time for this libtool
change to filter to some packages anyway, so in practice, anyone interested
in using binutils with AIX will have to use a locally-updated libtool.
They could easily modify the libtool patch to do what they want.

>> One fix would be to add "real" GNU ld support if the linker is
>> new enough.  However, GNU ld tries to be command-line compatible
>> with the native AIX linker, so the handling for native AIX linkers
>> works for binutils too.  I've therefore added a new variable that
>> controls whether the GNU or native interface should be used.
>
>> Tested using GCC.
>
> I'm not sure I understand correctly.  Did you run git Libtool's
> testsuite, after configuring it to use GCC and binutils?  In case not,
> please see the README file for how to run the test suites and report
> failures.

As you guessed, I hadn't done this, sorry.  All my testing at that
point had been using a cross GCC toolchain rather than a native one.
So it was tested using the GCC testsuite, but not the libtool testsuite.

I've now tested a native toolchain against both the gcc and libtool
testsuites.  The libtool testsuite did uncover other latent binutils
problems that still needed to be fixed, so thanks for that.  All the
fixes have now been applied to binutils CVS

> One thing that is missing from this patch is taking into account the
> non-whole-archive-like behavior of GNU ld on AIX.  I think it should
> cause some testsuite failures.  The setting of
> _LT_TAGVAR(whole_archive_flag_spec, TAG) may need adjustment (at least
> './libtool --config' output should differ in this respect from that with
> native ld).

Good catch, thanks.  Like you say, the libtool testsuite did show this.
(I deliberately ran it once without the fix as a smoke test.)

With this updated patch, the libtool results for GCC+binutils
are the same as those for the pre-installed GCC+native tools.

>>  * libltdl/m4/libtool.m4 (use_gnu_ld_interface): New variable
>>  to control whether the GNU ld or native ld interface is used.
>>  Set to "no" for GNU ld 2.19.50+ on AIX, otherwise mirror
>>  $with_gnu_ld.  Update the warning message that is printed
>>  when using GNU ld on AIX.
>
> Technical nit: use_gnu_ld_interface should probably be in the lt_*
> namespace.

Oops, done.

I've attached the updated patch.

Richard


* libltdl/m4/libtool.m4 (lt_use_gnu_ld_interface): New variable
to control whether the GNU ld or native ld interface is used.
Set to "no" for GNU ld 2.19.50+ on AIX, otherwise mirror
$with_gnu_ld.  Update the warning message that is printed
when using GNU ld on AIX.  Adjust the whole_archive_flag_spec
value for GNU ld on AIX

Re: [PATCH] Build AIX shared libraries with binutils 2.19.50+

[Ralf Wildenhues]

Hello Richard,

thanks for the patch.

* Richard Sandiford wrote on Mon, Mar 16, 2009 at 10:49:17AM CET:
> If you try to configure libtool to build shared libraries on AIX,
> it prints the following message when using GNU ld:
> 
> *** Warning: the GNU linker, at least up to release 2.9.1, is reported
> *** to be unable to reliably create shared libraries on AIX.
> *** Therefore, libtool is disabling shared libraries support.  If you
> *** really care for shared libraries, you may want to modify your PATH
> *** so that a non-GNU linker is found, and then restart.
> 
> binutils 2.19 doesn't work either, so the patch below updates the
> version number.  However, a series of AIX patches were applied to
> binutils CVS this weekend:
> 
>   http://sourceware.org/ml/binutils/2009-03/msg00172.html

For those interested in reading the whole thread, this is:


Do they target AIX 6 only, or earlier versions as well?
Which versions were they tested with?

We have users going back to 4.3.3 IIRC.  If the new support doesn't go
back all that far, we should only enable it for as far back as is safe.

> These patches are enough to build working GCC shared libraries
> (shared libgcc, libstdc++, libgfortran, libobjc, etc.).  We've also
> used them internally to build proprietary shared libraries.
> 
> Hopefully things are in good enough shape that any failures can be
> treated like any other binutils bug.  I'd therefore like to remove
> the restriction above for binutils 2.19.50+.

Sounds like a good idea.

> One fix would be to add "real" GNU ld support if the linker is
> new enough.  However, GNU ld tries to be command-line compatible
> with the native AIX linker, so the handling for native AIX linkers
> works for binutils too.  I've therefore added a new variable that
> controls whether the GNU or native interface should be used.

> Tested using GCC.

I'm not sure I understand correctly.  Did you run git Libtool's
testsuite, after configuring it to use GCC and binutils?  In case not,
please see the README file for how to run the test suites and report
failures.

One thing that is missing from this patch is taking into account the
non-whole-archive-like behavior of GNU ld on AIX.  I think it should
cause some testsuite failures.  The setting of
_LT_TAGVAR(whole_archive_flag_spec, TAG) may need adjustment (at least
'./libtool --config' output should differ in this respect from that with
native ld).

>   * libltdl/m4/libtool.m4 (use_gnu_ld_interface): New variable
>   to control whether the GNU ld or native ld interface is used.
>   Set to "no" for GNU ld 2.19.50+ on AIX, otherwise mirror
>   $with_gnu_ld.  Update the warning message that is printed
>   when using GNU ld on AIX.

Technical nit: use_gnu_ld_interface should probably be in the lt_*
namespace.

Cheers, and thanks again,
Ralf

[PATCH] Build AIX shared libraries with binutils 2.19.50+

[Richard Sandiford]

Hi,

If you try to configure libtool to build shared libraries on AIX,
it prints the following message when using GNU ld:

*** Warning: the GNU linker, at least up to release 2.9.1, is reported
*** to be unable to reliably create shared libraries on AIX.
*** Therefore, libtool is disabling shared libraries support.  If you
*** really care for shared libraries, you may want to modify your PATH
*** so that a non-GNU linker is found, and then restart.

binutils 2.19 doesn't work either, so the patch below updates the
version number.  However, a series of AIX patches were applied to
binutils CVS this weekend:

  http://sourceware.org/ml/binutils/2009-03/msg00172.html

These patches are enough to build working GCC shared libraries
(shared libgcc, libstdc++, libgfortran, libobjc, etc.).  We've also
used them internally to build proprietary shared libraries.

Hopefully things are in good enough shape that any failures can be
treated like any other binutils bug.  I'd therefore like to remove
the restriction above for binutils 2.19.50+.

One fix would be to add "real" GNU ld support if the linker is
new enough.  However, GNU ld tries to be command-line compatible
with the native AIX linker, so the handling for native AIX linkers
works for binutils too.  I've therefore added a new variable that
controls whether the GNU or native interface should be used.

Tested using GCC.  Does the patch look OK?  Please install if so.

Thanks,
Richard

PS. As far as copyright goes, this work is covered by the IBM assignment.


* libltdl/m4/libtool.m4 (use_gnu_ld_interface): New variable
to control whether the GNU ld or native ld interface is used.
Set to "no" for GNU ld 2.19.50+ on AIX, otherwise mirror
$with_gnu_ld.  Update the warning message that is printed
when using GNU ld on AIX.

Index: libltdl/m4/libtool.m4
===
--- libltdl/m4/libtool.m4   2009-03-16 09:15:55.0 +
+++ libltdl/m4/libtool.m4   2009-03-16 09:24:42.0 +
@@ -4192,7 +4192,33 @@ dnl Note also adjust exclude_expsyms for
   esac
 
   _LT_TAGVAR(ld_shlibs, $1)=yes
+
+  # On some targets, GNU ld is compatible enough with the native linker
+  # that we're better off using the native interface for both.
+  use_gnu_ld_interface=no
   if test "$with_gnu_ld" = yes; then
+case $host_os in
+  aix*)
+   # The AIX port of GNU ld has always aspired to compatibility
+   # with the native linker.  However, as the warning in the GNU ld
+   # block says, versions before 2.19.5* couldn't really create working
+   # shared libraries, regardless of the interface used.
+   case `$LD -v 2>&1` in
+ *\ \(GNU\ Binutils\)\ 2.19.5*) ;;
+ *\ \(GNU\ Binutils\)\ 2.[[2-9]]*) ;;
+ *\ \(GNU\ Binutils\)\ [[3-9]]*) ;;
+ *)
+   use_gnu_ld_interface=yes
+   ;;
+   esac
+   ;;
+  *)
+   use_gnu_ld_interface=yes
+   ;;
+esac
+  fi
+
+  if test "$use_gnu_ld_interface" = yes; then
 # If archive_cmds runs LD, not CC, wlarc should be empty
 wlarc='${wl}'
 
@@ -4225,11 +4251,12 @@ dnl Note also adjust exclude_expsyms for
_LT_TAGVAR(ld_shlibs, $1)=no
cat <<_LT_EOF 1>&2
 
-*** Warning: the GNU linker, at least up to release 2.9.1, is reported
+*** Warning: the GNU linker, at least up to release 2.19, is reported
 *** to be unable to reliably create shared libraries on AIX.
 *** Therefore, libtool is disabling shared libraries support.  If you
-*** really care for shared libraries, you may want to modify your PATH
-*** so that a non-GNU linker is found, and then restart.
+*** really care for shared libraries, you may want to install binutils
+*** 2.20 or above, or modify your PATH so that a non-GNU linker is found.
+*** You will then need to restart the configuration process.
 
 _LT_EOF
   fi