Re: preparing for 1.5.24

2007-04-25 Thread Ralf Wildenhues
Hi Charles,

* Charles Wilson wrote on Wed, Apr 25, 2007 at 05:38:49AM CEST:
 Ralf Wildenhues wrote:

 Thanks also for the documentation suggestion.  Slightly rewritten
 suggestion to come up.

 Ping? [antecedent reposted below]

Sigh.  I had hoped to have enough time to work on this.  Alas, I better
just post my thoughts.  Apologies for being the eye of the needle here
(can you use the phrase this way in English?).

First, I think this is a bit ad hoc here: the dllexport/dllimport issue,
or more generally, w32-specific issues (although as I understand it's
not /strictly/ w32-specific), don't start with using libltdl.  As such,
I think they deserves its own chapter in the manual, and then the
libltdl handling can refer to it, in shortened form.  Ideally, all the
gory details of how you can have several deplibs, some static some
shared, and currently need to define a macro for each based on that (but
maybe libtool can help there in the future to automatize that?).
AC_LIBTOOL_WIN32_DLL could refer to this section then.

I think some of this stuff has been discussed at length on the libtool
lists; I assume you were one of the people involved.

Even cooler would be to take into account the differences for when we
add support for MSVC, wgcc, ...

 Around line 3546 [probably 4476, now] in libtool.texi, something like:

 --%
 @emph{Win32 specific}

FWIW, I don't like this way of putting keywords much.  It should be a
heading, or part of the surrounding sentence.

 When building a DLL on windows (MinGW,Cygwin) that 

Let's not speak of a library as a DLL, except where we explain what a
DLL is, and except the strictly w32-specific parts of the manual.  We
should strive to use coherent names for things, here: shared library.
Just like you use convenience archive within the same sentence.

 uses @code{libltdl} as a convenience archive, there are some issues with 
 symbol decoration that must be carefully managed.  You must do one of the 
 following, within the code and/or build machinery of your library:

 @enumerate 1
 @item explicitly mark as @code{declspec(dllexport)} all symbols in your 
 library that should be exported when building your library's source code.  
 However, these symbols should @emph{not} be marked 
 @code{declspec(dllexport)} when building @emph{clients} of your library -- 
 instead, the symbols should be marked @code{declspec(dllimport)}. This 
 procedure will require careful macro design within your library's public 
 header files.  An example of this may be found in @file{zlib.h} from the 
 zlib distribution @url{http://www.zlib.net/}.

Let's rather not refer to an external source here: they could change,
the link is more than two links away from what the user really needs to
see when doing this, and zlib isn't GNU software (it's free, so there's
no real issue here).

Rather, a small, self-contained example would be great.  Ideally, that
example would be identical to a test in the new testsuite, so that we're
kept honest in that it actually works as expected.

 @item use the (win32-specific) linker option @code{-Xlinker 
 --export-all-symbols}.  This will restore the expected auto-export 
 behavior of the GNU linker on windows.  However, the 
 @code{--export-all-symbols} option is specific to the GNU linker's i386 PE 
 target -- @code{ld} for other targets will not recognize the option.  
 Therefore, using this option will require careful coding in your library's 
 @file{Makefile.am} and @file{configure.ac}.  For example:

 configure.ac:
 @example
 ...
 case $host_os in
 cygwin* | mingw* | pw32*)

This doesn't account for specifity to GNU binutils ld.  $with_gnu_ld
should help here, bugs in its setting notwithstanding.

   RESTORE_AUTOEXPORT=-Xlinker --export-all-symbols
[...]

Cheers, and thanks for all your work,
Ralf




Re: preparing for 1.5.24

2007-04-24 Thread Charles Wilson

Ralf Wildenhues wrote:


Thanks also for the documentation suggestion.  Slightly rewritten
suggestion to come up.


Ping? [antecedent reposted below]

--
Chuck

Around line 3546 [probably 4476, now] in libtool.texi, something like:

--%
@emph{Win32 specific} When building a DLL on windows (MinGW,Cygwin) that 
uses @code{libltdl} as a convenience archive, there are some issues with 
symbol decoration that must be carefully managed.  You must do one of 
the following, within the code and/or build machinery of your library:


@enumerate 1
@item explicitly mark as @code{declspec(dllexport)} all symbols in your 
library that should be exported when building your library's source 
code.  However, these symbols should @emph{not} be marked 
@code{declspec(dllexport)} when building @emph{clients} of your library 
-- instead, the symbols should be marked @code{declspec(dllimport)}. 
This procedure will require careful macro design within your library's 
public header files.  An example of this may be found in @file{zlib.h} 
from the zlib distribution @url{http://www.zlib.net/}.


@item use the (win32-specific) linker option @code{-Xlinker 
--export-all-symbols}.  This will restore the expected auto-export 
behavior of the GNU linker on windows.  However, the 
@code{--export-all-symbols} option is specific to the GNU linker's i386 
PE target -- @code{ld} for other targets will not recognize the option. 
 Therefore, using this option will require careful coding in your 
library's @file{Makefile.am} and @file{configure.ac}.  For example:


configure.ac:
@example
...
case $host_os in
cygwin* | mingw* | pw32*)
  RESTORE_AUTOEXPORT=-Xlinker --export-all-symbols
  ;;
*)
  RESTORE_AUTOEXPORT=
  ;;
esac
AC_SUBST(RESTORE_AUTOEXPORT)
...
@end example

Makefile.am
@example
...
libmylibrary_la_LDFLAGS = -no-undefined $(RESTORE_AUTOEXPORT)
...
@end example

@item Use libtool facilities to mimic @code{--export-all-symbols}. 
Unlike the solution above, this solution has the advantage of requiring 
no platform-specific workarounds.  Simply append 
@code{-export-symbols-regex .*} to the @code{_LDFLAGS} for your library:


Makefile.am
@example
...
libmylibrary_la_LDFLAGS = -no-undefined -export-symbols-regex .*
...
@end example
@end enumerate
--%




Re: preparing for 1.5.24

2007-02-18 Thread Ralf Wildenhues
Hello Charles, 


Charles Wilson writes:



- It would help me greatly if someone could look into the Cygwin and
  MinGW mdemo* failures; and documentation updates if needed.



Solution in this case is to turn auto-export back on


Or to mark all symbols as to be exported, no?
(Or to mark all other symbols explicitly.  But let's not go that way.) 

It'd be nice to be able to say whenever libltdl is used as a convenience 
library to build a DLL, on windows libtool should automatically turn on 
--export-all-symbols.  But that isn't true: if the target DLL is already 
doing explicit declspec(dllexport), then (1) the target library would 
work, today, without any changes, and (2) we don't want to override that 
target DLL's explicit export settings by saying, no, I'm going to export 
everything.


ACK. 

So, I'm not sure how to solve this, in a way that is platform-agnostic. 


I think the patch below should work, and be platform-agnostic.  WDYT?
OK for branch-1-5 and HEAD alike?  It fixes the similar failure there.
Do you think we should add some documentation bits a la if you have the
same problem, consider this solution as well? 


(FWIW, on HEAD there are other mdemo test failures which I haven't looked
at yet.) 

FWIW, it doesn't appear that libtool's -export-dynamic will do the right 
thing here.  I thought about setting export_dynamic_flag_spec to 
'${wl}--export-all-symbols' for mingw/cygwin (currently, they inherit the 
generic '${wl}--export-dynamic' setting [3]), and then using 
'-export-dynamic' in libmlib_la_LDFLAGS for all platforms.


I don't think -export-dynamic is intended to have the semantics of
--export-all-symbols. 

But I'm nervous about changing the definition of export_dynamic_flag_spec 
-- because that would have wide-ranging effects.  Esepcially in a release 
candidate, and especially especially in the RC for the last-ever 1.5.x 
libtool.


I guess I should have written the last 1.5.x release, unless serious
issues show up.  When writing, the intention was more of a after this,
we work towards 2.0 then anything else. 


[1] mdemo README:
Note that on Windows, for the purposes of illustrating ltdl, the
libraries *are* built as dll's, but do not have the __declspec
machinery to make them suitable for loading at link time.  This is
only for clarity inside this example, look at the example in ../demo
to see how the __declspec macros should be set up. 

[2] BTW, ../demo does not set up any __declspec macros. I believe the 
readme snippet above is really old, and pre-dates auto-export/auto-import 
functionality.  At best, this snippet is now specific only to non-gcc on 
Windows, which is at present only barely supported in libtool-1.5.x 
anyway.  Notwithstanding my reference to this snippet in the context of 
libltdl-as-convenience-lib, I think this snippet should be deleted 
entirely.


Indeed.  I guess for branch-1-5 it should be removed.  For HEAD and
eventual MSVC support, those would need to be added again. 

[3] I'm not sure, but it looks like cygwin's ld simply ignores 
--export-dynamic.  From a brief look at the bfd/ld source code, this ld 
option seems to be ELF-specific.


I guess.  I don't know what its effect on Cygwin should be though.
Not --export-all-symbols, that's not the right thing. 


Cheers, and thank you,
Ralf 


  * mdemo/Makefile.am (libmlib_la_LDFLAGS): Add -export-symbols-regex
  .* because the convenience libltdl uses export markers and thus
  turns off autoexport on w32.  Fixes test failures for Cygwin and
  MinGW.  Bug analysis by Charles Wilson. 



Index: mdemo/Makefile.am
===
RCS file: /sources/libtool/libtool/mdemo/Attic/Makefile.am,v
retrieving revision 1.46.2.2
diff -u -r1.46.2.2 Makefile.am
--- mdemo/Makefile.am   3 Feb 2006 09:37:34 -   1.46.2.2
+++ mdemo/Makefile.am   18 Feb 2007 12:07:21 -
@@ -19,9 +19,12 @@
 libsub_la_SOURCES = sub.c
 libsub_la_LDFLAGS = -no-undefined
 
+## Use -export-symbols-regex here explicitly because libltdl marks
+## its exported symbols, and we use libltdl as a convenience archive.
+## Thus, on w32, auto-exporting is turned off.
 libmlib_la_SOURCES = mlib.c
 libmlib_la_LIBADD = @LIBLTDL@ -dlopen foo1.la -dlopen libfoo2.la
-libmlib_la_LDFLAGS = -no-undefined
+libmlib_la_LDFLAGS = -no-undefined -export-symbols-regex .*
 libmlib_la_DEPENDENCIES = @LIBLTDL@ libsub.la foo1.la libfoo2.la
 
 noinst_HEADERS = foo.h


Re: preparing for 1.5.24

2007-02-18 Thread Charles Wilson

Ralf Wildenhues wrote:

Hello Charles,
Charles Wilson writes:



- It would help me greatly if someone could look into the Cygwin and
  MinGW mdemo* failures; and documentation updates if needed.



Solution in this case is to turn auto-export back on


Or to mark all symbols as to be exported, no?
(Or to mark all other symbols explicitly.  But let's not go that way.)


Yes, that should work, too.


I think the patch below should work, and be platform-agnostic.  WDYT?
OK for branch-1-5 and HEAD alike?  It fixes the similar failure there.


Hmm -- seems ok in principle.  I don't know if the end behavior is 
exactly the same, between ld's --export-all-symbols and libtool's 
-export-symbols-regex .*.  I've attached the exports list from my 
cygmlib-0.dll (taken from the output of objdump -p).  How does yours 
compare?



Do you think we should add some documentation bits a la if you have the
same problem, consider this solution as well?


Yes, I do.  Around line 3546 in libtool.texi, something like:

--%
@emph{Win32 specific} When building a DLL on windows (MinGW,Cygwin) that 
uses @code{libltdl} as a convenience archive, there are some issues with 
symbol decoration that must be carefully managed.  You must do one of 
the following, within the code and/or build machinery of your library:


@enumerate 1
@item explicitly mark as @code{declspec(dllexport)} all symbols in your 
library that should be exported when building your library's source 
code.  However, these symbols should @emph{not} be marked 
@code{declspec(dllexport)} when building @emph{clients} of your library 
-- instead, the symbols should be marked @code{declspec(dllimport)}. 
This procedure will require careful macro design within your library's 
public header files.  An example of this may be found in @file{zlib.h} 
from the zlib distribution @url{http://www.zlib.net/}.


@item use the (win32-specific) linker option @code{-Xlinker 
--export-all-symbols}.  This will restore the expected auto-export 
behavior of the GNU linker on windows.  However, the 
@code{--export-all-symbols} option is specific to the GNU linker's i386 
PE target -- @code{ld} for other targets will not recognize the option. 
 Therefore, using this option will require careful coding in your 
library's @file{Makefile.am} and @file{configure.ac}.  For example:


configure.ac:
@example
...
case $host_os in
cygwin* | mingw* | pw32*)
  RESTORE_AUTOEXPORT=-Xlinker --export-all-symbols
  ;;
*)
  RESTORE_AUTOEXPORT=
  ;;
esac
AC_SUBST(RESTORE_AUTOEXPORT)
...
@end example

Makefile.am
@example
...
libmylibrary_la_LDFLAGS = -no-undefined $(RESTORE_AUTOEXPORT)
...
@end example

@item Use libtool facilities to mimic @code{--export-all-symbols}. 
Unlike the solution above, this solution has the advantage of requiring 
no platform-specific workarounds.  Simply append 
@code{-export-symbols-regex .*} to the @code{_LDFLAGS} for your library:


Makefile.am
@example
...
libmylibrary_la_LDFLAGS = -no-undefined -export-symbols-regex .*
...
@end example
@end enumerate
--%


(FWIW, on HEAD there are other mdemo test failures which I haven't looked
at yet.)
FWIW, it doesn't appear that libtool's -export-dynamic will do the 
right thing here.  I thought about setting export_dynamic_flag_spec to 
'${wl}--export-all-symbols' for mingw/cygwin (currently, they inherit 
the generic '${wl}--export-dynamic' setting [3]), and then using 
'-export-dynamic' in libmlib_la_LDFLAGS for all platforms.


I don't think -export-dynamic is intended to have the semantics of
--export-all-symbols.


It seems to just delegate to ld's --export-dynamic, which from my 
reading of 'info ld' seems concerned mainly with the export of symbols 
from an executable. (kinda like --export-all-symbols, but .exe-specific, 
and doesn't really export ALL symbols, just most of them).  You're right 
in that we should not overload its behavior on win32 -- that way lies 
madness.



I guess I should have written the last 1.5.x release, unless serious
issues show up.  When writing, the intention was more of a after this,
we work towards 2.0 then anything else.


Heck yeah. I don't want to slow THAT down!



Indeed.  I guess for branch-1-5

[ the blurb in mdemo/README concerning win32 ]

should be removed.  For HEAD and
eventual MSVC support, those would need to be added again.


Right.

[3] I'm not sure, but it looks like cygwin's ld simply ignores 
--export-dynamic.  From a brief look at the bfd/ld source code, this 
ld option seems to be ELF-specific.


I guess.  I don't know what its effect on Cygwin should be though.
Not --export-all-symbols, that's not the right thing.


Ack. I think it should force a kind of --export-all-symbols, but only 
for .exe's.  Current ld behavior for .exe's is to NOT export any symbols 
(e.g. no auto-export) -- unless (a) there are symbols in the exe which 
were explicitly marked declspec(dllexport), or 

Re: preparing for 1.5.24

2007-02-17 Thread Peter O'Gorman


On Feb 17, 2007, at 6:54 PM, Ralf Wildenhues wrote:


Hi Peter, Tim, all,

* Peter O'Gorman wrote on Sun, Feb 11, 2007 at 01:20:24PM CET:

On Feb 11, 2007, at 7:24 PM, Ralf Wildenhues wrote:


Should we rather do a real prerelease, so it's archived on
alpha.gnu.org,[...]


This is good news, and great work. I'm pretty sure finch can handle
the load and bandwidth, so I think it is up to you, but I would not
bother with a formal alpha.


I've looked at the snapshot.  It uses Automake 1.9.6 and Autoconf
2.60.  Since I'd rather use 1.10 and 2.61 for 1.5.24, I preferred a
formal alpha.  I've now adjusted the URL for the branch-1-5 snapshot
to contain the string 1.5.23c in the name.  I assume the switch will
happen automatically with the next build?


Looks like I was on the ball the day I wrote that script, yeah, it  
handles the version change without modification.


Peter





Re: preparing for 1.5.24

2007-02-17 Thread Charles Wilson

Ralf Wildenhues wrote:

I would like to release 1.5.24 as soon as possible.  I've flushed my
queue now, except for the patch I just posted and whatever w32 changes
are still needed.



FWIW, I've done a test run of branch-1-5, here's some results:

PASS  SKIP  FAIL  config.guess  compiler used   notes

110 1 1   i686-pc-cygwingcc, g++, g77   fail: mdemo2-make


confirmed (1.5.23c, about two hours ago). I've diagnosed the reason and 
have a patch -- but I'm not sure if it should be applied.  I don't 
consider this a showstopper for 1.5.24, regardless of whether the 
attached patch is accepted or not.



- It would help me greatly if someone could look into the Cygwin and
  MinGW mdemo* failures; and documentation updates if needed.


I believe the mdemo2 failure on cygwin is another ripple from the ltdl 
declspec() change.  mlib uses libltdlc as convenience library, so when 
linking cygmlib-0.dll there exist symbols (lt_*) that are declared 
declspec(dllexport) -- so autoexport is turned off.  However, the 
symbols in mlib.c are NOT marked in that way (see mdemo/README snippet, 
[1]), so without autoexport, the output dll contains only lt_* symbols 
-- and mlib_func is missing.


Solution in this case is to turn auto-export back on (earlier 
suggestions conerning define LT_SCOPE before #inluding ltdl.h won't 
work, because we're linking in the ltdl.o object that was already built 
as PIC (==-DDLL_EXPORT), and LT_SCOPE has already ended up as 
declspec(dllexport) in that ltdl.o.


It'd be nice to be able to say whenever libltdl is used as a 
convenience library to build a DLL, on windows libtool should 
automatically turn on --export-all-symbols.  But that isn't true: if 
the target DLL is already doing explicit declspec(dllexport), then (1) 
the target library would work, today, without any changes, and (2) we 
don't want to override that target DLL's explicit export settings by 
saying, no, I'm going to export everything.


So, I'm not sure how to solve this, in a way that is platform-agnostic. 
(short of undoing the -DDLL_EXPORT change, like Ralf wants to.  But I 
won't rehash that argument, except to say that this problem would 
persist on mingw even if we fixed cygwin in that way)


A non-platform-agnostic mechanism could be described as:


When building a DLL on Windows (==MinGW,Cygwin) that uses libltdl as a 
convenience archive, you must do one of two things:

  (1) explicitly mark as declspec(dllexport) all symbols in _your_
  library that should be exported when building your library's
  source code -- but do NOT mark them dllexport when building
  clients; this procedure will require much macro magic in
  your library's public header files, or
  (2) use the (win32-specific) linker option
   -Xlinker --export-all-symbols
  to restore the expected auto-export behavior on windows.
  This will require some configure magic, as other platforms'
  linkers will not recognize this option. See mdemo/configure.ac
  and mdemo/Makefile.am for an example.


I used #2 in the attached patch, for the mdemo/mdemo2 issue.

FWIW, it doesn't appear that libtool's -export-dynamic will do the right 
thing here.  I thought about setting export_dynamic_flag_spec to 
'${wl}--export-all-symbols' for mingw/cygwin (currently, they inherit 
the generic '${wl}--export-dynamic' setting [3]), and then using 
'-export-dynamic' in libmlib_la_LDFLAGS for all platforms.


But I'm nervous about changing the definition of 
export_dynamic_flag_spec -- because that would have wide-ranging 
effects.  Esepcially in a release candidate, and especially especially 
in the RC for the last-ever 1.5.x libtool.



Anyway, with the attached patch, all tests pass on cygwin.

--
Chuck

[1] mdemo README:
Note that on Windows, for the purposes of illustrating ltdl, the
libraries *are* built as dll's, but do not have the __declspec
machinery to make them suitable for loading at link time.  This is
only for clarity inside this example, look at the example in ../demo
to see how the __declspec macros should be set up.

[2] BTW, ../demo does not set up any __declspec macros. I believe the 
readme snippet above is really old, and pre-dates 
auto-export/auto-import functionality.  At best, this snippet is now 
specific only to non-gcc on Windows, which is at present only barely 
supported in libtool-1.5.x anyway.  Notwithstanding my reference to this 
snippet in the context of libltdl-as-convenience-lib, I think this 
snippet should be deleted entirely.


[3] I'm not sure, but it looks like cygwin's ld simply ignores 
--export-dynamic.  From a brief look at the bfd/ld source code, this ld 
option seems to be ELF-specific.



2007-02-17  Charles Wilson  ...

* mdemo/configure.ac: add platform-specific