Ralf Wildenhues wrote:
Here's a review based on the question how such a concept could be
expressed portably. I don't see how something like this would be of
great help if it could not be made to gracefully decay into something
still usable, on at least a useful set of platforms if not all of them.
Agreed. While I was considering the problem, I kept getting the feeling
that either (1) I was missing something, or (2) this was very win32
specific: it's a problem only for platforms that require -no-undefined
for shared libraries, which I think is only win32 and AIX.
* Charles Wilson wrote on Sun, Mar 25, 2007 at 09:34:47PM CEST:
What I really want is a type of convenience library (call it a "resolver"
library?) that
(1) is built both PIC and non-PIC, depending of course on the value of
enable_shared and enable_static. So far, just like a "normal" convenience
lib.
Well, currently a convenience lib is not built in both ways at the same
time, but that's only a side issue for the discussion at hand.
Really? I thought I had both; perhaps it was just accumulated clutter
from multiple builds with different settings. In any case, I can see
where the current behavior you describe could be problematic on some
platforms, including win32, in certain cases.
(2) where the non-PIC resolver library is ignored when building dependent
static libraries (and dependent static [dlpreopened] modules). That is, NOT
exploded and included within the dependent. However, the non-PIC resolver
library should be used like a normal convenience library when building
dependent, in-package executables that depend on it directly
OK so far. You want a convenience archive without the whole-ness.
Should it be possible to install this thing or not?
Ummm...maybe? Let me ping back a few more questions: how would you
install/distinguish between the two versions (no-pic and pic)? Would we
need a naming convention of some sort and teach ld/gcc about it (a la'
.dll.a)
This could get ugly.
On the other hand, currently binutils/gcc/etc install libiberty.a -- but
do not install any headers for it. So the only was to use an
*installed* libiberty is to declare the relevant functions yourself (or
go dig around in the src for the headers), AND hope that whoever built
the libiberty.a you want to link against, built it in such a way that it
includes all the functions you want.
In short, *currently* libiberty is not well-handled. Since the intent
for that library is to share at the source level, a better policy would
be for binutils/gcc/etc to affirmatively change their build structure to
NOT install libiberty.a.
And nobody talks about installing a libgnulib.a -- especially as
gnulib-tool lets you change the library name...
So maybe the answer to the question SHOULD BE no, even if at present the
actual practice IS to install libiberty.a.
Can it itself have
nontrivial (i.e., non-convenience) dependencies?
Errr...the two extant examples (libiberty and gnulib) do not, but...
The first of these questions is really a crux of the matter: if libbfd
can be shared and installable, then really I don't see any way except
- either explicitly forbid any third party to use your library, even
though you install it shared,
- or have at least some sort of stable interface notion.
Yes. Since libiberty is explicitly unstable in its interface, I believe
it really makes no sense to install it; it's just of no use outside the
build tree.
Following this line of reasoning, I think my "resolver" library concept
is really just a variant of a convenience lib -- in the sense that
convenience libs are not usually (ever?) considered as installable objects.
(worry about
PIE here? We don't worry about it at present anywhere that I can see).
We have very minimal PIE support only currently in CVS HEAD, in compile
mode.
Don't know enough about this issue; I'll let it pass...
(3) where the PIC resolver library is used *like a static library* when
building dependent shared libraries -- that is, used to satisfy undefined
symbols in the shared library if -no-undefined, but where the objects in the
PIC resolver library are not included wholesale via
[ "not" correction included ]
--whole-archive/--no-whole-archive -- and better yet, on win32 they should
be excluded from auto-export using -Wl,--exclude-lib -Wl,<PIC resolver
library name>.
Lots of systems don't allow symbol hiding (or, at least, Libtool
currently doesn't support it well on most systems). What should happen
for them? More to the point: what if libbfd uses the resolver libiberty
release x, while libfoo used release y of libiberty, we're on a system
without symbol hiding, and my program links against both libbfd and
libfoo? The order of linkage will determine in which library the blow
up occurs, be that silent or spectacular, depending on how incompatible
x and y are.
But on how many systems are BOTH of the following true:
(1) to build shared requires -no-undefined
(2) but symbol hiding is unavailable
You know, thinking about this more, I guess (1b) from my previous email
is not THAT bad -- for shared libraries; just a little bloat of the
final library -- it requires a little platform-specific gobbledy-gook in
configure.ac/Makefile.am, but libbfd (and libobject, etc) already do that:
bfd/configure.in from gcc trunk:
-------------------------------------------------------------
# Horrible hacks to build DLLs on Windows.
WIN32LDFLAGS=
WIN32LIBADD=
case "${host}" in
*-*-cygwin*)
if test "$enable_shared" = "yes"; then
WIN32LDFLAGS="-no-undefined"
WIN32LIBADD="-L`pwd`/../libiberty -liberty -L`pwd`/../intl -lintl
-lcygwin -lkernel32"
fi
;;
*-*-linux*)
# We borrow WIN32LIBADD so that the shared libbfd won't depend on
# libiberty.a.
case "${host}" in
mips*-*-linux*)
# Linux/MIPS uses PIC by default.
if test "$enable_shared" = "yes"; then
WIN32LIBADD="-L../libiberty -liberty"
fi
;;
*)
changequote(,)dnl
x=`sed -n -e 's/^[ ]*PICFLAG[ ]*=[ ]*//p' <
../libiberty/Makefile | sed -n '$p'`
changequote([,])dnl
if test -n "$x"; then
WIN32LIBADD="-L../libiberty/pic -liberty"
fi
;;
esac
;;
esac
AC_SUBST(WIN32LDFLAGS)
AC_SUBST(WIN32LIBADD)
-------------------------------------------------------------
Now, there are things in the snippet above I'd do differently (like: no
need for cygwin and kernel32; shouldn't configure.ac just declare
conditionals, and the actual additions to _LIBADD/_LDFLAGS be done in
Makefile.am under the control of those conditionals; etc).
What's amusing though, is that what STARTED as a "horrible hack for
win32" got hijacked not just by *-*-linux, but also by EVERY other
platform in the default clause!
Anyway, this same machinery -- since it is there already -- can be used
to insert -Wl,--exclude-lib in the right place, for win32.
I'd still like to be able to build my convenience library as both pic
and non-pic tho. And I still want to prevent libiberty.a(non-pic) from
getting the --whole-archive treatment when it comes to libbfd.a.
Hmmm. More on that, below.
Also, these "static" libraries should not trigger the "you
can't create shared libs with static dependencies" filter within libtool.
Several systems simply don't allow to mix PIC and non-PIC symbols.
On w32 this warning is pathetic, but on others we would have a problem
here. (I can see why, on w32, you'd want to kill the warning here.)
No, you're missing the point of my statement: I *know* you don't want to
mix pic and non-pic. In fact, I'm sorta kinka trying to make win32
behave that way, too, at least in this case, as an 'inoculate against
the future' effort. (in case "true" -fpic support for win32 is merged
into gcc trunk).
What I meant is, since in my hypothetical, we are being careful to have
one .a full of pic symbols, and another .a full of non-pic symbols, then
when we are creating a shared library and are explicitly using the .a
full of pic symbols -- in THAT limited case, we should not get the
FAILURE message about "you can't create shared libs with static
dependencies"
This is not a warning, not even on win32. It's a failure; libtool dies,
make reports error, bang you are dead. This is because we currently
can't tell, just from looking at the .a, if the objects are pic or
non-pic. It took a lot of dancing just to teach func_win32_libid() how
to distinquish between import libraries and "regular" static libraries.
Now, *if* there truly were a difference (on win32) between "pic" and
"non-pic" objects, we'd have a fighting chance to educate
func_win32_libid -- but at present, it's a distinction without a difference.
So, that's why I suspected that an implementation of my idea would
require an additional keyword in the .la file -- so that libtool
linkmode could tell (from the .la) that
"yes yes, I know file_magic_cmd reports that this is a "static library"
and ordinarily that would be a problem, but since this particular
dependency claims to be a "resolver", I know I can use the "pic" version
(stored as the first element of library_names, where the .dll.a would
usually go) without trouble. So no warn, no fail..."
(Hmm. When -no-undefined is NOT specified, then there's a choice: you could
satisfy as many undefined symbols as you could by passing the resolver lib
to the linker, or you could simply drop the resolver entirely -- which would
match the current build procedure for libbfd & friends on non-win32...)
I don't think it would be that difficult to add this facility to libtool --
No, it wouldn't. The semantics are the hard part. If we can't get
well-defined semantics, the new baby will be useful for binutils (and of
course the set of packages that use it) only.
Possibly. Also might be useful for packages that use gnulib.
But, it still smells to me like this whole problem might be a win32-only
thing. Like THAT's new. So, since win32 has symbol hiding, maybe a
little bloat is not that bad: build as a regular convenience archive,
libtool will include it entirely into the eventual target lib via
--whole-archive, BUT a little configury/automake magic in these limited
cases to finagle --exclude-lib into _LDFLAGS.
(again: okay for shared lib. But static target lib...)
So, instead of new library "type", is there some sort of directive we
could implement instead? Say, something like:
--------- Makefile.am --------
noinst_LTLIBRARIES = libconvenience.la
LT_LIBRARIES = libtarget.la
libtarget_LDFLAGS = -no-undefined -exclude-lib libconvenience.la
libtarget_LIBADD = libconvenience.la
-------------------------------
Where the semantics of the libtool option '-exclude-lib' would be
(1) arguments must be convenience libraries (if functionality is useful,
later modifications could relax this)
(2) libconvenience.a is given the --whole-archive/-no-whole-archive
treatment when building the shared libtarget, but ONLY if the target
architecture supports symbol hiding (see below). libconvenience.a is
also marked with "-Wl,--exclude-libs -Wl,libtarget.a".
(2a) libconvenience.a is included in dependency calculations and
everything for the static libtarget.a, but is NOT given "explode and
incorporate into libtarget.a" treatment. In fact, libconvenience.a
would have no effect on the final 'ar' command line for libtarget; it
would/may only influence order-of-compilation due to dependency
calculations.
(3) Note that because --exclude-libs is supported only by the GNU
linker, and only for i386pe- and ELF- targetted ld's, on other platforms
it should degrade to "pretend argument is just a non-installed static
library". That is, -exclude-lib would
(a) ALWAYS "turn off" explode-and-incorporate behavior when building
static libtarget. (This means that libconvenience.a is more-or-less
removed from libtarget_LIBADD, as far as building libtarget.a is
concerned, on all platforms)
(b) When building shared libtarget on a platform that supports symbol
hiding (i386pe/ELF), do the --exclude-libs thing. Obviously,
libconvenience.a is still be included on the link command for shared
libtarget, sandwiched between --whole-archive / --no-whole-archive.
(c) when building shared libtarget where symbol hiding is NOT
available, just "turn off" the --whole-archive / --no-whole-archive
behavior. TBD:
(c1) keep libconvenience.a on the link command line, and treat it
like a "regular" static library.
(c2) drop libconvenience.a from the link command line entirely.
The choice betwen (c1) and (c2) might be conditionalized on whether
-no-undefined was specified, but that might be just a little too "cute".
Caveats: end user run into trouble is she explicitly decorates some
symbols in libconvenience.a with declspec(dllexport) [i386pe] or perhaps
sets visibility explicitly (ELF). Don't Do That.
Hmmm...now that I've gotten it all written down, it sounds very possible
that this could give all the required features my "new library type"
would, but without requiring any changes to the .la format, and putting
the control in the hands of the target library (e.g. 'how
libconvenience.la is used') rather than in the hands of libonvenience
itself (e.g. 'how libconvenience.la is built'). That's a plus.
There's still some issues with convenience libs themselves (where the
following discussion ignores ALL of the above, no new library types, no
new -exclude-lib option), like:
(a) we should build a pic version and stuff it inside .libs/ if
-enable-shared, and put a regular version in . if -enable-static
(with caveats for Linux/MIPS, etc, where everything truly is
pic -- unlike i386pe where we just pretend that everything is pic
and hope the Windows Runtime Loader fixes everything properly,
with all the attendant .rdata problems)
(b) When linking a shared lib where a convenience lib appears in the
deps, use the one in the .libs/ directory
(c) ??? when linking a PIE executable where a convenience lib appears
in the deps, use the one in the .libs/ directory
(d) when creating a static lib where a convenience lib appears in the
deps, use the one in the . directory when doing the explode-and-
incorporate thing.
(e) when creating a next-level convenience lib that has THIS
convenienece lib as a dep, again, create a pic version in .libs
and
a non-pic version in . In each case, use the "matching" version of
the dependency when doing the explode-and-incorporate thing.
(f) In all other cases, use the "regular" lib
[replace 'explode-and-incorporate thing' with 'partial linking thing' if
your platform supports it]
These improvements to "regular" convenience libraries are, IMO,
necessary for proper operation where pic/no-pic makes a difference.
Plus, they'd make the -exclude-lib stuff work a lot better, too. <g>
But I'd rather not -- especially this close (?) to 2.0final
Certainly not.
Yeah, regardless of where this discussion goes, it's not likely to have
any effect until after libtool-2.0-gold, and gcc-4.4+. So in the
interim, gcc/win32 will have to do _something_ else. I'll keep hacking
on it.
However, gcc steering committee policy is that external tools will be
either exact release versions, or top-of-development-tree for the
external tool. So, they would actually be *required* to use the
development versions after libtool-2.0 comes out...so maybe gcc can
benefit from something like the above, sooner than libtool-2.2.
--
Chuck
_______________________________________________
http://lists.gnu.org/mailman/listinfo/libtool