Re: Autoconf tests, libtool symlist files, undefined behavior, and LTO
Hello t7, * t66...@gmail.com wrote on Thu, Sep 23, 2010 at 03:01:31AM CEST: I don't know if my problem suites this description. No, it doesn't. Currently installed libtool on this system is, ltmain.sh (GNU libtool) 2.2.6b I recently tested the LTO feature of GCC (targeting windows) and I found it was unable to link due to the presence of duplicating lines of *crt* without compiling with -flto there were not such issues. It seems that libtool is emitting dllcrt2, crtbegin, crtend all over again after the first crtend. In the following line. g++ lib64/dllcrt2.o lib64/crtbegin.o ... _alot_of_other_link_arguments_ ... lib64/crtend.o lib64/dllcrt2.o lib64/crtbegin.o lib64/crtend.o These last three duplicating .o arguments are causing errors. lib64/dllcrt2.o:crtdll.c:(.text+0x50): multiple definition of `_CRT_INIT' lib64/dllcrt2.o:crtdll.c:(.text+0x50): first defined here Is this a know issue? You may have found a bug, or not, I cannot tell from the sparse description you've given. Please write to the bug-libtool at gnu.org mailing list (no subscription required) with full details about which GCC and Libtool versions you are using, after updating Libtool to 2.4 if you are trying to use LTO with it, write how you configured the software that fails to link, and please post the complete 'libtool --mode=link' command that fails, including all of its output. Then we will see further. On the odd chance of turning up a new GCC bug, we'll report back to the GCC bugzilla then. But until then there is no need to cross-post this to the GCC development mailing list. Thank you. Cheers, Ralf ___ http://lists.gnu.org/mailman/listinfo/libtool
Re: Autoconf tests, libtool symlist files, undefined behavior, and LTO
Hello all,
I don't know if my problem suites this description.
On 31/03/2010 6:52 AM, Ralf Wildenhues wrote:
Hello gcc and libtool lists,
Summary: both Autoconf-generated configure tests as well as some Libtool
construct invoke undefined behavior. Question is how to deal with it,
and whether GCC, as QoI, may want to define behavior in these cases.
Currently installed libtool on this system is,
ltmain.sh (GNU libtool) 2.2.6b
I recently tested the LTO feature of GCC (targeting windows) and I found
it was unable to link due to the presence of duplicating lines of *crt*
without compiling with -flto there were not such issues.
It seems that libtool is emitting dllcrt2, crtbegin, crtend all over
again after the first crtend. In the following line.
g++ lib64/dllcrt2.o lib64/crtbegin.o ... _alot_of_other_link_arguments_
... lib64/crtend.o lib64/dllcrt2.o lib64/crtbegin.o lib64/crtend.o
These last three duplicating .o arguments are causing errors.
lib64/dllcrt2.o:crtdll.c:(.text+0x50): multiple definition of `_CRT_INIT'
lib64/dllcrt2.o:crtdll.c:(.text+0x50): first defined here
Is this a know issue?
1) Autoconf-generated configure tests often fake the prototype of some
function; e.g., AC_CHECK_FUNC([func]) uses
char func();
and tries to link that. Using this is undefined according to C99, if
func has a different actual prototype, and when all system libraries are
LTO'ed, gcc -flto may even detect this kind of inconsistency and could
act accordingly (nasal demons and such).
2) libtool has a feature that makes it extract symbol lists from
objects and turn them into fake declarations and function/object
pointers: fake static preloaded modules.
It currently works by running nm or a similar tool over the object, then
converting the output with a couple of sed script or so
(global_symbol_pipe, global_symbol_to_cdecl, and a couple more) to a
synthesized extra source file that then contains code like this:
extern int func();
extern char variable;
typedef struct {
const char *name;
void *address;
} lt_dlsymlist;
extern const lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[];
const lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[] =
{ { @PROGRAM@, (void *) 0 },
{func, (void *)func},
{variable, (void *)variable},
{0, (void *) 0}
};
This is invoking undefined behavior in a couple of respects:
a) Pointers to functions are stored in pointer-to-void variables.
This could be fixed with an incompatible API change to using a union of
object and function pointer, I guess.
b) The symbols 'func' and 'variable' likely have the wrong prototypes,
i.e., elsewhere, they might be declared as
void func(int, double);
double variable[42];
instead. I haven't come across any actual issues with this yet, except
now LTO may rightfully complain about it.
Question is, what can we do about this? We could ensure to never pass
-flto or -fwhopr to the compilation of the libtool symfile object, and
remove it from some or all link tests done in configure. That's ugly.
Would that even be sufficient though? Conversely, would GCC developers
be willing to agree that, when GCC detects such inconsistencies, it
wouldn't take adverse advantage of it (e.g., by turning off LTO in this
case, or similar)?
Other possibilities for Autoconf would be to work toward a new set of
checking macros (or extensions of current one) where the configure.ac
author passes a full prototype for each function to check (Autoconf
could keep a list of known prototypes for often-checked functions).
I'm not sure how to fix the libtool symfile in a C99-conforming way.
Thanks for reading this far.
Cheers,
Ralf
___
http://lists.gnu.org/mailman/listinfo/libtool
Re: Autoconf tests, libtool symlist files, undefined behavior, and LTO
On Wed, Mar 31, 2010 at 8:33 PM, Ralf Wildenhues ralf.wildenh...@gmx.de wrote:
* Richard Guenther wrote on Wed, Mar 31, 2010 at 11:02:39AM CEST:
On Tue, Mar 30, 2010 at 8:52 PM, Ralf Wildenhues wrote:
1) Autoconf-generated configure tests often fake the prototype of some
function; e.g., AC_CHECK_FUNC([func]) uses
char func();
and tries to link that. Using this is undefined according to C99, if
func has a different actual prototype, and when all system libraries are
LTO'ed, gcc -flto may even detect this kind of inconsistency and could
act accordingly (nasal demons and such).
I suppose autoconf cannot do this for C++ functions then, because
of mangling issues?
Correct. For C++ libraries, it is more typical to just write a complete
test source and AC_COMPILE_IFELSE or AC_LINK_IFELSE it.
FWIW, there is an Autoconf patch pending to allow AC_CHECK_DECL with
declarations given by the user (in order to support overloaded
basename, for example).
Note that the only thing GCC with LTO might do here is to issue
a diagnostic (which of course might confuse the configure script),
but we cannot really reject such programs (as such errors are
unfortunately very common) and thus defer any problems to
link- and/or runtime.
That's almost exactly the kind of semantics I would like to see.
Can we get this documented in the manual? Something like this.
Note that it would explicitly contradict one of the design goals
listed in lto.pdf, which is that conflicting declarations might
provoke an error; so really GCC developers should make a conscious
design decision here.
* doc/invoke.texi (Optimize Options): Document that LTO
won't remove object access purely due to incompatible
declarations.
diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi
index 2226cad..85f9c5f 100644
--- a/gcc/doc/invoke.texi
+++ b/gcc/doc/invoke.texi
@@ -7294,6 +7294,12 @@ regular (non-LTO) compilation. This means that if
your build process
was mixing languages before, all you need to add is @option{-flto} to
all the compile and link commands.
+If LTO encounters objects with C linkage declared with incompatible
+types in separate translation units to be linked together (undefined
+behavior according to ISO C99 6.2.7), it might produce a warning, but
+this fact alone will not cause an access to an object to be optimized
+away.
+
If object files containing GIMPLE bytecode are stored in a library
archive, say @file{libfoo.a}, it is possible to extract and use them
in an LTO link if you are using @command{gold} as the linker (which,
Well, the wording is almost ok, but
+behavior according to ISO C99 6.2.7), a non-fatal diagnostic may
be issued. The behavior is still undefined at runtime.
would be more precise. Especially accesses to conflicting
declarations can end up being optimized away if there's unfortunate
inlining so that for example with
t1.c
float f;
t2.c
int f;
f[int] = 1.0;
f[float] = 1;
GCC can end up re-ordering the stores to f and thus effectively
optimize away one or the other.
With function calls there's no such issue, but argument passing
might be completely off (obviously).
Richard.
(In practice, Autoconf does not support -Werror at configure time; this
issue only reinforces that.)
b) The symbols 'func' and 'variable' likely have the wrong prototypes,
i.e., elsewhere, they might be declared as
void func(int, double);
double variable[42];
instead. I haven't come across any actual issues with this yet, except
now LTO may rightfully complain about it.
Same issue as above. We try to handle it - there might be bugs
in the current implementation of LTO though.
Bugs are no problem as long as they are acknowledged as such. I desire
future compatibility, i.e., being fairly certain autotools don't regress
just because of a good improvement in some other tool. Dealing with
existing cruft is abundant in autotools.
Question is, what can we do about this? We could ensure to never pass
-flto or -fwhopr to the compilation of the libtool symfile object, and
remove it from some or all link tests done in configure. That's ugly.
Would that even be sufficient though? Conversely, would GCC developers
be willing to agree that, when GCC detects such inconsistencies, it
wouldn't take adverse advantage of it (e.g., by turning off LTO in this
case, or similar)?
Other possibilities for Autoconf would be to work toward a new set of
checking macros (or extensions of current one) where the configure.ac
author passes a full prototype for each function to check (Autoconf
could keep a list of known prototypes for often-checked functions).
I'm not sure how to fix the libtool symfile in a C99-conforming way.
I'd say wait and see. What would be nice to have is a few testcases
that cover the autoconf cases in the GCC testsuite (feel free to
just file them into bugzilla).
I have been doing just that for
Re: Autoconf tests, libtool symlist files, undefined behavior, and LTO
On Tue, Mar 30, 2010 at 8:52 PM, Ralf Wildenhues ralf.wildenh...@gmx.de wrote:
Hello gcc and libtool lists,
Summary: both Autoconf-generated configure tests as well as some Libtool
construct invoke undefined behavior. Question is how to deal with it,
and whether GCC, as QoI, may want to define behavior in these cases.
1) Autoconf-generated configure tests often fake the prototype of some
function; e.g., AC_CHECK_FUNC([func]) uses
char func();
and tries to link that. Using this is undefined according to C99, if
func has a different actual prototype, and when all system libraries are
LTO'ed, gcc -flto may even detect this kind of inconsistency and could
act accordingly (nasal demons and such).
I suppose autoconf cannot do this for C++ functions then, because
of mangling issues?
Note that the only thing GCC with LTO might do here is to issue
a diagnostic (which of course might confuse the configure script),
but we cannot really reject such programs (as such errors are
unfortunately very common) and thus defer any problems to
link- and/or runtime.
2) libtool has a feature that makes it extract symbol lists from
objects and turn them into fake declarations and function/object
pointers: fake static preloaded modules.
It currently works by running nm or a similar tool over the object, then
converting the output with a couple of sed script or so
(global_symbol_pipe, global_symbol_to_cdecl, and a couple more) to a
synthesized extra source file that then contains code like this:
extern int func();
extern char variable;
typedef struct {
const char *name;
void *address;
} lt_dlsymlist;
extern const lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[];
const lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[] =
{ { @PROGRAM@, (void *) 0 },
{func, (void *) func},
{variable, (void *) variable},
{0, (void *) 0}
};
This is invoking undefined behavior in a couple of respects:
a) Pointers to functions are stored in pointer-to-void variables.
This could be fixed with an incompatible API change to using a union of
object and function pointer, I guess.
b) The symbols 'func' and 'variable' likely have the wrong prototypes,
i.e., elsewhere, they might be declared as
void func(int, double);
double variable[42];
instead. I haven't come across any actual issues with this yet, except
now LTO may rightfully complain about it.
Same issue as above. We try to handle it - there might be bugs
in the current implementation of LTO though.
Question is, what can we do about this? We could ensure to never pass
-flto or -fwhopr to the compilation of the libtool symfile object, and
remove it from some or all link tests done in configure. That's ugly.
Would that even be sufficient though? Conversely, would GCC developers
be willing to agree that, when GCC detects such inconsistencies, it
wouldn't take adverse advantage of it (e.g., by turning off LTO in this
case, or similar)?
Other possibilities for Autoconf would be to work toward a new set of
checking macros (or extensions of current one) where the configure.ac
author passes a full prototype for each function to check (Autoconf
could keep a list of known prototypes for often-checked functions).
I'm not sure how to fix the libtool symfile in a C99-conforming way.
I'd say wait and see. What would be nice to have is a few testcases
that cover the autoconf cases in the GCC testsuite (feel free to
just file them into bugzilla). We really do not want to break
working setups with LTO - any fancy ODR violation diagnostics
should IMHO be optional, only things that LTO does not get
correct are currently diagnosed IIRC.
Thanks,
Richard.
Thanks for reading this far.
Cheers,
Ralf
___
http://lists.gnu.org/mailman/listinfo/libtool
Autoconf tests, libtool symlist files, undefined behavior, and LTO
Hello gcc and libtool lists,
Summary: both Autoconf-generated configure tests as well as some Libtool
construct invoke undefined behavior. Question is how to deal with it,
and whether GCC, as QoI, may want to define behavior in these cases.
1) Autoconf-generated configure tests often fake the prototype of some
function; e.g., AC_CHECK_FUNC([func]) uses
char func();
and tries to link that. Using this is undefined according to C99, if
func has a different actual prototype, and when all system libraries are
LTO'ed, gcc -flto may even detect this kind of inconsistency and could
act accordingly (nasal demons and such).
2) libtool has a feature that makes it extract symbol lists from
objects and turn them into fake declarations and function/object
pointers: fake static preloaded modules.
It currently works by running nm or a similar tool over the object, then
converting the output with a couple of sed script or so
(global_symbol_pipe, global_symbol_to_cdecl, and a couple more) to a
synthesized extra source file that then contains code like this:
extern int func();
extern char variable;
typedef struct {
const char *name;
void *address;
} lt_dlsymlist;
extern const lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[];
const lt_dlsymlist
lt__PROGRAM__LTX_preloaded_symbols[] =
{ { @PROGRAM@, (void *) 0 },
{func, (void *) func},
{variable, (void *) variable},
{0, (void *) 0}
};
This is invoking undefined behavior in a couple of respects:
a) Pointers to functions are stored in pointer-to-void variables.
This could be fixed with an incompatible API change to using a union of
object and function pointer, I guess.
b) The symbols 'func' and 'variable' likely have the wrong prototypes,
i.e., elsewhere, they might be declared as
void func(int, double);
double variable[42];
instead. I haven't come across any actual issues with this yet, except
now LTO may rightfully complain about it.
Question is, what can we do about this? We could ensure to never pass
-flto or -fwhopr to the compilation of the libtool symfile object, and
remove it from some or all link tests done in configure. That's ugly.
Would that even be sufficient though? Conversely, would GCC developers
be willing to agree that, when GCC detects such inconsistencies, it
wouldn't take adverse advantage of it (e.g., by turning off LTO in this
case, or similar)?
Other possibilities for Autoconf would be to work toward a new set of
checking macros (or extensions of current one) where the configure.ac
author passes a full prototype for each function to check (Autoconf
could keep a list of known prototypes for often-checked functions).
I'm not sure how to fix the libtool symfile in a C99-conforming way.
Thanks for reading this far.
Cheers,
Ralf
___
http://lists.gnu.org/mailman/listinfo/libtool
