Re: OpenSSL breaks with gcc 4.2
On Fri, Nov 10, 2006, Dr. Stephen Henson wrote: OK, looks like inline calls is the way to go. Those at least have some advantages over the existing stuff. For gcc (4.0, 4.2 at least on X86) it looks like it will translate equivalent (at a machine level) function calls into a single jump instruction with -O3 -fomit-frame-pointer. In the inline case will optimize away the call entirely so that a call to sk_X509_new_null() ends up outputting identical code to sk_new_null(). It will not do the same for function pointers however so passing sk_X509_new_null() as an fp doesn't end up passing sk_new_null(). So it seems that at least on these platforms doing the right thing will have minimal overhead. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
The equivalent of the offending line would be foo((double *)(void *) j); since any access through a void pointer CAN change the value. '(double *)(void *)' does not keep some sort of remnant of the 'void *'. The final cast renders the end-result a 'double *' and negates the 'void *'. You cannot cast 'through' another type to get some remnant of that type. (In old C.) This is a very common 'fix' for aliasing bugs that does not work. Thank you for showing me why I should always include -fno-strict-aliasing in my gcc commandlines. It's broken, horribly. Actually, what's broken is not allowing a large class of useful optimizations for the occasional code that chooses to invoke undefined behavior. That said, I do exactly what you suggest. I've had too much old code break horribly and subtly to find the optimizations worth the risk. So my 'official' position is not the same as my personal position. C has unions and 'char *', and more recently 'void *'. However, you cannot invoke them and then discard them and expect some 'remnant'. An object is the type of the last cast. I'm going to go dig out my standard. Maybe I'm misremembering. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
An object may be the type of its last cast -- but it also can't exactly lose the benefit/cost of being cast to a pointer to an undefined type. As soon as you undefine the type of a pointer, it loses the remnant of ever having had the initial type in the first place. In KR C, you couldn't cast to void *, that was first introduced in C89. You had to cast to char *, and then you could cast to some other type in order to make your array indexes jump sizeof(type), essentially (on 8-bit byte machines) referencing array[(index)*(sizeof(type))]. On 11/10/06, David Schwartz [EMAIL PROTECTED] wrote: The equivalent of the offending line would be foo((double *)(void *) j); since any access through a void pointer CAN change the value. '(double *)(void *)' does not keep some sort of remnant of the 'void *'. The final cast renders the end-result a 'double *' and negates the 'void *'. You cannot cast 'through' another type to get some remnant of that type. (In old C.) (double *)(void *) can't keep any sort of remnant of the int, as it becomes never-was-an-int-at-all via the (void *) cast. It's also illegal to take the address of a register, which destroys the entire concept of aliasing in the first place -- core has always been core, and any operation can be performed on any part of core (with the proviso that the machine code generated by the C compiler must assure proper alignment for the processor). Trying to treat any piece of core as anything other than a union of all possible types (since it can't be a struct, because structs take up more memory to be able to do what they do, and that violates the assumptions on which the string library depends, among others) is completely foolhardy, and gives rise to a situation which would more properly be in the domain of 'pedantic'. Thank you for showing me why I should always include -fno-strict-aliasing in my gcc commandlines. It's broken, horribly. Actually, what's broken is not allowing a large class of useful optimizations for the occasional code that chooses to invoke undefined behavior. That said, I do exactly what you suggest. I've had too much old code break horribly and subtly to find the optimizations worth the risk. So my 'official' position is not the same as my personal position. 'occasional'? I've never examined any substantial program in C that doesn't invoke that -- even if just to be able to read data from disk into memory. C has unions and 'char *', and more recently 'void *'. However, you cannot invoke them and then discard them and expect some 'remnant'. An object is the type of the last cast. I'm going to go dig out my standard. Maybe I'm misremembering. DS An object is the type of the last cast -- but if it's ever cast to void *, it must be treated as if it were never any type before that. This may have changed in C99, but I'd really much rather have the option of selecting which dialect I'm using, if only because code that does work will suddenly not work if compiled with a new dialect. (Is the word 'father' or 'goat'? I can't tell...) -Kyle H __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
I found the rule -- at least for C99. It is ISO 9899:1999 section 6.2.5, rule 26 and footnote 39: 26) A pointer to void shall have the same representation and alignment requirements as a pointer to a character type. Similarly, pointers to qualified or unqualified versions of compatible types shall have the same representation and alignment requirements. All pointers to structure types shall have the same representation and alignment requirements as each other. All pointers to union types shall have the same representation and alignment requirements as each other. Pointers to other types need not have the same representation or alignment requirements. 39) The same representation and alignment requirements are meant to imply interchangeability as arguments to functions, return values from functions, and members of unions. This means you cannot pass an 'X509 **' as a 'char **'. Unfortunately 'X509 *' and 'char *' are not compatible types because 'X509' and 'char' are not compatible. They are not both unions, they are not both structures. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
An object may be the type of its last cast -- but it also can't
exactly lose the benefit/cost of being cast to a pointer to an
undefined type. As soon as you undefine the type of a pointer, it
loses the remnant of ever having had the initial type in the first
place.
Right, but that doesn't help you. For example:
int j=2;
double *d=(double *)(void *)j;
*d=1.0;
printf(%d\n, j);
This can output 2. Even though 'j' lost all intness by the cast to 'void *',
it got 100% doubleness by the cast to 'double *'. The compiler can still assume
that '*d' will not affect the value of 'j' because 'd' is a pointer to a type
that is incompatible with 'j's type.
The rule still stands -- a modification through a pointer to one type can be
assumed not to change the value of a variable of an incompatible type.
This is still what OpenSSL does when it passes an 'X509 **' as a 'char **'.
In KR C, you couldn't cast to void *, that was first introduced in
C89. You had to cast to char *, and then you could cast to some other
type in order to make your array indexes jump sizeof(type),
essentially (on 8-bit byte machines) referencing
array[(index)*(sizeof(type))].
The cast to 'char *' made things safe. However, once you cast it away from
'char *', it became dangerous again. There was no remnant of the 'char *' cast.
Yes, while it was a 'char *', it could alias anything, but as soon as you cast
it to something else, it can no longer legally alias an incompatible type.
However, it should be safe to pass a 'char *' to a function that takes a 'char
*', and cast that parameter back to its real value in the function. For
example, this should be legal:
int inc(int is_double, char *pointer)
{
if(is_double)
{
double *p=(double *) pointer;
*p=*p+1;
}
else
{
int *p=(int *) pointer;
*p=*p+1;
}
return 0;
}
---
int inc()
double j=1.0;
int i=to_int(1, (char *) j);
printf(%f, j);
That is because you are passing the function a compatible type, the compiler
must assume a 'char *' might alias anything, and you are only dereferencing the
pointer as a compatible type.
Actually, what's broken is not allowing a large class of useful
optimizations for the occasional code that chooses to invoke
undefined behavior. That said, I do exactly what you suggest.
I've had too much old code break horribly and subtly to find the
optimizations worth the risk. So my 'official' position is not
the same as my personal position.
'occasional'? I've never examined any substantial program in C that
doesn't invoke that -- even if just to be able to read data from disk
into memory.
Many programs do aliasing correctly, following the rules that permit pointers
to alias. However, I think it's a very fair point that nobody seems to totally
understand what the rules *are*, which makes it hard to be sure you're
following them.
An object is the type of the last cast -- but if it's ever cast to
void *, it must be treated as if it were never any type before that.
The problem is precisely that it loses its association with the type it first
was! If it kept its association with the first type, there'd be no problem.
Look at this code again:
int j=2;
double *d=(double *)(void *)j;
*d=1.0;
printf(%d\n, j);
The *problem* is that 'j' has list its intness.
DS
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
This means you cannot pass an 'X509 **' as a 'char **'.
Unfortunately 'X509 *' and 'char *' are not compatible types because
'X509' and 'char' are not compatible. They are not both unions, they
are not both structures.
But you can pass 'X509 **' as 'void *'. So...
void x509func(void* p) { X509** pp = (X509**)p; ... }
void trampoline(void* p) { x509func(p); }
void caller(void) { X509* p; trampoline(p); }
should (MUST?) work just fine.
/r$
--
STSM, Senior Security Architect
SOA Appliances
Application Integration Middleware
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
But you can pass 'X509 **' as 'void *'. So...
void x509func(void* p) { X509** pp = (X509**)p; ... }
void trampoline(void* p) { x509func(p); }
void caller(void) { X509* p; trampoline(p); }
should (MUST?) work just fine.
/r$
When I change my test programs from 'float *' to either 'void *' or 'char
*', the warning goes away. I believe this is completely safe.
So long as the pointer is a 'char *' or a 'void *' at the point of
contention, you are fine. A union would also work. (Whether you use a real
union with a pointer to every possible type or a bogus union containing just
a 'char **'.
The Linux kernel contains stuff like this:
/*
* Make sure gcc doesn't try to be clever and move things around
* on us. We need to use _exactly_ the address the user gave us,
* not some alias that contains the same information.
*/
typedef struct { volatile int counter; } atomic_t;
DS
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
David Schwartz wrote:
An object may be the type of its last cast -- but it also can't
exactly lose the benefit/cost of being cast to a pointer to an
undefined type. As soon as you undefine the type of a pointer, it
loses the remnant of ever having had the initial type in the first
place.
Right, but that doesn't help you. For example:
int j=2;
double *d=(double *)(void *)j;
*d=1.0;
printf(%d\n, j);
This can output 2. Even though 'j' lost all intness by the cast to 'void *',
it got 100% doubleness by the cast to 'double *'. The compiler can still assume
that '*d' will not affect the value of 'j' because 'd' is a pointer to a type that
is incompatible with 'j's type.
The rule still stands -- a modification through a pointer to one type can be
assumed not to change the value of a variable of an incompatible type.
This is still what OpenSSL does when it passes an 'X509 **' as a 'char **'.
Are you sure that problem is in cast ?
$ cat test.c
main() {
int j=2;
double *d=(double*)j;
*d=1.0;
printf(%d %e\n, j, *d);
printf(%d %e\n, j, *d);
}
gcc -O2 test.c ./a.out
2 1.00e+00
0 1.00e+00
Same result in case with line double *d=j; (but expected warning:
initialization from incompatible pointer type) and double
*d=(double*)(void*)j;
Thanks to point side effect of -O2 in gcc.
May be is related to -fstrict-aliasing.
The above code we can found in gcc documentation for -fstrict-aliasing.
More results:
$ gcc -O2 -fno-strict-aliasing test.c ./a.out
0 1.00e+00
0 1.00e+00
$ gcc -O1 -fstrict-aliasing test.c ./a.out
0 1.00e+00
0 1.00e+00
[SNIP]
It seem to me that all examples by David show noting related to the casts.
They only show problem by gcc -O2 .
Roumen
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
Are you sure that problem is in cast ?
$ cat test.c
main() {
int j=2;
double *d=(double*)j;
*d=1.0;
printf(%d %e\n, j, *d);
printf(%d %e\n, j, *d);
}
gcc -O2 test.c ./a.out
2 1.00e+00
0 1.00e+00
Same result in case with line double *d=j; (but expected warning:
initialization from incompatible pointer type) and double
*d=(double*)(void*)j;
No, the problem is not in the cast. The problem is that the cast doesn't
save you when you think it does. Without the cast, you are equally screwed,
it's just that it's obvious you deserve it in that case. ;)
It seem to me that all examples by David show noting related to the casts.
They only show problem by gcc -O2 .
The problem is that the casts don't do what people think they do and that
GCC assumes code obeys the standard.
With or without the cast, the compiler has every right to assume that a
modification to the thing a 'double *' points to won't change the value of
an integer. The exception would be code that uses the specifically allowed
aliasing techniques, such as unions and pointers to characters.
It's just that compilers haven't previously been smart enough to make these
kinds of optimizations. Now they are, so we'll have to either follow the
standard or disable the optimizations. Which option is a judgment call. I
generally try to do both out of abundance of caution.
The argument is that code that does this is rare enough and the
optimizations valuable enough that it's worth making everyone fix their code
to follow the aliasing rules. The counter-argument is that experts don't
agree on what the aliasing rules *are*, so how are mere mortals supposed to
follow them?
This is separate from the issue of calling a function with an incompatible
pointer type, which is also a violation of the standard.
There may be problems with newer versions of GCC even for code that complies
with all the parameter passing and aliasing rules, but I haven't seen it.
Does anyone know if turning off strict aliasing and changing the functions
to take 'char *'s solves the problem?
I might be able to run some tests myself in a few hours.
DS
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
Richard Levitte - VMS Whacker wrote: In message [EMAIL PROTECTED] on Wed, 8 Nov 2006 21:59:19 -0800, David Schwartz [EMAIL PROTECTED] said: davids You are correct, but that's not the issue. The issue is this davids simple -- if you are going to call a function whose types you davids don't know (through a prototype), you must cast each type you davids pass to the type the function expects. End of story. OpenSSL davids does not do this. This is not valid C whether or not the type davids sizes are the same. So basically, you're saying that KR-style functions (non-prototyped) aren't handled in a pre-prototype manner any more? That's a fairly extreme change of how C used to work. While I understand such a change, it's going to break quite a lot of things that still have non-prototyped functions. Can you provide something in the C standard that supports this change? Clearly this argument only applies to prototyped functions. -- http://www.apache-ssl.org/ben.html http://www.links.org/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
David Schwartz wrote: x is still just a pointer to data - so it's the same length in any case, all pointers to lvalues are the same length in C. The only issue there is whether it's aligned correctly - that's the programmers problem. Length is not the issue. There is no rule that says that two types must be passed the same way just because they're the same length. For example, some platforms may pass a 64-bit floating point number in a floating point register but pass a 64-bit integer in an integer register. All you can do is assume that it will work because it has always worked, until it stops working. Mixing something like char *(*d2i)(), and char ** IS problematic, since those aren't guaranteed to be the same length but as far as I can remember OpenSSL doesn't do that. If they are not the same length, it is not likely to work. That doesn't mean it's legal if they are the same length. ret=d2i(x,p,len); d2i is a function (prototype is unknown) but I've been told x is a pointer, p is a pointer, len is long. There's nothing indeterminate about the size of any of those. The issue is not size, it's type. C isn't a strongly typed language - there's no language requirement for an accurate function prototype or that the types of the arguments be correct - only that the size of them is correct. You are correct, but that's not the issue. The issue is this simple -- if you are going to call a function whose types you don't know (through a prototype), you must cast each type you pass to the type the function expects. End of story. OpenSSL does not do this. This is not valid C whether or not the type sizes are the same. C is not the same language as C++, this instance should work with a C compiler - whether or not the compiler is passing the arguments via the stack or in registers. Can you cite anything in the C standard that says this is allowed? As I understand it, you must pass a function precisely the types it expects, or use a prototype to force the compiler to cast them. Either way, the exact type must be known where the call is compiled. It appears to me, then, that the consequence of this is that we can't have type-safety and generic functions. This strikes me as a huge pain in the ass. Surely there's some way around it? -- http://www.apache-ssl.org/ben.html http://www.links.org/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
Length is not the issue. There is no rule that says that two types must be
passed the same way just because they're the same length. For example, some
platforms may pass a 64-bit floating point number in a floating point
register but pass a 64-bit integer in an integer register.
I'm not sure that's true, it's an optimization sure, but C doesn't care
about type. So there are optimizations that aren't legal for the C
language.
In this case I can't see a problem.
Sure if you were passing either a float or a long, which happened to be the
same size and you had a dedicated FPU register I can see a problem, (though
I suspect even in this case the optimzation is dubious for C ) but in this
case the ambiguity is at the thing the pointer points to. Any optimization
here would be in the called function - either loading the thing the pointer
points to into an integer or FPU register. They can't have separate address
spaces or this would be impossible:
typedef struct {
float f;
long l;
} X;
You are correct, but that's not the issue. The issue is this simple -- if
you are going to call a function whose types you don't know (through a
prototype), you must cast each type you pass to the type the function
expects. End of story. OpenSSL does not do this. This is not valid C
whether
or not the type sizes are the same.
Sorry, I think it's a compiler bug. The compiler DOES know the types of the
arguments in every case, it's what they are declared to be by default, it
even has the information necessary to decide the register the arguments
should be passed in (WRT to the FPU register argument above.)
Yes the programmer *might* have screwed up, but that's not the compiler
writers call. Emit a warning by all means but this is resolvable at compile
time.
And saying it's the compilers job to catch this is bogus as well, I can
cast the function to the wrong type and cause exactly the problems at
runtime that this optimization is supposed to protect against.
Can you cite anything in the C standard that says this is allowed? As I
understand it, you must pass a function precisely the types it expects, or
use a prototype to force the compiler to cast them. Either way, the exact
type must be known where the call is compiled.
printf() still works and that disagrees with your view of the universe.
And as noted above, the function is prototyped, just not explicitly,
assuming the programmer got the argument types correct and the type of the
variable holding the return value you have an implicit (and sufficient)
function prototype.
You can't have no function prototype at all at the point you make the call
in C. Yes, you can get it wrong and crash, but historically that's been the
programmers responsibility.
Your argument IS valid for C++, but I'm not yet convinced it's correct for
C.
Peter
https://cs.opensource.ibm.com/projects/icc
David Schwartz
[EMAIL PROTECTED]
.com To
Sent by: openssl-dev@openssl.org
owner-openssl-dev cc
@openssl.org
Subject
RE: OpenSSL breaks with gcc 4.2
09/11/06 03:59 PM
Please respond to
openssl-dev
x is still just a pointer to data - so it's the same
length in any case, all pointers to lvalues are the
same length in C. The only issue there is whether it's
aligned correctly - that's the programmers problem.
Length is not the issue. There is no rule that says that two types must be
passed the same way just because they're the same length. For example, some
platforms may pass a 64-bit floating point number in a floating point
register but pass a 64-bit integer in an integer register.
All you can do is assume that it will work because it has always worked,
until it stops working.
Mixing something like char *(*d2i)(), and char ** IS
problematic, since those aren't guaranteed to be the
same length but as far as I can remember OpenSSL doesn't
do that.
If they are not the same length, it is not likely to work. That doesn't
mean
it's legal if they are the same length.
ret
Re: OpenSSL breaks with gcc 4.2
davids simple -- if you are going to call a function whose types you davids don't know (through a prototype), you must cast each type you davids pass to the type the function expects. End of story. OpenSSL davids does not do this. This is not valid C whether or not the type davids sizes are the same. I don't think that's correct. If you are calling a function where a prototype is not in scope, the 'classic' promotion rules are used (float to double, char or short to int, etc). /r$ -- STSM, Senior Security Architect SOA Appliances Application Integration Middleware __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Thu, Nov 09, 2006, Richard Salz wrote: davids simple -- if you are going to call a function whose types you davids don't know (through a prototype), you must cast each type you davids pass to the type the function expects. End of story. OpenSSL davids does not do this. This is not valid C whether or not the type davids sizes are the same. I don't think that's correct. If you are calling a function where a prototype is not in scope, the 'classic' promotion rules are used (float to double, char or short to int, etc). Well just to add to the confusion... I've compiled up gcc4.2 myself from source just to see what is going on. Yes it does indeed give the reported issues but what is strange is that almost identical constructs (the safestack, and some related macros) don't even generate a warning. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
Thus spake Peter Waltenberg x is still just a pointer to data - so it's the same length in any case, all pointers to lvalues are the same length in C. The only issue there is whether it's aligned correctly - that's the programmers problem. It's not the only problem. Mixing something like char *(*d2i)(), and char ** IS problematic, since those aren't guaranteed to be the same length but as far as I can remember OpenSSL doesn't do that. ret=d2i(x,p,len); d2i is a function (prototype is unknown) but I've been told x is a pointer, p is a pointer, len is long. There's nothing indeterminate about the size of any of those. Incorrect. When the compiler encounters this statement, if there's no prototype for d2i() in scope, it is _required_ to act as if the prototype were: int d2i(int, int, int); Since you're passing two pointers and a long, this is simply invalid. It may work on some platforms, but it's undefined behavior and a compiler is free to do whatever it wants (including insert code to crash the program). C isn't a strongly typed language - there's no language requirement for an accurate function prototype or that the types of the arguments be correct - only that the size of them is correct. C89 had already deprecated non-prototyped functions. In C99, they're prohibited, so there is a requirement today. C is not the same language as C++, this instance should work with a C compiler - whether or not the compiler is passing the arguments via the stack or in registers. Again, not correct. There are implementations that pass ints, longs, and pointers all in different registers. Some platforms have specific registers for pointers, just like many have specific registers for floats/doubles. Others will have the top half of a long corrupted if sizeof(long)sizeof(int), which is common on many 64-bit machines. S Stephen Sprunk God does not play dice. --Albert Einstein CCIE #3723 God is an inveterate gambler, and He throws the K5SSSdice at every possible opportunity. --Stephen Hawking __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
Incorrect. When the compiler encounters this statement, if there's no prototype for d2i() in scope, it is _required_ to act as if the prototype were: int d2i(int, int, int); This is wrong. The usual integral promotions apply -- but only to integral parameters, not to ALL of them. /r$ -- STSM, Senior Security Architect SOA Appliances Application Integration Middleware __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
Stephen Sprunk wrote: Thus spake Peter Waltenberg x is still just a pointer to data - so it's the same length in any case, all pointers to lvalues are the same length in C. The only issue there is whether it's aligned correctly - that's the programmers problem. It's not the only problem. Mixing something like char *(*d2i)(), and char ** IS problematic, since those aren't guaranteed to be the same length but as far as I can remember OpenSSL doesn't do that. ret=d2i(x,p,len); d2i is a function (prototype is unknown) but I've been told x is a pointer, p is a pointer, len is long. There's nothing indeterminate about the size of any of those. Incorrect. When the compiler encounters this statement, if there's no prototype for d2i() in scope, it is _required_ to act as if the prototype were: int d2i(int, int, int); I believe this is incorrect. When there is no prototype in scope, I believe the compiler is _required_ to act as though there is a prototype that exactly matches the type and number of arguments in the function call after applying the usual arithmetic promotions. (I believe the intent was to preserve compatibility with old pre-prototype KR code. Assuming that all parameters are of type int would wreak havoc with old code, no?) - Ken __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
davids simple -- if you are going to call a function whose types you davids don't know (through a prototype), you must cast each type you davids pass to the type the function expects. End of story. OpenSSL davids does not do this. This is not valid C whether or not the type davids sizes are the same. I don't think that's correct. If you are calling a function where a prototype is not in scope, the 'classic' promotion rules are used (float to double, char or short to int, etc). /r$ Are you saying the call is legal even if the promotion rules do not result in calling the function with the exact types it expects? That conflicts massively with my understanding of the C language. As I understood it, calling a function without a prototype was precisely equivalent to declaring a prototype of the function with the exact parameter types passed (after promotion rules). Calling a function with the wrong pointer type *will* break in the face of optimization. That is why GCC 4.2 breaks when it tries to inline such function calls. Even if it wasn't a violation of C's function type rules, which I think it is, it's also a violation of C's aliasing rules (when the types are pointers). DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Thu, Nov 09, 2006, Dr. Stephen Henson wrote: On Thu, Nov 09, 2006, Richard Salz wrote: davids simple -- if you are going to call a function whose types you davids don't know (through a prototype), you must cast each type you davids pass to the type the function expects. End of story. OpenSSL davids does not do this. This is not valid C whether or not the type davids sizes are the same. I don't think that's correct. If you are calling a function where a prototype is not in scope, the 'classic' promotion rules are used (float to double, char or short to int, etc). Well just to add to the confusion... I've compiled up gcc4.2 myself from source just to see what is going on. Yes it does indeed give the reported issues but what is strange is that almost identical constructs (the safestack, and some related macros) don't even generate a warning. Which was mainly due to me not having the right versions of the safestack macros enabled but hey, its been a long week :-) Hmmm nasty. It will tolerate function pointer casting if a variable is used but I suspect that's more because the optimizer hasn't yet learned to stamp on it. OK, looks like inline calls is the way to go. Those at least have some advantages over the existing stuff. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
As I understood it, calling a function without a prototype was precisely
equivalent to declaring a prototype of the function with the exact
parameter
types passed (after promotion rules).
Nope. Calling a function without a prototype is precisely equivalent to
KR rules. If a function will be called in such KR style, and the
function prototype and/or definition have any non-promoted parameters, the
result is undefined. For example
void foo(short p1) { }
is *ILLEGAL* if foo will ever be called without a prototype in scope; p1
must be an int.
Calling a function with the wrong pointer type *will* break in the face
of
optimization. That is why GCC 4.2 breaks when it tries to inline such
function calls. Even if it wasn't a violation of C's function type
rules,
which I think it is, it's also a violation of C's aliasing rules (when
the
types are pointers).
Once KR is included, the situation becomes a lot less clear. Also, as I
read the thread on the GCC list, it looks like the situation is further
complicated by their desire to avoid an internal compiler error. Also**2,
I don't know what you mean by C's aliasing rules; to me that brings to
mind the ill-conceived 'noalias' qualifier.
If this is worth further discussion, a concrete example (code fragment)
would help me a lot.
/r$
--
STSM, Senior Security Architect
SOA Appliances
Application Integration Middleware
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
Once KR is included, the situation becomes a lot less clear. Also, as I
read the thread on the GCC list, it looks like the situation is further
complicated by their desire to avoid an internal compiler error.
Also**2,
I don't know what you mean by C's aliasing rules; to me that brings to
mind the ill-conceived 'noalias' qualifier.
If this is worth further discussion, a concrete example (code fragment)
would help me a lot.
/r$
What I mean by aliasing rules is that with only a few exceptions, a compiler
is allowed to assume that an access through a pointer of one type cannot
affect the value of a variable of a different type.
GCC's optimization in this area has gotten to the point where it bites a lot
of people, and I think it's the aliasing issue that's the problem for
OpenSSL, not really the function prototype issue. Right now, the GCC bug
with the most duplicates is due to aliasing errors.
The GCC not a bug page gives this example:
int main()
{
short a[2];
a[0]=0x;
a[1]=0x;
*(int *)a = 0x; /* violation of aliasing rules */
printf(%x %x\n, a[0], a[1]);
return 0;
}
The compiler has every right to print and , because under C rules,
no modification to an integer can change the value of a short.
However, this is also an example:
extern void foo();
int main(void)
{
int j=2;
foo((double *) j);
printf(%d\n, j);
return 0;
}
The compiler has every right to assume that 'j' still has the value '2' in
the 'printf' call because 'j' is local to 'main' and no modification through
a 'double *' pointer can possibly change the value of an 'int'.
GCC 403 warns about this (with flags: -O3 -Wall)
test.c:9: warning: dereferencing type-punned pointer will break
strict-aliasing rules
Line 9 is the call to 'foo'.
OpenSSL's PEM #define's do the same thing as this example.
However, if OpenSSL fixes its aliasing issues and *still* has a problem,
then I think you can complain to the GCC guys. I think the current GCC issue
can be tripped even in code which doesn't violate aliasing rules, but I
think you still have to violate the parameter passing rules.
So either way, it's not GCC's fault strictly speaking. Though GCC's behavior
is suboptimal, certainly. GCC will take care of its problems, but OpenSSL
should take care of its problems too. A lot of older code is getting bitten
by compilers that more heavily optimize and break code that uses illegal
type-punning.
DS
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On 11/9/06, David Schwartz [EMAIL PROTECTED] wrote:
Once KR is included, the situation becomes a lot less clear. Also, as I
read the thread on the GCC list, it looks like the situation is further
complicated by their desire to avoid an internal compiler error.
Also**2,
I don't know what you mean by C's aliasing rules; to me that brings to
mind the ill-conceived 'noalias' qualifier.
If this is worth further discussion, a concrete example (code fragment)
would help me a lot.
/r$
What I mean by aliasing rules is that with only a few exceptions, a compiler
is allowed to assume that an access through a pointer of one type cannot
affect the value of a variable of a different type.
GCC's optimization in this area has gotten to the point where it bites a lot
of people, and I think it's the aliasing issue that's the problem for
OpenSSL, not really the function prototype issue. Right now, the GCC bug
with the most duplicates is due to aliasing errors.
The GCC not a bug page gives this example:
int main()
{
short a[2];
a[0]=0x;
a[1]=0x;
*(int *)a = 0x; /* violation of aliasing rules */
printf(%x %x\n, a[0], a[1]);
return 0;
}
The compiler has every right to print and , because under C rules,
no modification to an integer can change the value of a short.
Ummm... could you please cite where in the standard (and which
standard within which) you're seeing that? That is completely
opposite of how every compiler I've ever used, across seven OSes and
five platforms, has acted -- and I would argue that it makes no sense.
Any memory location (pointer) can be seen as different things,
depending on the cast that is used. If you're worried, you can look
at the *(int a) = 0x as creating a const qualifier, but even
then the issue becomes murky -- if the pointer is cast to (void *) and
passed to another function, then the called function has no idea what
to look at it as without extra context, which could be wildly
inaccurate. (cast drops const qualifier is a warning, not an
error.)
However, this is also an example:
extern void foo();
int main(void)
{
int j=2;
foo((double *) j);
printf(%d\n, j);
return 0;
}
The compiler has every right to assume that 'j' still has the value '2' in
the 'printf' call because 'j' is local to 'main' and no modification through
a 'double *' pointer can possibly change the value of an 'int'.
A pointer to an address in memory should much more effectively and
efficiently be viewed as a union of all possible types, including void
* (which, by its definition, must be cast to another type before it
can be accessed).
The equivalent of the offending line would be
foo((double *)(void *) j);
since any access through a void pointer CAN change the value.
Thank you for showing me why I should always include
-fno-strict-aliasing in my gcc commandlines. It's broken, horribly.
-Kyle H
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
David Schwartz wrote: On Tue, Nov 07, 2006, Bernhard Rosenkraenzer wrote: gcc 4.2 no longer allows function casting - which is used rather heavily by openssl. (To make things worse, it compiles but inserts abort() statements resulting in app crashes). Ugh, I would've thought that flagging a compiliation error would make more sense rather that outputting bad code... It is perfectly reasonable (though certainly not optimal) for a compiler to output bad code when the code it's compiling invokes undefined behavior. Aren't they doing it when the code it's compiling _may_ invoke undefined behaviour on other platforms? This does not strike me as perfectly reasonable. You can see http://gcc.gnu.org/ml/gcc/2006-07/msg00037.html for a discussion on the gcc devel list. Is there a fix to make OpenSSL compatible with gcc 4.2/4.3 yet? No there isn't yet, some technique will be needed to make this still work by tweaking some of the macros. Does casting a function to (void *) first then the function cast work? Not if by work you mean anything more than accidentally happen to work on some platforms. There is no reason a 'void (*) (struct bar *);' needs to be even remotely like a 'void (*) (char *);'. Nevertheless, it is exactly like it on many processors. OpenSSL should be fixed to cast the parameters, not the functions. Casting the functions gives you type-safety. Casting the parameters means you can have any old crap in the parameters and you don't find out at compile-time. Look at this code: char * PEM_ASN1_read_bio(char *(*d2i)(),const char *name,BIO *bp,char **x, pem_password_cb *cb, void *u); #define PEM_read_X509(fp,x,cb,u) (X509 *)PEM_ASN1_read( \ (char *(*)())d2i_X509,PEM_STRING_X509,fp,(char **)x,cb,u) #define PEM_read_SSL_SESSION(fp,x,cb,u) (SSL_SESSION *)PEM_ASN1_read( \ (char *(*)())d2i_SSL_SESSION,PEM_STRING_SSL_SESSION,fp,(char **)x,cb,u) This is just wrong. A pointer to d2i_X509 and a pointer to d2i_SSL_SESSION are of different types. You cannot cast them into a common type and then write a function that can call either of them through such a pointer. The error is *NOT* from casting function pointers (like you must do to the return from 'dlsym'). It's from casting a pointer to a function of one type into a function of *another* type and then *calling* it through that pointer. But it gets cast back to the correct type before it is called. These casts are done the way they are to get type-safety. Removing that option strikes me as a bad thing. Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.links.org/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Wed, Nov 08, 2006, Ben Laurie wrote: But it gets cast back to the correct type before it is called. These casts are done the way they are to get type-safety. Removing that option strikes me as a bad thing. Yes and that happened to be a way that worked on all the compilers OpenSSL used at the time. The main purpose of the casting it to provide type safe checking at compile time and have no runtime overhead. If there is an alternative to achieve the same result (inline calls perhaps?) then we can have that as a compilation option. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Tuesday, 7. November 2006 15:54, Dr. Stephen Henson wrote: On Tue, Nov 07, 2006, Bernhard Rosenkraenzer wrote: gcc 4.2 no longer allows function casting - which is used rather heavily by openssl. (To make things worse, it compiles but inserts abort() statements resulting in app crashes). Ugh, I would've thought that flagging a compiliation error would make more sense rather that outputting bad code... Agreed... Unfortunately sometimes gcc developers' decisions make much sense from the point of the compiler implementor than from that of someone actually using it. Is there a fix to make OpenSSL compatible with gcc 4.2/4.3 yet? No there isn't yet, some technique will be needed to make this still work by tweaking some of the macros. Does casting a function to (void *) first then the function cast work? Yes, but only in a really really ugly way. I've attached a patch that make it compile (and work), but I can't say I like it. I've tried all sorts of other things (like putting the void* cast directly into the defines, or even adding a (char*) cast and running +1-1 type arithmetics on it), but gcc's optimizer is too smart and throws it away, and errors out just as if it wasn't there), but the attached patch is the only quick fix that works. I hope someone can come up with a better idea... -- Get your own Linux distribution -- http://www.yold.org/ --- openssl-0.9.8d/crypto/asn1/asn1.h.ark 2006-11-08 19:03:36.0 +0100 +++ openssl-0.9.8d/crypto/asn1/asn1.h 2006-11-08 19:32:10.0 +0100 @@ -902,23 +902,26 @@ /* Used to implement other functions */ void *ASN1_dup(i2d_of_void *i2d, d2i_of_void *d2i, char *x); +static void * __attribute__((unused)) __ASN1_dup=openssl_fcast(ASN1_dup); #define ASN1_dup_of(type,i2d,d2i,x) \ - ((type *(*)(I2D_OF(type),D2I_OF(type),type *))openssl_fcast(ASN1_dup))(i2d,d2i,x) + ((type *(*)(I2D_OF(type),D2I_OF(type),type *))__ASN1_dup)(i2d,d2i,x) #define ASN1_dup_of_const(type,i2d,d2i,x) \ - ((type *(*)(I2D_OF_const(type),D2I_OF(type),type *))openssl_fcast(ASN1_dup))(i2d,d2i,x) + ((type *(*)(I2D_OF_const(type),D2I_OF(type),type *))__ASN1_dup)(i2d,d2i,x) void *ASN1_item_dup(const ASN1_ITEM *it, void *x); #ifndef OPENSSL_NO_FP_API void *ASN1_d2i_fp(void *(*xnew)(void), d2i_of_void *d2i, FILE *in, void **x); +static void * __attribute__((unused)) __ASN1_d2i_fp=openssl_fcast(ASN1_d2i_fp); #define ASN1_d2i_fp_of(type,xnew,d2i,in,x) \ - ((type *(*)(type *(*)(void),D2I_OF(type),FILE *,type **))openssl_fcast(ASN1_d2i_fp))(xnew,d2i,in,x) + ((type *(*)(type *(*)(void),D2I_OF(type),FILE *,type **))__ASN1_d2i_fp)(xnew,d2i,in,x) void *ASN1_item_d2i_fp(const ASN1_ITEM *it, FILE *in, void *x); int ASN1_i2d_fp(i2d_of_void *i2d,FILE *out,void *x); +static void * __attribute__((unused)) __ASN1_i2d_fp=openssl_fcast(ASN1_i2d_fp); #define ASN1_i2d_fp_of(type,i2d,out,x) \ - ((int (*)(I2D_OF(type),FILE *,type *))openssl_fcast(ASN1_i2d_fp))(i2d,out,x) + ((int (*)(I2D_OF(type),FILE *,type *))__ASN1_i2d_fp)(i2d,out,x) #define ASN1_i2d_fp_of_const(type,i2d,out,x) \ - ((int (*)(I2D_OF_const(type),FILE *,type *))openssl_fcast(ASN1_i2d_fp))(i2d,out,x) + ((int (*)(I2D_OF_const(type),FILE *,type *))__ASN1_i2d_fp)(i2d,out,x) int ASN1_item_i2d_fp(const ASN1_ITEM *it, FILE *out, void *x); int ASN1_STRING_print_ex_fp(FILE *fp, ASN1_STRING *str, unsigned long flags); #endif @@ -927,14 +930,16 @@ #ifndef OPENSSL_NO_BIO void *ASN1_d2i_bio(void *(*xnew)(void), d2i_of_void *d2i, BIO *in, void **x); +static void * __attribute__((unused)) __ASN1_d2i_bio=openssl_fcast(ASN1_d2i_bio); #define ASN1_d2i_bio_of(type,xnew,d2i,in,x) \ - ((type *(*)(type *(*)(void),D2I_OF(type),BIO *,type **))openssl_fcast(ASN1_d2i_bio))(xnew,d2i,in,x) + ((type *(*)(type *(*)(void),D2I_OF(type),BIO *,type **))__ASN1_d2i_bio)(xnew,d2i,in,x) void *ASN1_item_d2i_bio(const ASN1_ITEM *it, BIO *in, void *x); int ASN1_i2d_bio(i2d_of_void *i2d,BIO *out, unsigned char *x); +static void * __attribute__((unused)) __ASN1_i2d_bio=openssl_fcast(ASN1_i2d_bio); #define ASN1_i2d_bio_of(type,i2d,out,x) \ - ((int (*)(I2D_OF(type),BIO *,type *))openssl_fcast(ASN1_i2d_bio))(i2d,out,x) + ((int (*)(I2D_OF(type),BIO *,type *))__ASN1_i2d_bio)(i2d,out,x) #define ASN1_i2d_bio_of_const(type,i2d,out,x) \ - ((int (*)(I2D_OF_const(type),BIO *,const type *))openssl_fcast(ASN1_i2d_bio))(i2d,out,x) + ((int (*)(I2D_OF_const(type),BIO *,const type *))__ASN1_i2d_bio)(i2d,out,x) int ASN1_item_i2d_bio(const ASN1_ITEM *it, BIO *out, void *x); int ASN1_UTCTIME_print(BIO *fp,ASN1_UTCTIME *a); int ASN1_GENERALIZEDTIME_print(BIO *fp,ASN1_GENERALIZEDTIME *a); @@ -977,8 +982,9 @@ void *ASN1_item_unpack(ASN1_STRING *oct, const ASN1_ITEM *it); ASN1_STRING *ASN1_pack_string(void *obj, i2d_of_void *i2d, ASN1_OCTET_STRING **oct); +static void * __attribute__((unused)) __ASN1_pack_string=openssl_fcast(ASN1_pack_string); #define ASN1_pack_string_of(type,obj,i2d,oct) \ - ((ASN1_STRING *(*)(type
RE: OpenSSL breaks with gcc 4.2
But it gets cast back to the correct type before it is called. These
casts are done the way they are to get type-safety. Removing that option
strikes me as a bad thing.
It does not. Look closely at how these functions work:
char *PEM_ASN1_read_bio(char *(*d2i)(), const char *name, BIO *bp, char **x,
pem_password_cb *cb, void *u)
{
unsigned char *p=NULL,*data=NULL;
long len;
char *ret=NULL;
if (!PEM_bytes_read_bio(data, len, NULL, name, bp, cb, u))
return NULL;
p = data;
ret=d2i(x,p,len);
if (ret == NULL)
PEMerr(PEM_F_PEM_ASN1_READ_BIO,ERR_R_ASN1_LIB);
OPENSSL_free(data);
return(ret);
}
Please tell me how the compiler knows what type 'x' should be passed
as. If
you pass a pointer to a function as 'd2i' whose first type is not defined as
a 'char **', you get undefined behavior -- how can the compiler possibly use
the correct type's passing rules when it thinks the function takes a 'char
**' and it actuall takes an 'X509 **'.
OpenSSL does *not* cast the function back to the correct (exact) type
before it calls it. Neither does it cast the function's parameters to the
right type. As a result, the code only works by luck. In the case of
'PEM_read_X509', it works if 'char **' and 'X509 **' happen to have the same
function parameter rules. Nothing requires this to be the case.
There is now way the compiler can know how to properly pass 'x' to
'd2i'. A
function cannot call another function whose parameter types it does not know
and can vary.
DS
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
Please tell me how the compiler knows what
type 'x' should be passed as. If
you pass a pointer to a function as 'd2i' whose first type is not defined
as
a 'char **', you get undefined behavior -- how can the compiler possibly
use
the correct type's passing rules when it thinks the function takes a 'char
**' and it actuall takes an 'X509 **'.
x is still just a pointer to data -
so it's the same length in any case, all pointers to lvalues are the same
length in C. The only issue there is whether it's aligned correctly - that's
the programmers problem.
Mixing something like char *(*d2i)(),
and char ** IS problematic, since those aren't guaranteed to be the same
length but as far as I can remember OpenSSL doesn't do that.
ret=d2i(x,p,len);
d2i is a function (prototype is unknown)
but I've been told x is a pointer,
p is a pointer, len is long. There's nothing indeterminate about the
size of any of those.
C isn't a strongly typed language -
there's no language requirement for an accurate function prototype or that
the types of the arguments be correct - only that the size of them is correct.
C is not the same language as C++, this
instance should work with a C compiler - whether or not the compiler is
passing the arguments via the stack or in registers.
Peter
Peter Waltenberg
David Schwartz
[EMAIL PROTECTED]
Sent by: [EMAIL PROTECTED]
09/11/2006 06:47 AM
Please respond to
openssl-dev
To
openssl-dev@openssl.org
cc
Subject
RE: OpenSSL breaks with gcc
4.2
But it gets cast back to the correct type before it is called. These
casts are done the way they are to get type-safety. Removing that
option
strikes me as a bad thing.
It does not. Look closely at how these functions work:
char *PEM_ASN1_read_bio(char *(*d2i)(), const char *name, BIO *bp, char
**x,
pem_password_cb *cb, void *u)
{
unsigned char *p=NULL,*data="">
long len;
char *ret=NULL;
if (!PEM_bytes_read_bio(data, len,
NULL, name, bp, cb, u))
return NULL;
p = data;
ret=d2i(x,p,len);
if (ret == NULL)
PEMerr(PEM_F_PEM_ASN1_READ_BIO,ERR_R_ASN1_LIB);
OPENSSL_free(data);
return(ret);
}
Please tell me how the compiler knows what type 'x' should be passed as.
If
you pass a pointer to a function as 'd2i' whose first type is not defined
as
a 'char **', you get undefined behavior -- how can the compiler possibly
use
the correct type's passing rules when it thinks the function takes a 'char
**' and it actuall takes an 'X509 **'.
OpenSSL does *not* cast the function back to the correct (exact) type
before it calls it. Neither does it cast the function's parameters to the
right type. As a result, the code only works by luck. In the case of
'PEM_read_X509', it works if 'char **' and 'X509 **' happen to have the
same
function parameter rules. Nothing requires this to be the case.
There is now way the compiler can know how to properly pass 'x' to 'd2i'.
A
function cannot call another function whose parameter types it does not
know
and can vary.
DS
__
OpenSSL Project
http://www.openssl.org
Development Mailing List
openssl-dev@openssl.org
Automated List Manager
[EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
Dr. Stephen Henson wrote: On Wed, Nov 08, 2006, Ben Laurie wrote: But it gets cast back to the correct type before it is called. These casts are done the way they are to get type-safety. Removing that option strikes me as a bad thing. Yes and that happened to be a way that worked on all the compilers OpenSSL used at the time. The main purpose of the casting it to provide type safe checking at compile time and have no runtime overhead. If there is an alternative to achieve the same result (inline calls perhaps?) then we can have that as a compilation option. Sure, I have no objection to replacing the kludge. So long as we don't lose compile-time type-safety. -- http://www.apache-ssl.org/ben.html http://www.links.org/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Thu, Nov 09, 2006, Peter Waltenberg wrote: Please tell me how the compiler knows what type 'x' should be passed as. If you pass a pointer to a function as 'd2i' whose first type is not defined as a 'char **', you get undefined behavior -- how can the compiler possibly use the correct type's passing rules when it thinks the function takes a 'char **' and it actuall takes an 'X509 **'. x is still just a pointer to data - so it's the same length in any case, all pointers to lvalues are the same length in C. The only issue there is whether it's aligned correctly - that's the programmers problem. Mixing something like char *(*d2i)(), and char ** IS problematic, since those aren't guaranteed to be the same length but as far as I can remember OpenSSL doesn't do that. ret=d2i(x,p,len); d2i is a function (prototype is unknown) but I've been told x is a pointer, p is a pointer, len is long. There's nothing indeterminate about the size of any of those. C isn't a strongly typed language - there's no language requirement for an accurate function prototype or that the types of the arguments be correct - only that the size of them is correct. C is not the same language as C++, this instance should work with a C compiler - whether or not the compiler is passing the arguments via the stack or in registers. The d2i() case BTW can be replaced in most cases by an equivalent which uses ASN1_ITEM which doesn't have this problem. There are a few ASN1_ITEM replacements for some functions but nothing for PEM (yet). Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Wed, Nov 08, 2006 at 12:47:03PM -0800, David Schwartz wrote:
But it gets cast back to the correct type before it is called. These
casts are done the way they are to get type-safety. Removing that option
strikes me as a bad thing.
It does not. Look closely at how these functions work:
char *PEM_ASN1_read_bio(char *(*d2i)(), const char *name, BIO *bp, char **x,
pem_password_cb *cb, void *u)
That seems to be 0.9.7 code, in 0.9.8 it looks like:
void *PEM_ASN1_read_bio(d2i_of_void *d2i, const char *name, BIO *bp, void **x,
pem_password_cb *cb, void *u)
And we have:
typedef void *d2i_of_void(void **,const unsigned char **,long);
The example mentioned in on the gcc list seems to be:
X509 *PEM_read_X509_AUX(FILE *fp, X509 **x, pem_password_cb *cb, void *u)
{
return(((X509 *(*)(X509 *(*)(X509 **,const unsigned char **,long),char
*,FILE *,X509 **,pem_password_cb *,void
*))((openssl_fptr)PEM_ASN1_read))(d2i_X509_AUX, TRUSTED
CERTIFICATE,fp,x,cb,u));
}
We have:
void * PEM_ASN1_read(d2i_of_void *d2i, const char *name, FILE *fp, void **x,
pem_password_cb *cb, void *u);
What happens here is that PEM_ASN1_read() first gets cast to an
openssl_fptr which is:
typedef void (*openssl_fptr)(void);
And then to an:
X509 *(*)(X509 *(*)(X509 **,const unsigned char **,long),char *,FILE
*,X509 **,pem_password_cb *,void *)
And we have:
X509 * d2i_X509_AUX(X509 **a,const unsigned char **pp,long length);
What I think the problem is, is that you use d2i_of_void *d2i while
you mean d2i_of_void d2i.
d2i_of_void *d2i would be a pointer to a pointer to a function, while
d2i_of_void d2i would be just a pointer to a function.
Kurt
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
x is still just a pointer to data - so it's the same length in any case, all pointers to lvalues are the same length in C. The only issue there is whether it's aligned correctly - that's the programmers problem. Length is not the issue. There is no rule that says that two types must be passed the same way just because they're the same length. For example, some platforms may pass a 64-bit floating point number in a floating point register but pass a 64-bit integer in an integer register. All you can do is assume that it will work because it has always worked, until it stops working. Mixing something like char *(*d2i)(), and char ** IS problematic, since those aren't guaranteed to be the same length but as far as I can remember OpenSSL doesn't do that. If they are not the same length, it is not likely to work. That doesn't mean it's legal if they are the same length. ret=d2i(x,p,len); d2i is a function (prototype is unknown) but I've been told x is a pointer, p is a pointer, len is long. There's nothing indeterminate about the size of any of those. The issue is not size, it's type. C isn't a strongly typed language - there's no language requirement for an accurate function prototype or that the types of the arguments be correct - only that the size of them is correct. You are correct, but that's not the issue. The issue is this simple -- if you are going to call a function whose types you don't know (through a prototype), you must cast each type you pass to the type the function expects. End of story. OpenSSL does not do this. This is not valid C whether or not the type sizes are the same. C is not the same language as C++, this instance should work with a C compiler - whether or not the compiler is passing the arguments via the stack or in registers. Can you cite anything in the C standard that says this is allowed? As I understand it, you must pass a function precisely the types it expects, or use a prototype to force the compiler to cast them. Either way, the exact type must be known where the call is compiled. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
In message [EMAIL PROTECTED] on Wed, 8 Nov 2006 21:59:19 -0800, David Schwartz [EMAIL PROTECTED] said: davids You are correct, but that's not the issue. The issue is this davids simple -- if you are going to call a function whose types you davids don't know (through a prototype), you must cast each type you davids pass to the type the function expects. End of story. OpenSSL davids does not do this. This is not valid C whether or not the type davids sizes are the same. So basically, you're saying that KR-style functions (non-prototyped) aren't handled in a pre-prototype manner any more? That's a fairly extreme change of how C used to work. While I understand such a change, it's going to break quite a lot of things that still have non-prototyped functions. Can you provide something in the C standard that supports this change? Cheers, Richard - Please consider sponsoring my work on free software. See http://www.free.lp.se/sponsoring.html for details. -- Richard Levitte [EMAIL PROTECTED] http://richard.levitte.org/ When I became a man I put away childish things, including the fear of childishness and the desire to be very grown up. -- C.S. Lewis __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
OpenSSL breaks with gcc 4.2
gcc 4.2 no longer allows function casting - which is used rather heavily by openssl. (To make things worse, it compiles but inserts abort() statements resulting in app crashes). You can see http://gcc.gnu.org/ml/gcc/2006-07/msg00037.html for a discussion on the gcc devel list. Is there a fix to make OpenSSL compatible with gcc 4.2/4.3 yet? __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: OpenSSL breaks with gcc 4.2
On Tue, Nov 07, 2006, Bernhard Rosenkraenzer wrote: gcc 4.2 no longer allows function casting - which is used rather heavily by openssl. (To make things worse, it compiles but inserts abort() statements resulting in app crashes). Ugh, I would've thought that flagging a compiliation error would make more sense rather that outputting bad code... You can see http://gcc.gnu.org/ml/gcc/2006-07/msg00037.html for a discussion on the gcc devel list. Is there a fix to make OpenSSL compatible with gcc 4.2/4.3 yet? No there isn't yet, some technique will be needed to make this still work by tweaking some of the macros. Does casting a function to (void *) first then the function cast work? Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: OpenSSL breaks with gcc 4.2
On Tue, Nov 07, 2006, Bernhard Rosenkraenzer wrote: gcc 4.2 no longer allows function casting - which is used rather heavily by openssl. (To make things worse, it compiles but inserts abort() statements resulting in app crashes). Ugh, I would've thought that flagging a compiliation error would make more sense rather that outputting bad code... It is perfectly reasonable (though certainly not optimal) for a compiler to output bad code when the code it's compiling invokes undefined behavior. You can see http://gcc.gnu.org/ml/gcc/2006-07/msg00037.html for a discussion on the gcc devel list. Is there a fix to make OpenSSL compatible with gcc 4.2/4.3 yet? No there isn't yet, some technique will be needed to make this still work by tweaking some of the macros. Does casting a function to (void *) first then the function cast work? Not if by work you mean anything more than accidentally happen to work on some platforms. There is no reason a 'void (*) (struct bar *);' needs to be even remotely like a 'void (*) (char *);'. OpenSSL should be fixed to cast the parameters, not the functions. Look at this code: char * PEM_ASN1_read_bio(char *(*d2i)(),const char *name,BIO *bp,char **x, pem_password_cb *cb, void *u); #define PEM_read_X509(fp,x,cb,u) (X509 *)PEM_ASN1_read( \ (char *(*)())d2i_X509,PEM_STRING_X509,fp,(char **)x,cb,u) #define PEM_read_SSL_SESSION(fp,x,cb,u) (SSL_SESSION *)PEM_ASN1_read( \ (char *(*)())d2i_SSL_SESSION,PEM_STRING_SSL_SESSION,fp,(char **)x,cb,u) This is just wrong. A pointer to d2i_X509 and a pointer to d2i_SSL_SESSION are of different types. You cannot cast them into a common type and then write a function that can call either of them through such a pointer. The error is *NOT* from casting function pointers (like you must do to the return from 'dlsym'). It's from casting a pointer to a function of one type into a function of *another* type and then *calling* it through that pointer. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
