That's not the problem, I think, because if the only instruction of the
function is:
int rsa_keygen (RSA *rsa, int bits, BIGNUM *e, BN_GENCB *cb) {
return 1;
}
I get the "segmentation fault" when the rsa_keygen ends. But if I return 0
or -1 (errors) OpenSSL reports me "error in genrsa" and call ends ok (no
segmentation fault or something like that).
2010/9/22 Christian Hohnstaedt <christ...@hohnstaedt.de>
> On Wed, Sep 22, 2010 at 03:41:30PM +0200, Nacho ?lvarez wrote:
> > Hello everybody
> >
> > Several months ago I developed an OpenSSL PKCS#11 engine for Windows XP
> and
> > it worked ok (it was compiled with MinGW).
> > Now I'm trying to compile it on Linux (Debian 5 with GCC 4.3.1) but I
> have
> > the next problem:
> >
> > In the overwritten function *rsa_keygen* (whose signature is *int
> rsa_keygen
> > (RSA *rsa, int bits, BIGNUM *e, BN_GENCB *cb)*) always when the function
> > ends (after the return instrucction) System reports me *Segmentation
> Fault*.
>
> You overwrote you return address on the stack most probably by
> accessing a locally declared array out of bounds.
>
> Cheers
>
> Christian
> ______________________________________________________________________
> OpenSSL Project http://www.openssl.org
> User Support Mailing List openssl-users@openssl.org
> Automated List Manager majord...@openssl.org
>
