Hi, OpenSSL developers. I try to compile openssl-0.9.8-beta3 on Solaris 8, using 64-bit Sun Forte compiler (cc: Sun C 5.7 2005/01/07)
When doing 'make test' I get a core dump. But after applying a simple patch, I am unable to track down the bug any further. The Solaris PRNG patch 112438 (as mentioned in the FAQ) is installed. $ ./Configure solaris64-sparcv9-cc shared $ make $ make test [...] ----- Country Name (2 letter code) [AU]:AU Organization Name (eg, company) []:Dodgy Brothers Common Name (eg, YOUR name) []:Dodgy CA convert the certificate request into a self signed certificate using 'x509' unable to load 'random state' This means that the random number generator has not been seeded with much random data. Consider setting the RANDFILE environment variable to point at a file that 'random' data can be kept in (the file will be overwritten). Signature ok subject=/C=AU/O=Dodgy Brothers/CN=Dodgy CA Getting Private key Segmentation Fault - core dumped error using 'x509' to self sign a certificate request make[1]: *** [test_ss] Error 1 make[1]: Leaving directory `/opt/home/jtv/apps/openssl-0.9.8-beta3/test' make: *** [tests] Error 2 core dump stack trace: program terminated by signal SEGV (no mapping at the fault address) 0xffffffff7f0af3cc: BN_BLINDING_set_thread_id+0x0004: stx %o1, [%o0 + 32] (dbx) where =>[1] BN_BLINDING_set_thread_id(0x0, 0x5b0c, 0x1001dcca0, 0x0, 0xffffffff7f2881d8, 0x30), at 0xffffffff7f0af3cc [2] RSA_setup_blinding(0x0, 0x1001eb000, 0x1001eb000, 0x1b87c4,0x0, 0x1001dcc10), at 0xffffffff7f0cfc44 [3] RSA_eay_private_encrypt(0x0, 0x1001aeaf0, 0x1001eb000,0x1001dcd00, 0x1, 0x1001e91a0), at 0xffffffff7f0cd6b8 [4] RSA_sign(0x1001aeaf0, 0xffffffff7fffdf70, 0x23,0x1001ea470, 0xffffffff7fffe0ac, 0x1001dcd00), at0xffffffff7f0d007c [5] EVP_SignFinal(0xffffffff7fffe088, 0x1001ea470,0xffffffff7fffe0ac, 0x100190860, 0xffffffff7fffdf70, 0x18fe9c),at 0xffffffff7f0f8414 [6] ASN1_item_sign(0x1001ea470, 0x100194620, 0x100194540,0x1001dbbf0, 0x40, 0x100190860), at 0xffffffff7f1049e4 [7] X509_sign(0x1001ae480, 0x100190860, 0xffffffff7f2a7f50,0x100187e68, 0x15895c, 0x3c00), at 0xffffffff7f12f8ac [8] sign(0x1001ae480, 0x100190860, 0x1e, 0x0,0xffffffff7f2a7f50, 0x5c00), at 0x100031dc0 [9] x509_main(0x2bc8, 0xffffffff7ffff19d, 0x0, 0x100169c68,0x0, 0x1001800e0), at 0x1000310dc [10] do_cmd(0x10017fe10, 0x10, 0xffffffff7fffed70,0xffffffff7ffff128, 0x158ed4, 0xffffffff7fffed70), at0x100010dcc [11] main(0x10017fe10, 0xffffffff7fffed68, 0x100176a68,0x100176a90, 0xffffffff7e7bf5f0, 0x1001d3450), at 0x100010b18 Then I applied the following patch to prevent the core dump: diff -ur openssl-0.9.8-beta3-orig/crypto/rsa/rsa_lib.c openssl-0.9.8-beta3-work/crypto/rsa/rsa_lib.c --- openssl-0.9.8-beta3-orig/crypto/rsa/rsa_lib.c 2005-05-11 03:45:33.000000000 +0000 +++ openssl-0.9.8-beta3-work/crypto/rsa/rsa_lib.c 2005-06-01 12:31:33.490002000 +0000 @@ -402,6 +402,11 @@ ret = BN_BLINDING_create_param(NULL, e, rsa->n, ctx, rsa->meth->bn_mod_exp, rsa->_method_mod_n); + if (!ret) + { + RSAerr(RSA_F_RSA_SETUP_BLINDING, ERR_R_MALLOC_FAILURE); + goto err; + } BN_BLINDING_set_thread_id(ret, CRYPTO_thread_id()); err: BN_CTX_end(ctx); And then compiled again: $ ./Configure solaris64-sparcv9-cc shared $ make $ make test [...] convert the certificate request into a self signed certificate using 'x509' unable to load 'random state' This means that the random number generator has not been seeded with much random data. Consider setting the RANDFILE environment variable to point at a file that 'random' data can be kept in (the file will be overwritten). Signature ok subject=/C=AU/O=Dodgy Brothers/CN=Dodgy CA Getting Private key 29274:error:24064064:random number generator:SSLEAY_RAND_BYTES:PRNG not seeded:md_rand.c:503:You need to read the OpenSSL FAQ, http://www.openssl.org/support/faq.html 29274:error:04088041:rsa routines:RSA_setup_blinding:malloc failure:rsa_lib.c:407: 29274:error:04066044:rsa routines:RSA_EAY_PRIVATE_ENCRYPT:internal error:rsa_eay.c:364: 29274:error:0D0C3006:asn1 encoding routines:ASN1_item_sign:EVP lib:a_sign.c:276: error using 'x509' to self sign a certificate request make[1]: *** [test_ss] Error 1 make[1]: Leaving directory `/opt/home/jtv/apps/openssl-0.9.8-beta3-work/test' make: *** [tests] Error 2 It seemes to be some kind of problem with RNG seeding. I tried to set RANDFILE, but that didn't help. Is it possible to turn on some RNG debugging? Regards, -- Jostein Tveit <[EMAIL PROTECTED]> ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]