Hello,
I have been trying to decrypt the "client pre master secret" sent in a SSL
handshake using the server's private key with OpenSSL's Crypto library. But
RSA_private_decrypt() always returns -1!
I have set up a test apache2 server with modssl and have generated the
server's private key and certificate. I sniffed the packets off the network
from a single SSL session using a single TCP connection between a client and
my test server, and now I am trying to decrypt the "client pre-master
secret".
In the code below I have opened the server's private key file, and read it
into a RSA object using PEM_read_RSAPrivateKey(). The RSA_check_key()
function does not return any errors. I pass this key with my encrypted data,
its length, a buffer to hold the decrypted data, and the padding setting to
RSA_private_decrypt() but this always returns -1.
I have checked to make sure my encrypted and decrypted buffers are correct,
and the length is correct. Apparently the RSA key struct is correct since no
errors came from RSA_check_key(), and I am using the only private key for
the server, so I can't be using the wrong key. I'm pretty sure the padding
is RSA_PKCS1_PADDING, is this correct for apache2 w/ modssl using rsa
handshakes for ssl? I have also tried the RSA_PKCS1_OAEP_PADDING and the
RSA_SSLv3_PADDING, and RSA_NO_PADDING without succes.
The cipher suite used was TLS_RSA_WITH_RC4_128_MD5.
The length of the encrypted-client-pre-master-secret is 258 bytes, when
decrypted it should be 48 bytes.
Can anyone see a step I'm missing, anyone know what's going on here?
Thanks!
Will
.
.
.
//Steps
//1. Using serverPrivateKey, decrypt the client_pre_master_secret
//1.1 Open server's private key file
if((fp = fopen("server.key", "rb")) == NULL)
{
printf("Cannot open server key file.\n");
return;
}
//1.2 Generate RSA struct from private key file
PEM_read_RSAPrivateKey(fp, &pRsa, NULL, NULL);
//1.3 Check for succesfull key generation
if(RSA_check_key(pRsa) != 1)
{
printf("RSA_check_key(): PrivateKey check failed\n");
return;
}
//1.4 Using Private RSA Key, decode the client_pre_master_secret
check = RSA_private_decrypt(clientPreMasterLength, toDecrypt,
decryptedPreMasterSecret, pRsa, RSA_PKCS1_PADDING);
if(check == -1)
{
printf("RSA_private_decrypt() failed");
exit(1);
}
.
.
.
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
