At one point in time, RSA_PKCS1_PADDING was evidently #defined as '11',
the size in bytes of the extra room needed for PKCS1 padding in an RSA
block. In the current CVS version of OpenSSL it is #defined to 1 and
is just used as a selector in switch statements. Except in rsa_sign.c:
if(type == NID_md5_sha1) {
...
i = SSL_SIG_LENGTH;
} else {
...
i=i2d_X509_SIG(&sig,NULL);
}
j=RSA_size(rsa);
if ((i-RSA_PKCS1_PADDING) > j)
...
Even if RSA_PKCS1_PADDING is replaced with 11, the logic is still wrong
here I believe. It's if the hash *plus* the pad is greater than the
keysize that you run into problems.
If I'm completely missing the point of this check, I'd be interested in
what the real reason for it is... muddling through this stuff makes my
brain hurt.
Thanks,
Eric
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
