Hi, This is a resend [1], please redirect me if I am off-topic.
while modifying some IKE code to use OpenSSL in order to have RSA signature done by engines (support via opensc on Linux, more specifically eGate and eToken PRO devices), I encountered the following issue. RSA_sign() has a 'type' argument that describes the hash that was used for generating the data. It is used to add digest info before padding and private key encryption occur to complete the signature. There is a specific case in the code, namely for SSL which does not use a digest info but a fixed length 36 bytes hash value (concatenation of md5 and sha1 hashes). IKEv1 has the same kind of requirement. RFC 2409 has the following in Section 5.1: Since the hash algorithm used is already known there is no need to encode its OID into the signature. In addition, there is no binding between the OIDs used for RSA signatures in PKCS #1 and those used in this document. Therefore, RSA signatures MUST be encoded as a private key encryption in PKCS #1 format and not as a signature in PKCS #1 format (which includes the OID of the hash algorithm). DSS signatures MUST be encoded as r followed by s. Note that I cannot call RSA_private_encrypt() as a workaround because the devices do not support it (even if on tokens, I have added specific keys that allow decryption) and it does not seem the expected way to do it. At the moment, I am using NID_md5_sha1 type for RSA_sign() and have temporarily deactivated the length check (36 bytes) in libp11 code but this is also a temporary workaround. What would be the correct way to handle that issue? Initially, I wanted to add a new NID_IKE_digest type that would make RSA_sign() behave like with NID_md5_sha1 but with a relaxed check on the length. Because the removal of the length check basically provides access to private key encryption via signature primitive, I thought I should ask before. Comments welcome. Cheers, a+ [1]: http://thread.gmane.org/gmane.comp.encryption.openssl.devel/14093 ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
