Hello, I'm trying to validate a sign of message digest, sent over a protected channel. On one node digest has been generated by using RSA_sign(NID_sha1, digest, SHA_DIGEST_LENGTH, out, &outLen, rsaPrivKey), where rsaPrivKey is RSA private key extracted from PEM enceded file with PEM_read_RSAPrivateKey; on the other node sign is verified with RSA_verify(NID_sha1 ,digest, SHA_DIGEST_LENGTH, out, outLen, rsaPubKey), where rsaPubKey is RSA public key extracted from certificate.
RSA_verify always returns 0 and I get the following output: Error checking RSA sign: error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01 Error checking RSA sign: error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed Received signed digest message is OK and I guess it has PKCS padding (Does RSA_sign care about it?). Any idea why does it fails? I also noticed that if I generate sign by using openssl rsautl, sign is different and has different size. I really appreciete your help ;) Thank you
