RSA_Verify error:04077064:rsa routines:RSA_verify:algorithm mismatch

2011-09-16 Thread brajan

hi can any one tell whats wrong i am doing in this code...

I sign the Message using Java IBMFIPS compliant the code to sign the message
is

//Signs the hash of each chunk and adds it to the Message Header
//data is 256Kb length  
//Get private key function is created by me to read the DER key formated
file
GetPrivateKey privkey = new GetPrivateKey();   
Signature genSign = Signature.getInstance(SHA1withRSA,IBMJCEFIPS);
genSign.initSign(privkey.get());
genSign.update(data.getBytes());
byte[] byteSignedData = genSign.sign();

Get Private Key function code

   File privateKeyFile = new File(Certificates +
File.separator+mykey.der);
byte[] encodedKey = new byte[(int)privateKeyFile.length()];
new FileInputStream(privateKeyFile).read(encodedKey);
PKCS8EncodedKeySpec privateKeySpec = new
PKCS8EncodedKeySpec(encodedKey);
KeyFactory kf = KeyFactory.getInstance(RSA,IBMJCEFIPS);
RSAPrivateCrtKey privatekey = (RSAPrivateCrtKey)   
.generatePrivate(privateKeySpec);
return privatekey;

I am verifying this signature using openssl 0.9.8g, RSA_Verify() function.
-- 
View this message in context: 
http://old.nabble.com/RSA_Verify-error%3A04077064%3Arsa-routines%3ARSA_verify%3Aalgorithm-mismatch-tp32478403p32478403.html
Sent from the OpenSSL - User mailing list archive at Nabble.com.

__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org


RE: RSA_Verify error:04077064:rsa routines:RSA_verify:algorithm mismatch

2011-09-16 Thread Dave Thompson
 From: owner-openssl-us...@openssl.org On Behalf Of brajan
 Sent: Friday, 16 September, 2011 08:07

 I sign the Message using Java IBMFIPS compliant [provider]
 the code to sign the message is
wrapping mostly fixed
 GetPrivateKey privkey = new GetPrivateKey();   
 Signature genSign = Signature.getInstance(SHA1withRSA,IBMJCEFIPS);
 genSign.initSign(privkey.get());
 genSign.update(data.getBytes());
 byte[] byteSignedData = genSign.sign();
 
 Get Private Key function code
 
 File privateKeyFile = new File(Certificates +
File.separator+mykey.der);
 byte[] encodedKey = new byte[(int)privateKeyFile.length()];
 new FileInputStream(privateKeyFile).read(encodedKey);

This can leak the file open (descriptor). No problem for a 
run-and-done utility, but may be for a long-running server 
depending on when and how this gets garbage-collected.

Also, using FIPS crypto but having privatekey in a clear file 
doesn't really make security sense, but that's your concern.
Perhaps this is only a test environment.

 PKCS8EncodedKeySpec privateKeySpec = new PKCS8EncodedKeySpec(encodedKey);
 KeyFactory kf = KeyFactory.getInstance(RSA,IBMJCEFIPS);
 RSAPrivateCrtKey privatekey = (RSAPrivateCrtKey)   
 .generatePrivate(privateKeySpec);

Obviously that should be kf.generatePrivate 

 return privatekey;
 
 I am verifying this signature using openssl 0.9.8g, 
 RSA_Verify() function.

0.9.8g is four years old. I don't recall (and won't 
search through to see) if any of the things fixed 
since then would affect this basic lowlevel usage;
it's less likely than other areas but not impossible.

I don't have that IBM provider, but using the Sun provider 
it works for me; you are verifying as NID_sha1 right?

Maybe the IBM provider is encoding the algid wrong or oddly.
To check, get that (raw) signature in a file and the key 
(at least the publickey) in a format OpenSSL likes, then 
rsautl -verify (which does only the RSA_public_decrypt step) 
and asn1parse the result. Since you apparently don't mind 
having the privatekey clear, 'traditional' PEM is easy:
  openssl pkcs8 pkcs8derfile -inform der -nocrypt rsapemfile
  openssl rsautl -verify -in sigfile -inkey rsapemfile temp
  openssl asn1parse temp -inform der 
See if the OBJECT (algorithm OID) is :sha1.
If not, but it is an OID known to openssl, just use its NID 
(assuming it is at least consistent across all signatures).
If it's not known (e.g. nonstandard), it appears to me 
adding it dynamically with OBJ_create should work, but 
that's a little more complicated (and I haven't tested).



__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager   majord...@openssl.org