I'm using openssl 0.9.7e on Unix (The example output below is from Windows version of openssl [a 0.9.7X derived binary version from stunnel.org], but is consistent with AIX version as regards the failure. I will repeat this tomorrow when I have access to the Unix box if that helps).
openssl pkcs7 -inform DER -in file.der This outputs a PEM file (topped and tailed with the '------ xxxx PKCS7 -----' line) *But* openssl smime -decrypt -in file.der -inform DER -recip cert.pem -inkey private.pem gives Error decrypting PKCS#7 structure 172:error:0D07207B:asn1 encoding routines:ASN1_get_object:header too long:crypto/asn1/asn1_lib.c:140: 172:error:0D068066:asn1 encoding routines:ASN1_CHECK_TLEN:bad object header:crypto/asn1/tasn_dec.c:935: 172:error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:crypto/asn1/tasn_dec.c:628: 172:error:0D08606D:asn1 encoding routines:ASN1_TYPE_get_int_octetstring:data is wrong:crypto/asn1/evp_asn1.c:179: 172:error:21072077:PKCS7 routines:PKCS7_decrypt:decrypt error:crypto/pkcs7/pk7_smime.c:414: (If the file.der originated from one of our own computers, using the same public key to encrypt then the above decrypt line works). [If I redirect the output from the pkcs7 line into say fred.pem, and try decrypting this (using -in fred.pem -inform PEM ) naturally get the same error.] It may be our client is doing something wrong, but as with any interoperability testing I always assume the fault is my end until I have proof it isn't. Peter -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dr. Stephen Henson Sent: 06 March 2005 01:13 To: openssl-users@openssl.org Subject: Re: Re(2): Decryption Problem On Fri, Mar 04, 2005, Peter Cope wrote: > Steve, sorry forget to include the asn1parse output ... > > I've X'd out sensitive stuff: > > 0:d=0 hl=4 l=57226 cons: SEQUENCE > 4:d=1 hl=2 l= 9 prim: OBJECT :pkcs7-envelopedData > 15:d=1 hl=4 l=57211 cons: cont [ 0 ] > 19:d=2 hl=4 l=57207 cons: SEQUENCE > 23:d=3 hl=2 l= 1 prim: INTEGER :00 > 26:d=3 hl=4 l= 308 cons: SET > 30:d=4 hl=4 l= 304 cons: SEQUENCE > 34:d=5 hl=2 l= 1 prim: INTEGER :00 > 37:d=5 hl=3 l= 152 cons: SEQUENCE > 40:d=6 hl=3 l= 146 cons: SEQUENCE > 43:d=7 hl=2 l= 11 cons: SET > 45:d=8 hl=2 l= 9 cons: SEQUENCE > 47:d=9 hl=2 l= 3 prim: OBJECT :countryName > 52:d=9 hl=2 l= 2 prim: PRINTABLESTRING :GB > 56:d=7 hl=2 l= 19 cons: SET > 58:d=8 hl=2 l= 17 cons: SEQUENCE > 60:d=9 hl=2 l= 3 prim: OBJECT :stateOrProvinceName > 65:d=9 hl=2 l= 10 prim: PRINTABLESTRING :XXXXXXXXXX > 77:d=7 hl=2 l= 15 cons: SET > 79:d=8 hl=2 l= 13 cons: SEQUENCE > 81:d=9 hl=2 l= 3 prim: OBJECT :localityName > 86:d=9 hl=2 l= 6 prim: PRINTABLESTRING :XXXXXX > 94:d=7 hl=2 l= 12 cons: SET > 96:d=8 hl=2 l= 10 cons: SEQUENCE > 98:d=9 hl=2 l= 3 prim: OBJECT :organizationName > 103:d=9 hl=2 l= 3 prim: PRINTABLESTRING :XXX > 108:d=7 hl=2 l= 12 cons: SET > 110:d=8 hl=2 l= 10 cons: SEQUENCE > 112:d=9 hl=2 l= 3 prim: OBJECT :organizationalUnitName > 117:d=9 hl=2 l= 3 prim: PRINTABLESTRING :xxx > 122:d=7 hl=2 l= 21 cons: SET > 124:d=8 hl=2 l= 19 cons: SEQUENCE > 126:d=9 hl=2 l= 3 prim: OBJECT :commonName > 131:d=9 hl=2 l= 12 prim: PRINTABLESTRING :xxxxxxxxxxxx > 145:d=7 hl=2 l= 42 cons: SET > 147:d=8 hl=2 l= 40 cons: SEQUENCE > 149:d=9 hl=2 l= 9 prim: OBJECT :emailAddress > 160:d=9 hl=2 l= 27 prim: IA5STRING :[EMAIL PROTECTED] > 189:d=6 hl=2 l= 1 prim: INTEGER :1C > 192:d=5 hl=2 l= 13 cons: SEQUENCE > 194:d=6 hl=2 l= 9 prim: OBJECT :rsaEncryption > 205:d=6 hl=2 l= 0 prim: NULL > 207:d=5 hl=3 l= 128 prim: OCTET STRING > 338:d=3 hl=4 l=56888 cons: SEQUENCE > 342:d=4 hl=2 l= 9 prim: OBJECT :pkcs7-data > 353:d=4 hl=2 l= 15 cons: SEQUENCE > 355:d=5 hl=2 l= 8 prim: OBJECT :rc2-cbc > 365:d=5 hl=2 l= 3 cons: SEQUENCE > 367:d=6 hl=2 l= 1 prim: INTEGER :3A > 370:d=4 hl=4 l=56856 prim: cont [ 0 ] > > The block beyond 370 is not ASN.1 (which I understand is OK, according to S/MIME). > OK, what does: openssl pkcs7 -inform DER -in file.der do? Do you get an error or just the PEM output? Also what version of OpenSSL are you using? Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]