On Mon, Jul 11, 2011, Michael Tabolsky wrote: > On Monday 11 July 2011 01:18:22 Dr. Stephen Henson wrote: > > Is that big file base64 encoded? If so then the decode process can increase > > the memory considerably. I'd suggest you base64 decode it and then try the > > -inform DER option to the smime -decrypt command. It will still need the > > lot in memory but not as much. > > Thank you for your prompt reply Steve, I've already tried to de-base64 first. > The DER file is 3.6GB and it fails more or less the same (at least from my > point of view): > 140690942056104:error:0D06B041:asn1 encoding > routines:ASN1_D2I_READ_BIO:malloc > failure:a_d2i_fp.c:230: > my prinfs show the same numbers as with PEM: > asn1_d2i_read_bio is going to grow the buff to 2006569233 > asn1_d2i_read_bio is going to grow the buff to 2006573333 > asn1_d2i_read_bio is going to grow the buff to 2006577433 > asn1_d2i_read_bio is going to grow the buff to 2006581533 > ret=OPENSSL_realloc_clean num gets num 2675447504 > Passed num == NULL, return the same > > (double checked now to avoid 5am effects) > > > If that doesn't help there are other things that can be done, one of which > > is to manually extract the decryption key and decrypt the content. A sort > > of horribly hacky streaming decrypt. > > I would do any hack whatever horrible it is to get the file back, > unfortunately > I am not skilled enough and I can't understand what you suggest. Does pkcs7 > consists of "pieces" that can be decoded one by one? > >
Well getting the DER file is a good start. Hmmm... I wonder if you could mmap the file then call d2i_PKCS7() on it. There is an example minimal decrypter in demos/smime/p7decr.c which might be adaptable. The alternative is to manually extract the content, decrypt key and IV and decrypt that as a stream. I'll have a think about that. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
