On Mon, Jul 11, 2011, Michael Tabolsky wrote: > Dear OpenSSL Community, > > I am struggling for four days trying to workaround the problem... > > I have a big smime encrypted file, approximately 5GB which was encrypted > using openssl: > > openssl smime -encrypt -stream -des3 -binary cert.crt -in data.file -out > data.file.enc > > The file is a bziped backup of some websites which due to some circumstances > (explainable by Murphy laws) exist only in this file now. > These are years of content publishing there. Many people contributions. > > Now when I try to decrypt it I hit the problem already noted here on the list: > > openssl smime -decrypt -inkey mykey -in data.file.enc -out data.file > > it runs for some time and the fails as following: > 1195:error:07069041:memory buffer routines:BUF_MEM_grow_clean:malloc > failure:buffer.c:152: > 1195:error:0D06B041:asn1 encoding routines:ASN1_D2I_READ_BIO:malloc > failure:a_d2i_fp.c:229: > 1195:error:21078082:PKCS7 routines:B64_READ_PKCS7:decode error:pk7_mime.c:140: > 1195:error:2107A08B:PKCS7 routines:SMIME_read_PKCS7:pkcs7 parse > error:pk7_mime.c:373: > > The first thought was that I was hitting some OS memory limit, so I've > straced it and see > nothing that can hint me an error: > read(3, "qp4j3Uhgw8STUMtxKH6FnH6/kIAVqxWV"..., 4096) = 4096 > read(3, "12qPS5kvzTUmGEtun/JOKX47gO9cY+dK"..., 4096) = 4096 > read(3, "7yewf3BWSrWaG4oenlKJIWEMXzgcyauB"..., 4096) = 4096 > read(3, "17PyAFW1uuZy5Bko+7DLyhQXgtEgVRwh"..., 4096) = 4096 > munmap(0x2aaaacaee000, 2006585344) = 0 > lseek(3, -2881, SEEK_CUR) = 2717246655 > write(2, "Error reading S/MIME message\n", 29) = 29 > > Worth mentioning, I've hit it first on a smaller file (~1.5GB) on a 32bit > host, I've tried it on a 64bit with > 8GB of memory and it was decrypted. The one I am struggling with is now on a > x86_64/16GB Linux on amazon EC3. > > I am not a C programmer but I can put some printf's in the other's people > code and recompile, > so I've followed this path to track where it fails what I could find was in > BUF_MEM_grow_clean > that calls OPENSSL_realloc_clean which returns NULL because > CRYPTO_realloc_clean > gets NULL as its last parameter "num" which is correctly set to 2675447504 in > BUF_MEM_grow_clean. > > Now I can feel it has to do something with the number length > (long/int/size_t) > which is not the thing I can follow easily. I've tried to change the types to > larger > ones but it only gets me to segmentation faults. > > Anybody any clues on what I can try at this point? > > Thank everyone in advance. > > P.S. The problem is can be reproduced with any smime file larger than 4GB. > dd'ing some data, encrypting it cannot ever be decrypted. >
Yes you will hit that problem with large files. The S/MIME encrypt streams with a low memory overhead while decrypt does not stream and needs the whole file in memory. It's a serious amount of effort to stream decrypt and so far there hasn't been enough interest to do that. That of course wont help your immediate problem. Here's a possible solution... Is that big file base64 encoded? If so then the decode process can increase the memory considerably. I'd suggest you base64 decode it and then try the -inform DER option to the smime -decrypt command. It will still need the lot in memory but not as much. If that doesn't help there are other things that can be done, one of which is to manually extract the decryption key and decrypt the content. A sort of horribly hacky streaming decrypt. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
