Dear OpenSSL Community, I am struggling for four days trying to workaround the problem...
I have a big smime encrypted file, approximately 5GB which was encrypted using openssl: openssl smime -encrypt -stream -des3 -binary cert.crt -in data.file -out data.file.enc The file is a bziped backup of some websites which due to some circumstances (explainable by Murphy laws) exist only in this file now. These are years of content publishing there. Many people contributions. Now when I try to decrypt it I hit the problem already noted here on the list: openssl smime -decrypt -inkey mykey -in data.file.enc -out data.file it runs for some time and the fails as following: 1195:error:07069041:memory buffer routines:BUF_MEM_grow_clean:malloc failure:buffer.c:152: 1195:error:0D06B041:asn1 encoding routines:ASN1_D2I_READ_BIO:malloc failure:a_d2i_fp.c:229: 1195:error:21078082:PKCS7 routines:B64_READ_PKCS7:decode error:pk7_mime.c:140: 1195:error:2107A08B:PKCS7 routines:SMIME_read_PKCS7:pkcs7 parse error:pk7_mime.c:373: The first thought was that I was hitting some OS memory limit, so I've straced it and see nothing that can hint me an error: read(3, "qp4j3Uhgw8STUMtxKH6FnH6/kIAVqxWV"..., 4096) = 4096 read(3, "12qPS5kvzTUmGEtun/JOKX47gO9cY+dK"..., 4096) = 4096 read(3, "7yewf3BWSrWaG4oenlKJIWEMXzgcyauB"..., 4096) = 4096 read(3, "17PyAFW1uuZy5Bko+7DLyhQXgtEgVRwh"..., 4096) = 4096 munmap(0x2aaaacaee000, 2006585344) = 0 lseek(3, -2881, SEEK_CUR) = 2717246655 write(2, "Error reading S/MIME message\n", 29) = 29 Worth mentioning, I've hit it first on a smaller file (~1.5GB) on a 32bit host, I've tried it on a 64bit with 8GB of memory and it was decrypted. The one I am struggling with is now on a x86_64/16GB Linux on amazon EC3. I am not a C programmer but I can put some printf's in the other's people code and recompile, so I've followed this path to track where it fails what I could find was in BUF_MEM_grow_clean that calls OPENSSL_realloc_clean which returns NULL because CRYPTO_realloc_clean gets NULL as its last parameter "num" which is correctly set to 2675447504 in BUF_MEM_grow_clean. Now I can feel it has to do something with the number length (long/int/size_t) which is not the thing I can follow easily. I've tried to change the types to larger ones but it only gets me to segmentation faults. Anybody any clues on what I can try at this point? Thank everyone in advance. P.S. The problem is can be reproduced with any smime file larger than 4GB. dd'ing some data, encrypting it cannot ever be decrypted. -- Michael Tabolsky, Independent IT Professional Milan Area, Italy Italy +393462223547,+3902320626558, Israel +97239150702, UK +441422571312 msn/gtalk/jabber: gf...@gfdsa.org ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
