On 17/12/2018 22:02, Jakob Bohm via openssl-users wrote:
> A simpler way is to realize that the formats used by SMIME/CMS (specifically
> the PKCS#7 formats) allow almost unlimited file size, and any 2GiB limit is
> probably an artifact of either the openssl command line tool or some of the
> underlying OpenSSL libraries.
Yes. I started using openssl's smime implementation, then backed out when I
realised there were indeed limits - apparently in the underlying libraries.
On decrypting I got the same kind of errors described in this bug report thread
(and elsewhere if you search, but this is the most recent discussion I could
find).
"Attempting to decrypt/decode a large smime encoded file created with openssl
fails regardless of the amount of OS memory available".
https://mta.openssl.org/pipermail/openssl-dev/2016-August/008237.html
The key points are:
- streaming smime *encryption* has been implemented, but
- smime *decryption* is done in memory, consequentially you can't decrypt
anything over 1.5G
- possibly this is related to the BUF_MEM structure's dependency on the size of
an int
There's an RT ticket but I could not log in to read this. But it appears to
have been migrated to Git-hub:
https://github.com/openssl/openssl/issues/2515
It's closed - I infer as "won't fix" (yet?) and this is still an issue as my
experience suggests, at least in the versions distributed for systems I will be
using.
I was using openssl 1.0.2g-1ubuntu4.14 (Xenial) and I've verified it with
openssl 1.1.0g-2ubuntu4.3 (Bionic, the latest LTS release fro Ubuntu):
$ openssl version -a
OpenSSL 1.1.0g 2 Nov 2017
built on: reproducible build, date unspecified
platform: debian-amd64
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS
-DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2
-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM
-DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM
-DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\""
-DENGINESDIR="\"/usr/lib/x86_64-linux-gnu/engines-1.1\""
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"
$ dd if=/dev/zero of=sample.txt count=2M bs=1024
$ openssl req -x509 -nodes -newkey rsa:2048 -keyout
mysqldump-secure.priv.pem -out mysqldump-secure.pub.pem
$ openssl smime -encrypt -binary -text -aes256 -in sample.txt -out
sample.txt.enc -outform DER -stream mysqldump-secure.pub.pem
$ openssl smime -decrypt -binary -inkey mysqldump-secure.priv.pem -inform
DEM -in sample.txt.enc -out sample.txt.restored
Error reading S/MIME message
139742630175168:error:07069041:memory buffer
routines:BUF_MEM_grow_clean:malloc failure:../crypto/buffer/buffer.c:138:
139742630175168:error:0D06B041:asn1 encoding
routines:asn1_d2i_read_bio:malloc failure:../crypto/asn1/a_d2i_fp.c:191
> Anyway, setting up an alternative data format might be suitable if combined
> with other functionality requiring chunking, such as recovery from
> lost/corrupted data "blocks" (where each block is much much larger than
> a 1K "disk block").
I should add that I don't really care about the format, or even the use of
openssl - just the ability to tackle large files with the benefits of public key
encryption, in a self-contained way without needing fiddly work deploying the
keys (as GnuPG seems to require for its keyring, judging from my experience
deploying Backup-Ninja / Duplicity using Ansible.) So other solutions, if tried
and tested, might work for me.
Cheers,
Nick
--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
