Re: [openssl-users] A script for hybrid encryption with openssl
On 18/12/2018 18:04, Sam Roberts wrote: > Maybe you should look at gpg directly, `gpg --symmetric` uses a passphrase, > which doesn't sound fiddly. Unfortunately that doesn't do what I want: I'm after something using public key encryption (asymmetric, or a hybrid). This is so I don't need to deploy the decryption key on the server. N -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] A script for hybrid encryption with openssl
On Tue, Dec 18, 2018 at 3:18 AM Nick wrote: > I should add that I don't really care about the format, or even the use of > openssl - just the ability to tackle large files with the benefits of public > key encryption, in a self-contained way without needing fiddly work deploying > the keys (as GnuPG seems to require for its keyring, judging from my > experience deploying Backup-Ninja / Duplicity using Ansible.) Maybe you should look at gpg directly, `gpg --symmetric` uses a passphrase, which doesn't sound fiddly. -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] A script for hybrid encryption with openssl
On 17/12/2018 22:02, Jakob Bohm via openssl-users wrote: > A simpler way is to realize that the formats used by SMIME/CMS (specifically > the PKCS#7 formats) allow almost unlimited file size, and any 2GiB limit is > probably an artifact of either the openssl command line tool or some of the > underlying OpenSSL libraries. Yes. I started using openssl's smime implementation, then backed out when I realised there were indeed limits - apparently in the underlying libraries. On decrypting I got the same kind of errors described in this bug report thread (and elsewhere if you search, but this is the most recent discussion I could find). "Attempting to decrypt/decode a large smime encoded file created with openssl fails regardless of the amount of OS memory available". https://mta.openssl.org/pipermail/openssl-dev/2016-August/008237.html The key points are: - streaming smime *encryption* has been implemented, but - smime *decryption* is done in memory, consequentially you can't decrypt anything over 1.5G - possibly this is related to the BUF_MEM structure's dependency on the size of an int There's an RT ticket but I could not log in to read this. But it appears to have been migrated to Git-hub: https://github.com/openssl/openssl/issues/2515 It's closed - I infer as "won't fix" (yet?) and this is still an issue as my experience suggests, at least in the versions distributed for systems I will be using. I was using openssl 1.0.2g-1ubuntu4.14 (Xenial) and I've verified it with openssl 1.1.0g-2ubuntu4.3 (Bionic, the latest LTS release fro Ubuntu): $ openssl version -a OpenSSL 1.1.0g 2 Nov 2017 built on: reproducible build, date unspecified platform: debian-amd64 compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/x86_64-linux-gnu/engines-1.1\"" OPENSSLDIR: "/usr/lib/ssl" ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1" $ dd if=/dev/zero of=sample.txt count=2M bs=1024 $ openssl req -x509 -nodes -newkey rsa:2048 -keyout mysqldump-secure.priv.pem -out mysqldump-secure.pub.pem $ openssl smime -encrypt -binary -text -aes256 -in sample.txt -out sample.txt.enc -outform DER -stream mysqldump-secure.pub.pem $ openssl smime -decrypt -binary -inkey mysqldump-secure.priv.pem -inform DEM -in sample.txt.enc -out sample.txt.restored Error reading S/MIME message 139742630175168:error:07069041:memory buffer routines:BUF_MEM_grow_clean:malloc failure:../crypto/buffer/buffer.c:138: 139742630175168:error:0D06B041:asn1 encoding routines:asn1_d2i_read_bio:malloc failure:../crypto/asn1/a_d2i_fp.c:191 > Anyway, setting up an alternative data format might be suitable if combined > with other functionality requiring chunking, such as recovery from > lost/corrupted data "blocks" (where each block is much much larger than > a 1K "disk block"). I should add that I don't really care about the format, or even the use of openssl - just the ability to tackle large files with the benefits of public key encryption, in a self-contained way without needing fiddly work deploying the keys (as GnuPG seems to require for its keyring, judging from my experience deploying Backup-Ninja / Duplicity using Ansible.) So other solutions, if tried and tested, might work for me. Cheers, Nick -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] A script for hybrid encryption with openssl
On 17/12/2018 16:21, Nick wrote: Hello, I've written a script to try and work around openssl's lack of a way to encrypt large files with public key or hybrid cryptography. I gather SMIME works for files < ~ 2.5GB but the current implementation cannot decrypt files larger than this. My use case is automated server back-ups, for which I need to back up arbitrarily large files and copy the result to S3 for storage, but I don't want to store a decryption key on the server. I contemplated splitting the archives, except this seemed about as much work as writing something which stored an encrypted one-time password with the payload and using symmetric encryption. As I'm not really a crypto/security expert, I thought I'd post it here and ask for some feedback on it. https://github.com/wu-lee/hencrypt A simpler way is to realize that the formats used by SMIME/CMS (specifically the PKCS#7 formats) allow almost unlimited file size, and any 2GiB limit is probably an artifact of either the openssl command line tool or some of the underlying OpenSSL libraries. It would be interesting to hear from someone familiar with that part of the OpenSSL API which calls to use to actually do CMS signing/encryption (and verification/decryption) of data too large to fit in available memory, and how to handle the data length BER encoding for values larger than a size_t. Anyway, setting up an alternative data format might be suitable if combined with other functionality requiring chunking, such as recovery from lost/corrupted data "blocks" (where each block is much much larger than a 1K "disk block"). Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
[openssl-users] A script for hybrid encryption with openssl
Hello, I've written a script to try and work around openssl's lack of a way to encrypt large files with public key or hybrid cryptography. I gather SMIME works for files < ~ 2.5GB but the current implementation cannot decrypt files larger than this. My use case is automated server back-ups, for which I need to back up arbitrarily large files and copy the result to S3 for storage, but I don't want to store a decryption key on the server. I contemplated splitting the archives, except this seemed about as much work as writing something which stored an encrypted one-time password with the payload and using symmetric encryption. As I'm not really a crypto/security expert, I thought I'd post it here and ask for some feedback on it. https://github.com/wu-lee/hencrypt Thanks! Nick -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users