Thank you Steve (and Rich) for your insights so far. It looks like BUF_MEM is the structure in question? If so, that might also explain the odd behavior with smime encrypting which makes this issue sort of moot. Even if I could fix this, the encrypted files I've created so far would be truncated I think.
I guess the safety check is to ensure that the structure works correctly with other functions that put the result into an int type internally? https://github.com/openssl/openssl/blob/3ce2fdabe6e33952bf3011acf5b68107e6352603/crypto/buffer/buffer.c#L19 Regardless of input size (2GB or 30GB) to the smime application, the resulting encrypted file is only 1.9GB on disk. Unless smime format has some very serious compression, it looks like it is silently truncating input. A 32 bit integer dependency in the read buffer might explain that. Is it related, or should I file that separately? Brian On Wed, Aug 17, 2016 at 6:54 PM, Stephen Henson via RT <r...@openssl.org> wrote: > On Wed Aug 17 18:16:41 2016, bmor...@mortoninsights.com wrote: > > That doesn't sound like an ideal case for a bugfix. Any other creative > > ideas on how to fix this one? Some suggestions I read previously included > > adding support for streaming decode to avoid such a large memory > > allocation. This may not easily be feasible because of needing to verify > > signatures on the message. > > > > A streaming decode is one option but this is far from a trivial > undertaking, > Verifying signatures would be handled on the fly but you'll only know the > signature is valid after all content has been processed of course. > > > If not, I'll try out the size_t change. > > > > This is a significant job too. There is a major knock on effect with > several > APIs so it's not just a case of changing a few structures or we'd have > done it > already. It is planned for a future release though. > > As RIch mentioned one of the key structures has a dependency on int which > is > often 32 bits even on 64 bit systems. > > Steve. > -- > Dr Stephen N. Henson. OpenSSL project core developer. > Commercial tech support now available see: http://www.openssl.org > > -- > Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4651 > Please log in as guest with password guest if prompted > -- Ticket here: http://rt.openssl.org/Ticket/Display.html?id=4651 Please log in as guest with password guest if prompted -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev