[openssl.org #2938] [PATCH] Severe resource leak in tls_P_hash() (v1.0.1 and up)

2014-09-09 Thread Rich Salz via RT
From an internal review of the patch:
Contexts are meant to be reused and (for example) reusing the same context and
digest avoids having to reallocate a buffer for the internal context structure.

I think this might actually be a problem in the ENGINE not handling reuse
properly, or the EVP API not having a way to signal reuse properly.

Either way I don't think this is the correct fix.
--
Rich Salz, OpenSSL dev team; rs...@openssl.org

__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org


Re: [openssl.org #2938] [PATCH] Severe resource leak in tls_P_hash() (v1.0.1 and up)

2014-09-09 Thread Andrey Kulikov via RT
Indeed,

Improved version of the patch are in:
[openssl.org #2937] Handshake performance degradation in 1.0.1 and up.

On 9 September 2014 21:16, Rich Salz via RT r...@openssl.org wrote:

 From an internal review of the patch:
 Contexts are meant to be reused and (for example) reusing the same context
 and
 digest avoids having to reallocate a buffer for the internal context
 structure.

 I think this might actually be a problem in the ENGINE not handling reuse
 properly, or the EVP API not having a way to signal reuse properly.

 Either way I don't think this is the correct fix.
 --
 Rich Salz, OpenSSL dev team; rs...@openssl.org



__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org


Re: [openssl.org #2938] [PATCH] Severe resource leak in tls_P_hash() (v1.0.1 and up)

2014-09-09 Thread Andrey Kulikov
Indeed,

Improved version of the patch are in:
[openssl.org #2937] Handshake performance degradation in 1.0.1 and up.

On 9 September 2014 21:16, Rich Salz via RT r...@openssl.org wrote:

 From an internal review of the patch:
 Contexts are meant to be reused and (for example) reusing the same context
 and
 digest avoids having to reallocate a buffer for the internal context
 structure.

 I think this might actually be a problem in the ENGINE not handling reuse
 properly, or the EVP API not having a way to signal reuse properly.

 Either way I don't think this is the correct fix.
 --
 Rich Salz, OpenSSL dev team; rs...@openssl.org




[openssl.org #2938] [PATCH] Severe resource leak in tls_P_hash() (v1.0.1 and up)

2014-09-09 Thread Rich Salz via RT
Closing this in favor of 2937
--
Rich Salz, OpenSSL dev team; rs...@openssl.org

__
OpenSSL Project http://www.openssl.org
Development Mailing List   openssl-dev@openssl.org
Automated List Manager   majord...@openssl.org


[openssl.org #2938] [PATCH] Severe resource leak in tls_P_hash() (v1.0.1 and up)

2012-12-09 Thread Andrey Kulikov via RT
Hello,

In v1.0.1 tls_P_hash() has been changed in comparison to early OpenSSL
versions.
Used hash objects is re-initializing during P_hash calculation.
It looks harmless, but only until we come to hash objects, holding
references to external objects.
E.g. engine-specific hashes, allocating handles to hardware device or some
external library.

Then re-initing these MACs does not result in memory loss, but  handlers to
external objects, stored in that memory being lost.
(this is why I call it resource leak, not memory).

In my case each handshake leads to 87 unfreed handles, and library,
interacting with hardware device starts to fail due to out of memory after
about 6000 handshakes.

Please find attached patch, correcting discovered issue.
It can be applied to 1.0.1, 1.0.2 and to latest source tarball.

make test
report success for 1.0.1c with this patch applied.

If anyone see any issues in supplied patch - please let me know.

Best wishes,
Andrey Kulikov.

Hello,In v1.0.1 tls_P_hash() has been changed in comparison to early OpenSSL versions.Used hash objects is re-initializing during P_hash calculation.It looks harmless, but only until we come to hash objects, holding references to external objects.
E.g. engine-specific hashes, allocating handles to hardware device or some external library.Then re-initing these MACs does not result in memory loss, but? handlers to external objects, stored in that memory being lost.
(this is why I call it resource leak, not memory).In my case each handshake leads to 87 unfreed handles, and library, interacting with hardware device starts to fail due to out of memory after about 6000 handshakes.
Please find attached patch, correcting discovered issue.It can be applied to 1.0.1, 1.0.2 and to latest source tarball.make testreport success for 1.0.1c with this patch applied.If anyone see any issues in supplied patch - please let me know.
Best wishes,Andrey Kulikov.


tls1_P_hash_resource_leak.patch
Description: Binary data