Hello, In v1.0.1 tls_P_hash() has been changed in comparison to early OpenSSL versions. Used hash objects is re-initializing during P_hash calculation. It looks harmless, but only until we come to hash objects, holding references to external objects. E.g. engine-specific hashes, allocating handles to hardware device or some external library.
Then re-initing these MACs does not result in memory loss, but handlers to external objects, stored in that memory being lost. (this is why I call it resource leak, not memory). In my case each handshake leads to 87 unfreed handles, and library, interacting with hardware device starts to fail due to out of memory after about 6000 handshakes. Please find attached patch, correcting discovered issue. It can be applied to 1.0.1, 1.0.2 and to latest source tarball. make test report success for 1.0.1c with this patch applied. If anyone see any issues in supplied patch - please let me know. Best wishes, Andrey Kulikov.Hello,
In v1.0.1 tls_P_hash() has been changed in comparison to early OpenSSL versions.
Used hash objects is re-initializing during P_hash calculation.
It looks harmless, but only until we come to hash objects, holding references to external objects.
E.g. engine-specific hashes, allocating handles to hardware device or some external library.
Then re-initing these MACs does not result in memory loss, but? handlers to external objects, stored in that memory being lost.
(this is why I call it resource leak, not memory).
In my case each handshake leads to 87 unfreed handles, and library, interacting with hardware device starts to fail due to out of memory after about 6000 handshakes.
Please find attached patch, correcting discovered issue.
It can be applied to 1.0.1, 1.0.2 and to latest source tarball.
make test
report success for 1.0.1c with this patch applied.
If anyone see any issues in supplied patch - please let me know.
Best wishes,
Andrey Kulikov.
tls1_P_hash_resource_leak.patch
Description: Binary data
