Re: [openssl.org #3201] EVP_DigestUpdate crashes because of a NULL pointer
We have encountered a Segmentation Fault while trying to send a SSL
packet via Oracle VM agent.
The Segmentation Fault occurred when EVP_MD_CTX_copy() failed in tls1_mac().
tls1_mac() doesn't check the return code of EVP_MD_CTX_copy() and keep
going, which results in Segmentation Fault at EVP_DigestUpdate().
The following change in tls1_mac() fixes the segfault issue.
1 Index: openssl/ssl/t1_enc.c
2
3 $ diff -ru ssl/t1_enc.c ssl/t1_enc.c
4 --- t1_enc.c.orig Tue Dec 10 15:36:05 2013
5 +++ t1_enc.cWed Dec 11 09:29:02 2013
6 @@ -980,7 +980,10 @@
7 }
8 else
9 {
10 - EVP_MD_CTX_copy(hmac,hash);
11 + if (EVP_MD_CTX_copy(hmac,hash) != 1)
12 + {
13 + return (0);
14 + }
15 mac_ctx =hmac;
16 }
17
The return value indicating error is -1 in this case.
http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=03da57fe14f2de5bde9d4496a2ae9a4ae8879f88
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
Re: [openssl.org #3201] EVP_DigestUpdate crashes because of a NULL pointer
EVP_MD_CTX_copy() also can fail due to external engine usage (including HW engine). In this case underlying work can be more complicated that memory allocation. Descriptors shortage, amount of simultaneous contexts constraints, etc. Or just bugs in engine code. On 27 December 2013 21:39, Misaki.Miyashita via RT r...@openssl.org wrote: I agree that the return value should be checked but I'd like to know the underlying cause. EVP_MD_CTX_copy() shouldn't normally fail unless something is seriously wrong, e.g. memory allocation failures.
Re: [openssl.org #3201] EVP_DigestUpdate crashes because of a NULL pointer
Hi Steve, Sorry for the late response. Thank you for looking into the bug. In our case, EVP_MD_CTX_copy() failure was caused by an application bug. A child process was trying to use the session from its parent process, and that caused an issue down in pkcs11 engine. The application will be fixed. At the same time, please consider returning an error so that segmentation fault can be prevented. Thank you, -- misaki On 12/17/13 10:39, Stephen Henson via RT wrote: On Sat Dec 14 08:42:01 2013, misaki.miyash...@oracle.com wrote: The Segmentation Fault occurred when EVP_MD_CTX_copy() failed in tls1_mac(). tls1_mac() doesn't check the return code of EVP_MD_CTX_copy() and keep going, which results in Segmentation Fault at EVP_DigestUpdate(). The following change in tls1_mac() fixes the segfault issue. I agree that the return value should be checked but I'd like to know the underlying cause. EVP_MD_CTX_copy() shouldn't normally fail unless something is seriously wrong, e.g. memory allocation failures. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #3201] EVP_DigestUpdate crashes because of a NULL pointer
Hi Steve, Sorry for the late response. Thank you for looking into the bug. In our case, EVP_MD_CTX_copy() failure was caused by an application bug. A child process was trying to use the session from its parent process, and that caused an issue down in pkcs11 engine. The application will be fixed. At the same time, please consider returning an error so that segmentation fault can be prevented. Thank you, -- misaki On 12/17/13 10:39, Stephen Henson via RT wrote: On Sat Dec 14 08:42:01 2013, misaki.miyash...@oracle.com wrote: The Segmentation Fault occurred when EVP_MD_CTX_copy() failed in tls1_mac(). tls1_mac() doesn't check the return code of EVP_MD_CTX_copy() and keep going, which results in Segmentation Fault at EVP_DigestUpdate(). The following change in tls1_mac() fixes the segfault issue. I agree that the return value should be checked but I'd like to know the underlying cause. EVP_MD_CTX_copy() shouldn't normally fail unless something is seriously wrong, e.g. memory allocation failures. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #3201] EVP_DigestUpdate crashes because of a NULL pointer
On Sat Dec 14 08:42:01 2013, misaki.miyash...@oracle.com wrote: The Segmentation Fault occurred when EVP_MD_CTX_copy() failed in tls1_mac(). tls1_mac() doesn't check the return code of EVP_MD_CTX_copy() and keep going, which results in Segmentation Fault at EVP_DigestUpdate(). The following change in tls1_mac() fixes the segfault issue. I agree that the return value should be checked but I'd like to know the underlying cause. EVP_MD_CTX_copy() shouldn't normally fail unless something is seriously wrong, e.g. memory allocation failures. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #3201] EVP_DigestUpdate crashes because of a NULL pointer
Hello,
We have encountered a Segmentation Fault while trying to send a SSL
packet via Oracle VM agent.
The Segmentation Fault occurred when EVP_MD_CTX_copy() failed in tls1_mac().
tls1_mac() doesn't check the return code of EVP_MD_CTX_copy() and keep
going, which results in Segmentation Fault at EVP_DigestUpdate().
The following change in tls1_mac() fixes the segfault issue.
1 Index: openssl/ssl/t1_enc.c
2
3 $ diff -ru ssl/t1_enc.c ssl/t1_enc.c
4 --- t1_enc.c.orig Tue Dec 10 15:36:05 2013
5 +++ t1_enc.cWed Dec 11 09:29:02 2013
6 @@ -980,7 +980,10 @@
7 }
8 else
9 {
10 - EVP_MD_CTX_copy(hmac,hash);
11 + if (EVP_MD_CTX_copy(hmac,hash) != 1)
12 + {
13 + return (0);
14 + }
15 mac_ctx =hmac;
16 }
17
I would greatly appreciate it if the suggest fix gets integrated into
the next release of OpenSSL.
Best regards,
Misaki Miyashita
Oracle Solaris Security
Senior Software Engineer
Austin, TX, US
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager majord...@openssl.org
