[openssl-dev] [openssl.org #2591] bug report : cryptlib.c : within CRYPTO_thread_id() use pthread_self() instead of getpid()
Please see https://github.com/openssl/openssl/pull/451 which is what we'll be doing for threads moving forward -- Rich Salz, OpenSSL dev team; rs...@openssl.org ___ openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
[openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8
working as designed and required. no bug. closing ticket. -- Rich Salz, OpenSSL dev team; rs...@openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
RE: [openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8
Hi, Thank you. I just notice that the main time consumer is not the getpid(). I also found thousands of “FIPS_selftest_failed” during FIPS mode setup which indeed induce the getpid call. Is it normal that FIPS_mode_set(1) needs 5 seconds to finish? Do you have any idea about the main time consumer here? The call stack is followed. If the self-test has some problem, how should I debug to root cause? Here is the call stack, the following call log just repeated thousands of times in my case. Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40b5ec in FIPS_selftest_failed+0x6c () from libcrypto.sl.0.9.8 #3 0x7a420764 in EVP_DigestInit_ex+0x34 () from libcrypto.sl.0.9.8 #4 0x7a4251b8 in ssleay_rand_add#HLO_CL_#i1_0x14+0x130 () from libcrypto.sl.0.9.8 #5 0x7a4248e4 in ssleay_rand_bytes+0x1a4 () from libcrypto.sl.0.9.8 #6 0x7a4265b8 in RAND_bytes+0x80 () from libcrypto.sl.0.9.8 #7 0x7a40bd58 in FIPS_mode_set+0x270 () from libcrypto.sl.0.9.8 Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40b3b0 in FIPS_mode+0x70 () from libcrypto.sl.0.9.8 #3 0x7a420824 in EVP_DigestInit_ex+0xf4 () from libcrypto.sl.0.9.8 #4 0x7a4251b8 in ssleay_rand_add#HLO_CL_#i1_0x14+0x130 () from libcrypto.sl.0.9.8 #5 0x7a4248e4 in ssleay_rand_bytes+0x1a4 () from libcrypto.sl.0.9.8 #6 0x7a4265b8 in RAND_bytes+0x80 () from libcrypto.sl.0.9.8 #7 0x7a40bd58 in FIPS_mode_set+0x270 () from libcrypto.sl.0.9.8 Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a425120 in ssleay_rand_add#HLO_CL_#i1_0x14+0x98 () from libcrypto.sl.0.9.8 #3 0x7a4248e4 in ssleay_rand_bytes+0x1a4 () from libcrypto.sl.0.9.8 #4 0x7a4265b8 in RAND_bytes+0x80 () from libcrypto.sl.0.9.8 #5 0x7a40bd58 in FIPS_mode_set+0x270 () from libcrypto.sl.0.9.8 Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40b5ec in FIPS_selftest_failed+0x6c () from libcrypto.sl.0.9.8 #3 0x7a420764 in EVP_DigestInit_ex+0x34 () from libcrypto.sl.0.9.8 #4 0x7a4251b8 in ssleay_rand_add#HLO_CL_#i1_0x14+0x130 () from libcrypto.sl.0.9.8 #5 0x7a4248e4 in ssleay_rand_bytes+0x1a4 () from libcrypto.sl.0.9.8 #6 0x7a4265b8 in RAND_bytes+0x80 () from libcrypto.sl.0.9.8 #7 0x7a40bd58 in FIPS_mode_set+0x270 () from libcrypto.sl.0.9.8 Best Regards, -Meiling -Original Message- From: Stephen Henson via RT [mailto:r...@openssl.org] Sent: Thursday, December 27, 2012 6:26 AM To: Ge, Meiling Cc: openssl-dev@openssl.org Subject: [openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8 [meiling...@emc.com - Wed Dec 26 21:07:57 2012]: Hi Openssl team, I have an performance issue with openssl_fips. My application use openssl_fips version 0.9.8. Recently, I found that the fips lib make my application slow. When my application initialize the fips setting, it introduces 7000+ getpid() call. And this will cost 5 seconds. Is this an real issue? Looking forward to your reply. Thanks. The call trace is as followed: Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40bbd8 in FIPS_mode_set+0xf0 () from libcrypto.sl.0.9.8 #3 0x2695f8 in main+0x168 () These all go through a user settable callback which defaults to getpid() on most platforms. You can supply a more efficient equivalent in an application. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8
On Thu Jan 10 15:33:12 2013, meiling...@emc.com wrote: Hi, Thank you. I just notice that the main time consumer is not the getpid(). I also found thousands of “FIPS_selftest_failed” during FIPS mode setup which indeed induce the getpid call. Is it normal that FIPS_mode_set(1) needs 5 seconds to finish? For the 1.2 module it can take some time to finish the self tests. One of the requirements for the 2.0 module was to improve the efficiency of the self tests, so that should be much quicker if you can use the 2.0 module. Do you have any idea about the main time consumer here? The call stack is followed. If the self-test has some problem, how should I debug to root cause? The main time consumer is likely to be a call to generate DSA parameters in FIPS_selftest_dsa(). There isn't anything you can do with the 1.2 module to spped that up without changing the source and the result being no longer validated. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8
Hi Openssl team, I have an performance issue with openssl_fips. My application use openssl_fips version 0.9.8. Recently, I found that the fips lib make my application slow. When my application initialize the fips setting, it introduces 7000+ getpid() call. And this will cost 5 seconds. Is this an real issue? Looking forward to your reply. Thanks. The call trace is as followed: Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40bbd8 in FIPS_mode_set+0xf0 () from libcrypto.sl.0.9.8 #3 0x2695f8 in main+0x168 () Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40b3b0 in FIPS_mode+0x70 () from libcrypto.sl.0.9.8 #3 0x7a4445a4 in HMAC_Init_ex+0x224 () from libcrypto.sl.0.9.8 #4 0x7a44471c in HMAC_Init+0x6c () from libcrypto.sl.0.9.8 #5 0x7a40b890 in FIPS_incore_fingerprint+0x98 () from libcrypto.sl.0.9.8 #6 0x7a40b9b8 in FIPS_check_incore_fingerprint+0x30 () from libcrypto.sl.0.9.8 #7 0x7a40bd08 in FIPS_mode_set+0x220 () from libcrypto.sl.0.9.8 #8 0x2695f8 in main+0x168 () Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40b5ec in FIPS_selftest_failed+0x6c () from libcrypto.sl.0.9.8 #3 0x7a420764 in EVP_DigestInit_ex+0x34 () from libcrypto.sl.0.9.8 #4 0x7aa4 in HMAC_Init_ex+0x124 () from /libcrypto.sl.0.9.8 #5 0x7a44471c in HMAC_Init+0x6c () from libcrypto.sl.0.9.8 #6 0x7a40b890 in FIPS_incore_fingerprint+0x98 () from libcrypto.sl.0.9.8 #7 0x7a40b9b8 in FIPS_check_incore_fingerprint+0x30 () from libcrypto.sl.0.9.8 #8 0x7a40bd08 in FIPS_mode_set+0x220 () from libcrypto.sl.0.9.8 #9 0x2695f8 in main+0x168 () Best Regards, -Meiling Hi Openssl team,I have an performance issue with openssl_fips.My application use openssl_fips version 0.9.8.Recently, I found that the fips lib make my application slow.When my application initialize the fips setting, it introduces 7000+ getpid() call.And this will cost 5 seconds.Is this an real issue?Looking forward to your reply.Thanks.The call trace is as followed:Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2#0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2#1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8#2 0x7a40bbd8 in FIPS_mode_set+0xf0 () from libcrypto.sl.0.9.8#3 0x2695f8 in main+0x168 ()Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2#0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2#1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8#2 0x7a40b3b0 in FIPS_mode+0x70 () from libcrypto.sl.0.9.8#3 0x7a4445a4 in HMAC_Init_ex+0x224 () from libcrypto.sl.0.9.8#4 0x7a44471c in HMAC_Init+0x6c () from libcrypto.sl.0.9.8#5 0x7a40b890 in FIPS_incore_fingerprint+0x98 () from libcrypto.sl.0.9.8#6 0x7a40b9b8 in FIPS_check_incore_fingerprint+0x30 () from libcrypto.sl.0.9.8#7 0x7a40bd08 in FIPS_mode_set+0x220 () from libcrypto.sl.0.9.8#8 0x2695f8 in main+0x168 ()Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2#0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2#1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8#2 0x7a40b5ec in FIPS_selftest_failed+0x6c () from libcrypto.sl.0.9.8#3 0x7a420764 in EVP_DigestInit_ex+0x34 () from libcrypto.sl.0.9.8#4 0x7aa4 in HMAC_Init_ex+0x124 () from /libcrypto.sl.0.9.8#5 0x7a44471c in HMAC_Init+0x6c () from libcrypto.sl.0.9.8#6 0x7a40b890 in FIPS_incore_fingerprint+0x98 () from libcrypto.sl.0.9.8#7 0x7a40b9b8 in FIPS_check_incore_fingerprint+0x30 () from libcrypto.sl.0.9.8#8 0x7a40bd08 in FIPS_mode_set+0x220 () from libcrypto.sl.0.9.8#9 0x2695f8 in main+0x168 ()Best Regards,-Meiling
Re: [openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8
On Wed, Dec 26, 2012 at 09:07:58PM +0100, Ge, Meiling via RT wrote: Hi Openssl team, I have an performance issue with openssl_fips. My application use openssl_fips version 0.9.8. Recently, I found that the fips lib make my application slow. When my application initialize the fips setting, it introduces 7000+ getpid() call. And this will cost 5 seconds. Is this an real issue? Yes, the cost of the getpid() calls made by OpenSSL is real, and users often patch them away. If these calls come from code within the FIPS canister, you may not be able to easily do so except by linking to or preloading another library that reimplements fork() and getpid(), so that the PID is cached in a global variable. There's just no reason why getpid() should require a trap into the kernel... Thor __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #2948] thousands of getpid called inside libcrypto.sl.0.9.8
[meiling...@emc.com - Wed Dec 26 21:07:57 2012]: Hi Openssl team, I have an performance issue with openssl_fips. My application use openssl_fips version 0.9.8. Recently, I found that the fips lib make my application slow. When my application initialize the fips setting, it introduces 7000+ getpid() call. And this will cost 5 seconds. Is this an real issue? Looking forward to your reply. Thanks. The call trace is as followed: Breakpoint 1, 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #0 0x7af3a278 in getpid+0 () from /usr/lib/libc.2 #1 0x7a41c454 in CRYPTO_thread_id+0x24 () from libcrypto.sl.0.9.8 #2 0x7a40bbd8 in FIPS_mode_set+0xf0 () from libcrypto.sl.0.9.8 #3 0x2695f8 in main+0x168 () These all go through a user settable callback which defaults to getpid() on most platforms. You can supply a more efficient equivalent in an application. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
[openssl.org #2591] bug report : cryptlib.c : within CRYPTO_thread_id() use pthread_self() instead of getpid()
Hello OpenSSL Team, OS : MAC OS 10.6 i386 Version : openssl-0.9.8r/openssl-1.0.0d i got alloc/free errors under MAC OS 10.6 using OpenSSL within ERR_set_error_data(), because CRYPTO_thread_id() calling getpid() returns the same ERR_get_state() for all threads. Using pthread_self() instead of getpid() solved the problem for me. -- kind regards Wolfgang Christ ? Hello OpenSSL Team, OS : MAC OS 10.6 i386 Version : openssl-0.9.8r/openssl-1.0.0d i got alloc/free errors under MAC OS 10.6 using OpenSSL within ERR_set_error_data(), because CRYPTO_thread_id() calling getpid() returns the same ERR_get_state() for all threads. Usingpthread_self() instead of getpid() solved the problem for me. -- kind regards Wolfgang Christ
getpid() not unique for threads on Linux 2.6 + NPTL
Hi,
I've been debugging my multithreaded application using OpenSSL for three
consequtive days, which crashed after processing about 5-200 SSL
transactions.
The reason turned out to be the default CRYPTO_thread_id()
implementation which uses getpid() as a thread identifier. This used to
work on non-NPTL linuxthreads but does not work on anything using Linux
2.6 and NPTL as in this combination getpid() returns the same value for
all threads.
I have solved the problem myself by supplying a custom callback using
CRYPTO_set_id_callback which uses the pthread id, however this problem
probably affects a lot of openssl users, who have applications that run
flawlessly on Linux 2.4 and after upgrading to a newer distribution the
application will crash randomly as the issue causes a heap corruption.
A clean non-pthread depending solution would be to use gettid(2) when
available, it returns the pid of the thread instead of the application.
(gettid is equal to pid in a non-threaded application).
The problem with gettid() that it is not available in the libc and one
has to use crude hacks to be able to use it. The following C programs
works for me:
#include unistd.h
#include stdio.h
#include sys/types.h
#include linux/unistd.h
#include errno.h
_syscall0(pid_t,gettid)
pid_t gettid(void);
int main()
{
printf(%d\n, gettid());
}
That code already contains a couple of #ifdefs anyway. The problem with
the current situation is that everything _seems_ to work well, but
whenever load hits the application it crashes and it is not easy to
debug especially when one is looking for an error in his own code :)
I would have opened a bug ticket but I have not found an obvious bug
tracking system, so I decided to send it to openssl-dev in the hope that
this is the right list. Please enlighten me if I am mistaken.
--
Bazsi
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: getpid() not unique for threads on Linux 2.6 + NPTL
Balazs Scheidler, 2006-03-31: The problem with the current situation is that everything _seems_ to work well, but whenever load hits the application it crashes and it is not easy to debug especially when one is looking for an error in his own code :) IMHO, the sooner the problem is detected, the better. Even if this implies a brutal crash of the application. On Linux, the current CRYPTO_thread_id() behavior with multithreaded applications hides the fact that the application is *broken*. For example, MySQL with OpenSSL has been broken *for years*. The problem was much more harder to trigger on Linux, because of the default CRYPTO_thread_id() behavior. Platforms without the getpid() - pthread_self() bijection (Solaris, NetBSD = 2, ...) happily crashed sooner, and more importantly: the problem gets fixed sooner, as well. Leandro __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: getpid()
On Thu, Jun 13, 2002 at 05:20:42PM +0100, Ben Laurie wrote: However, the number of calls is astonishing - and must be significantly expensive, too. Memory debugging *is* expensive. It is only enabled by default in debug configurations, where (starting with 0.9.7) it can be disabled by setting environment variable OPENSSL_DEBUG_MEMORY to off. (In non-debug configurations, it can be enabled by setting OPENSSL_DEBUG_MEMORY to a string other than off.) -- Bodo Möller [EMAIL PROTECTED] PGP http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/moeller/0x36d2c658.html * TU Darmstadt, Theoretische Informatik, Alexanderstr. 10, D-64283 Darmstadt * Tel. +49-6151-16-6628, Fax +49-6151-16-6036 __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: getpid()
On Sat, Jun 01, 2002 at 01:18:35PM +0100, Ben Laurie wrote: Also, the thread id may be used elsewhere - is there any point if its actually the PID? Applications that are actually multi-threaded should (and indeed, on most platforms, must) use CRYPTO_set_id_callback() so that OpenSSL can use appropriate thread IDs. To use the PID is merely the default behaviour of CRYPTO_thread_id(). This is generally not good enough for most purposes that OpenSSL uses the thread ID for (it can be helpful for memory debugging in programs that use fork(), though). -- Bodo Möller [EMAIL PROTECTED] PGP http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/moeller/0x36d2c658.html * TU Darmstadt, Theoretische Informatik, Alexanderstr. 10, D-64283 Darmstadt * Tel. +49-6151-16-6628, Fax +49-6151-16-6036 __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: getpid()
Bodo Moeller wrote: On Sat, Jun 01, 2002 at 01:18:35PM +0100, Ben Laurie wrote: Also, the thread id may be used elsewhere - is there any point if its actually the PID? Applications that are actually multi-threaded should (and indeed, on most platforms, must) use CRYPTO_set_id_callback() so that OpenSSL can use appropriate thread IDs. To use the PID is merely the default behaviour of CRYPTO_thread_id(). This is generally not good enough for most purposes that OpenSSL uses the thread ID for (it can be helpful for memory debugging in programs that use fork(), though). However, the number of calls is astonishing - and must be significantly expensive, too. Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
getpid()
Long ago, in a galaxy far far away, Solar Designer asked wtf openssl md5 calls getpid() a zillion times. The answer is memory debugging, which checks the thread id on every allocation/free. For reasons I haven't entirely fathomed, unless you are on Windows, what's returned is the PID. Whether this makes any sense at all, I don't know. Someone might care to think about it at some point. Also, the thread id may be used elsewhere - is there any point if its actually the PID? Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: getpid()
On linux, getpid() is different for different threads. /r$ __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
Re: getpid()
Rich Salz wrote: On linux, getpid() is different for different threads. /r$ Well... on FreeBSD (and Solaris) it isn't... Cheers, Ben. -- http://www.apache-ssl.org/ben.html http://www.thebunker.net/ There is no limit to what a man can do or how far he can go if he doesn't mind who gets the credit. - Robert Woodruff __ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
