Re: [openssl.org #1346] Re: SSL_accept concurrency in 0.9.7j and 0.9.8b
On Tue, Jun 20, 2006 at 07:03:49PM +0200, Kurt Roeckx wrote: Applications are also expected to provide a thread ID callback by calling CRYPTO_set_id_callback(), although the failure to do so should not be a problem on Linux where different threads run with different PIDs, since OpenSSL uses the PID as a default for the thread ID. I believe this is the only LinuxThreads implementation that you're talking about. Since kernel 2.6 there is also support for the Native POSIX Thread Library (NPTL). Afaik, that doesn't change PID for each process anymore. I guess in that case one must use pthread_self(), which returns a pthread_t. Yes, true. Sorry for the incomplete and incorrect description. (OpenSSL requires the thread ID that is an unsigned long. Not all systems may provide this, but in practice, you can work around this problem by casting a pointer of any per-thread object in shared memory space into an unsigned long; e.g., do foo=malloc(1); and then use (unsigned long)(void *)foo as the thread ID. You might want to add assert(sizeof(void *) = sizeof(long)); to the program if you use this approach.) This would a problem on platforms like windows x64 which are LLP64, where a long is still 32 bit and a pointer is 64 bit. Fortuantly, we don't need that on windows. OK, I have implemented something new for OpenSSL 0.9.9-dev (this will become available in openssl-SNAP-20060624.tar.gz at ftp://ftp.openssl.org/snapshot/ in about 12 hours, and of course in later 0.9.9-dev snapshots): In addition to void CRYPTO_set_id_callback(unsigned long (*func)(void)); there will be void CRYPTO_set_idptr_callback(void *(*func)(void)); Same thing, just here the type of the ID is void * rather than unsigned long. Thus the malloc() trick will work. OpenSSL compares both IDs and believes that it is in a previous thread only if both values agree with what they previously were. The default value I have chosen for the pointer-type thread ID (if an application does not provide a callback) is errno. For most, if not all, platforms, this default might end all worries about CRYPTO_set_id_callback(). __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #1346] Re: SSL_accept concurrency in 0.9.7j and 0.9.8b
On Fri, Jun 09, 2006 at 07:02:36PM +0200, Kurt Roeckx wrote: On Fri, Jun 09, 2006 at 12:58:56PM +0200, Howard Chu via RT wrote: Howard Chu wrote: I'm seeing a lot of bad record mac errors when receiving a lot of connection requests at once. It sounds the same as this email http://www.redhat.com/archives/rhl-list/2005-May/msg01506.html which unfortunately was never replied to. Surrounding the SSL_accept call with its own mutex seems to resolve the problem. Is that supposed to be necessary? Given the lack of response here, we're tracking this now as http://www.openldap.org/its/index.cgi/Software%20Bugs?id=4583 The same problem occurs with 0.9.8b. There are various bugs open in Debian that might also be related to this: http://bugs.debian.org/198746 http://bugs.debian.org/212410 Please try verifying the bugs using the latest snapshot of your preferred version branch (0.9.7, 0.9.8, or 0.9.9-dev) from ftp://ftp.openssl.org/source and make sure that the affected multi-threaded applications do provide a locking callback by calling CRYPTO_set_locking_callback(). There are some recent changes in OpenSSL that may help avoid the bugs you are observing. Applications are also expected to provide a thread ID callback by calling CRYPTO_set_id_callback(), although the failure to do so should not be a problem on Linux where different threads run with different PIDs, since OpenSSL uses the PID as a default for the thread ID. (OpenSSL requires the thread ID that is an unsigned long. Not all systems may provide this, but in practice, you can work around this problem by casting a pointer of any per-thread object in shared memory space into an unsigned long; e.g., do foo=malloc(1); and then use (unsigned long)(void *)foo as the thread ID. You might want to add assert(sizeof(void *) = sizeof(long)); to the program if you use this approach.) __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
[openssl.org #1346] Re: SSL_accept concurrency in 0.9.7j and 0.9.8b
Current snapshots use a more thorough locking approach that takes into account inconsistent cache views on multi-processor or multi-core systems (where consistency can be reached by obtaining locks). The application has to call CRYPTO_set_id_callback() for OpenSSL to work properly. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #1346] Re: SSL_accept concurrency in 0.9.7j and 0.9.8b
On Tue, Jun 20, 2006 at 02:06:25PM +0200, Bodo Moeller wrote: On Fri, Jun 09, 2006 at 07:02:36PM +0200, Kurt Roeckx wrote: On Fri, Jun 09, 2006 at 12:58:56PM +0200, Howard Chu via RT wrote: Given the lack of response here, we're tracking this now as http://www.openldap.org/its/index.cgi/Software%20Bugs?id=4583 The same problem occurs with 0.9.8b. There are various bugs open in Debian that might also be related to this: http://bugs.debian.org/198746 http://bugs.debian.org/212410 Please try verifying the bugs using the latest snapshot of your preferred version branch (0.9.7, 0.9.8, or 0.9.9-dev) from ftp://ftp.openssl.org/source and make sure that the affected multi-threaded applications do provide a locking callback by calling CRYPTO_set_locking_callback(). There are some recent changes in OpenSSL that may help avoid the bugs you are observing. That is alteast good to know. Applications are also expected to provide a thread ID callback by calling CRYPTO_set_id_callback(), although the failure to do so should not be a problem on Linux where different threads run with different PIDs, since OpenSSL uses the PID as a default for the thread ID. I believe this is the only LinuxThreads implementation that you're talking about. Since kernel 2.6 there is also support for the Native POSIX Thread Library (NPTL). Afaik, that doesn't change PID for each process anymore. I guess in that case one must use pthread_self(), which returns a pthread_t. On my system both a long and pthread_t appear to be a unsigned long int, so I guess it should work in most cases without problems. Anyway, this is all explained in the documentation anyway, and I knew about it. I've also mentioned that in one of the bug reports. (OpenSSL requires the thread ID that is an unsigned long. Not all systems may provide this, but in practice, you can work around this problem by casting a pointer of any per-thread object in shared memory space into an unsigned long; e.g., do foo=malloc(1); and then use (unsigned long)(void *)foo as the thread ID. You might want to add assert(sizeof(void *) = sizeof(long)); to the program if you use this approach.) This would a problem on platforms like windows x64 which are LLP64, where a long is still 32 bit and a pointer is 64 bit. Fortuantly, we don't need that on windows. Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
[openssl.org #1346] Re: SSL_accept concurrency in 0.9.7j and 0.9.8b
Howard Chu wrote: I'm seeing a lot of bad record mac errors when receiving a lot of connection requests at once. It sounds the same as this email http://www.redhat.com/archives/rhl-list/2005-May/msg01506.html which unfortunately was never replied to. Surrounding the SSL_accept call with its own mutex seems to resolve the problem. Is that supposed to be necessary? Given the lack of response here, we're tracking this now as http://www.openldap.org/its/index.cgi/Software%20Bugs?id=4583 The same problem occurs with 0.9.8b. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sunhttp://highlandsun.com/hyc OpenLDAP Core Teamhttp://www.openldap.org/project/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: [openssl.org #1346] Re: SSL_accept concurrency in 0.9.7j and 0.9.8b
On Fri, Jun 09, 2006 at 12:58:56PM +0200, Howard Chu via RT wrote: Howard Chu wrote: I'm seeing a lot of bad record mac errors when receiving a lot of connection requests at once. It sounds the same as this email http://www.redhat.com/archives/rhl-list/2005-May/msg01506.html which unfortunately was never replied to. Surrounding the SSL_accept call with its own mutex seems to resolve the problem. Is that supposed to be necessary? Given the lack of response here, we're tracking this now as http://www.openldap.org/its/index.cgi/Software%20Bugs?id=4583 The same problem occurs with 0.9.8b. There are various bugs open in Debian that might also be related to this: http://bugs.debian.org/198746 http://bugs.debian.org/212410 Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
