Renier raised questions about the SSL initialization routines in this e-mail. I was able to get many of them answered using the OpenSSL user's mailing list, and that response is appended to this e-mail.
Below, I've tried to answer Renier's questions, based on my understanding and the mailing list response. Bryan On Mon, 2008-03-10 at 14:23 -0400, Renier Morales wrote: > > OpenSSL requires some library initialization before use. > Additionally, > > for a multi-threaded environment, the library requires that the user > > provide some thread-safe setup, implemented as a set of calls the > user > > must make before using the library. > > > > It's difficult for a plug-in to do this. By the time the plug-in is > > initialized, other plug-ins may be called. Each plug-in doesn't > know if > > it's the first, and therefore whether to do the initialization. > It's > > possible that the SSL library initialization call can be performed > > multiple times (would have to check the SSL source code to be sure) > > Would like to know for certain about this. The response from the mailing list indicated that it only should be done once. Given that the thread setup code really can only be done once, I'm not sure this really matters. (If we have to make sure one piece of this is only done once, then we might as well do the other pieces only once.) > >, but it's unlikely that the thread setup can be done more than once, > looking > > at the implementation of those calls. > > Is there a way to know if the openssl thread init has already been > done? Maybe by checking if there are any callbacks set? The response indicated that there were "get" as well as "set" calls, though they are not in the man pages. I will try this ASAP and report back. > Yes, the initialization code can be run from OpenHPI's initialization > code and we can create an ssl directory, like the snmp directory, that > contains your ssl convenience functions for others to use. > > But before this, some questions: > - What number will CRYPTO_num_locks() return and do you need to create > all of them upfront? I'm worried about creating a bunch of lock > instances unnecessarily. OpenSSL currently wants us to allocate 39 locks. My quick experiment indicates that 16 of these were actually used during a run of a program using my plug-in. We discussed, on IRC, that it might be best to allocate locks the first time they are used, to avoid the overhead of creating locks that a) aren't used or b) when no plug-in is using OpenSSL. I will pursue such an implementation and post a follow-up here. > - Also, is it possible to use glib locking mechanisms, instead of > directly using pthread_*? OpenHPI uses glib locks for all locking > needs and it works on more platforms. Yes...I'll re-implement. > - One plugin won't use the SSL connection of another, so I don't see > thread-safety issues going across plugins. Am I right? Also, I don't > see any thread-safety issues with SSL connections unless a plugin uses > the same SSL connection from different threads. Is this how your > plugins will work? If so, would there be a problem locking around your > SSL connections using glib locks? (I think this page, > http://openssl.org/docs/crypto/threads.html, implies locking around > ssl connections is not enough). The response from the mailing list, as well as the man page and FAQ indicate that the locks are necessary to protect OpenSSL's internal data structures, independent of connection. In other words, OpenSSL needs them for any multi-threaded application, *and* a multi-threaded application is still not allowed to share connections between threads. This is the rule our plugin follows, not sharing connections. Yet, apparently, the locks are still needed. RE locking around all SSL calls, this is sufficient, but would need to be done across all plugins. For example, if one makes an SSL call, then no other plugin could make an SSL call. During, e.g. discovery, this could cause one plugin to have to wait for another. I wouldn't recommend this approach. > - Is it really necessary to use dynlock callback/functions? According > to http://openssl.org/support/faq.html#PROG1, a minumum of two > functions are required. The notes at > http://openssl.org/docs/crypto/threads.html say that dynamic locks are > currently not used internally in openssl. Again, the mailing list confirms the notes you cite. The dynlock functions are not necessary right now. On the other hand, some future version of OpenSSL will use them, and then we'll have subtle bugs. For the dynlock functions, especially, I think we should just provide them. They don't require allocating a bunch of locks like the first two calls do, so there is little overhead in setting up the dynlock callbacks. As you pointed out earlier, the implementation should use glibc calls instead of pthread calls. -- Bryan Sutula <[EMAIL PROTECTED]> -------------------------------------------------------- OpenSSL User's Mailing List Response Subject: Re: Clarification questions on OpenSSL thread-safe support From: Geoff Thorpe <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Cc: Bryan Sutula <[EMAIL PROTECTED]> In-Reply-To: <[EMAIL PROTECTED]> Date: Tue, 11 Mar 2008 11:37:49 -0400 Message-Id: <[EMAIL PROTECTED]> On Mon, 2008-03-10 at 17:23 -0600, Bryan Sutula wrote: > My questions: > 1. What I understand from this is that OpenSSL can be thread safe. > In order for it to be safely used in multi-threaded > applications, it needs: > A. to be built with multi-threaded versions of the standard > libraries, > B. to have the application provide the two callback > functions, and > C. the application must avoid using the same SSL connection > by two different threads. > All of the above are necessary. In other words, it isn't > sufficient that OpenSSL was built with the multi-threaded > versions of the standard libraries. The application must also > set up the callbacks. (True or False, please?) True. OpenSSL's role is not to be a threads implementation, nor a wrapper around all known or conceivable threads implementations, it's role is to allow you to glue it to the thread implementation you intend to use. Hence the callbacks. > 2. Related to question 1, the thread-safe requirements (A and B > above) are needed even if the different threads are not sharing > an SSL connection. (My understanding is that connections can't > ever be shared, and that the library still needs A and B in > order to be thread-safe.) (True or false?) True. The different SSL contexts operating in different threads will still be using shared structures that need to be synchronised. Eg. the SSL_CTX that the SSL objects were derived from, certificates and certificate-stores, public-keys, some global data specific to some other corners of the libcrypto and/or libssl implementations, etc. > 3. Instead of B (implementing the two callback functions), is it > sufficient for the application to provide it's own locking > around all SSL library calls? In other words, if the > application guarantees that only one thread will be in the > library at a time, is that sufficient? Sure, but you'd have to mutex all accesses to *any* openssl interface. Eg. see my answer to 2. If time spent in openssl is a miniscule percentage of your run-time profile, then this may be tolerable. But otherwise, you really don't want to be serialising all threads for any use of openssl interfaces. The locking callbacks allow you to implement the minimum required locking at the appropriate granularities (there are CRYPTO_num_locks() different locks). The contention is likely to be a lot less than if you lock your own code before it calls into openssl ... > 4. I'm guessing from the semantics of CRYPTO_set_locking_callback() > and CRYPTO_set_id_callback(), that they are not to be called > more than once from an application. It seems like they have to > be called only at the beginning of the program, and not ever > again. (True or False?) Is there a way to know if they have > already been called later on? True, I think. I guess if you have some guarantee that the process is quiesced (ie. "everyone please shut up for a moment while I change the locking"), then you could conceivably change these. But not "on the fly", if you get my point. There may also be some "fail if they've already been set" checking in openssl, in which case this would fail. As always, you have access to the source code - feel free to dig. Finally, yes, you can call CRYPTO_get_locking_callback() and CRYPTO_get_id_callback() to see if they've already been called. > 5. There are some other "dynlock" functions described in the > threads(3) man page. The wording on that page implies that they > are only needed for performance, or maybe in a future version. > In my current application, they don't seem to be called. Is it > necessary to implement these? Will they only be for > performance? If I don't implement them, will my application > break in some future version of OpenSSL, or will it just run > slower? (The confusion results because the current man page has > wording: "Multi-threaded applications might crash at random if > it is not set", but also says "dynamic locks are currently not > used internally by OpenSSL, but may do so in the future" and > "some parts of OpenSSL need it for better performance".) What's > the real situation here? I agree that this is fuzzy. It should be fine not to provide dynlock functionality for now, but of course be aware that if you maintain your code going forward there may come a time where functionality within openssl (or more likely, run-time/loadable extensions to openssl, such as hardware/enhancement support) will fail due to its inability to create new locks on the fly. Only you can judge whether this is or will be an issue. > 6. Question 4 applies to the dynlock setup functions as well. Same > answer about calling them multiple times? Any user-callable API > to know whether they've already been called? Yes, they all have s/set/get/ counterparts you can call to retrieve the currently registered callbacks. They should be NULLs until set otherwise by the user of the API. BTW, please correct me if you find this not to be the case (eg. if there is an internal lazy-initialisation to a non-NULL callback) because any "falling back to a default implementation" should be done without needing to internally set callbacks non-NULL. > 7. Not specifically concerning threads, but is it safe to call > SSL_library_init() more than once? (Does the library protect > against that, returning immediately if the initialization is > already done? Many libraries do this.) How about > SSL_load_error_strings()? ERR_load_BIO_strings()? AFAICT no, you should avoid calling these multiple times from the same address space. But as you're the guy providing the locking callbacks, I'm sure you can find some mechanism on "your side" to make sure the initialisation only happens once? :-) Cheers, Geoff ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED] ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Openhpi-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/openhpi-devel
