On Thu, Aug 15, 2013 at 10:58 PM, Patrick Pelletier <c...@funwithsoftware.org> wrote: > On 8/15/13 10:24 AM, Nico Williams wrote: >> . Recent developments, like Android's failure to properly initialize >> OpenSSL's PRNG make me think it's time to table (in the British sense) >> the issue once more. > > Can you point to any article or post which explains exactly what the OpenSSL > half of the Android issue was? (I understand the Harmony SecureRandom > issue, but that's a separate thing.) OpenSSL is supposed to call RAND_poll > on the first call to RAND_bytes, and RAND_poll knows how to seed from > /dev/urandom on systems that have it, which should include Android. Neither > I nor others speculating on Google+ could figure out why this wasn't the > case, and why explicit seeding would have been necessary: > > https://plus.google.com/+AndroidDevelopers/posts/YxWzeNQMJS2
Hmm, I've only read the article linked from there: http://android-developers.blogspot.com/2013/08/some-securerandom-thoughts.html Not enough info there :( I don't really feel like finding the relevant OpenJDK JCA code, nor the Android derivative of it. It'd be easier to ask them for more details. If they were wrong about OpenSSL in this respect and the problem was truly specific to Android then I apologize for spreading a falsehood. > Figuring out the right sequence of initialization functions to call, even > when just one application is using OpenSSL, has not been entirely clear. In > particular, see my rambling talk-page discourse on the OpenSSL Wiki about > what is and isn't necessary in order to get ENGINEs initialized, and how it > depends upon some build-time #defines in a non-obvious way: I'll take a look. For now it seems there should be no need to set any thread-related callbacks, no? Or if they are needed, we should make them no-ops on OSes with thread libraries. > > [...] > Thank you! I'm glad I'm not the only one who feels this is a big problem. > It's something I've expressed concern about in the past, albeit in a > parenthesized paragraph about halfway through a long, far-reaching rant: > > http://lists.randombit.net/pipermail/cryptography/2012-October/003388.html I've ranted myself about this privately many a time. I... just never got involved. I felt this [possibly incorrectly attributed to OpenSSL] event was the straw that broke the camel's back for me. I think that a crypto library should have no worse initialization/finalization/thread safety/fork safety semantics than libpkcs11 in Solaris/Illumos, which: a) thread-safely ref-counts C_Initialize()/C_Finalize() calls, b) leaves locking around objects to the app, c) re-initializes (and loses all objects) on the child-side of fork(), d) no thread/lock callback setters. (It's necessary to finalize all objects that refer to sessions for crypto coprocessors, TPMs, tokens, as well as any other stateful objects on the child side of fork(), either that or establish new sessions. It shouldn't be necessary to finalize key objects, say, but hey, it's what PKCS#11 requires.) Better yet: implied initialization (using pthread_once()). > * using the OpenSSL Ruby Gem, while also using another Ruby Gem that depends > on OpenSSL indirectly > > * using OpenSSL directly while also using libevent's optional integration > with OpenSSL I'm sure there's many more :( >> Huh? Which is it? Must apps call CRYPTO_THREADID_set_callback() even >> though "[o]n Windows and many Unix systems, OpenSSL automatically uses >> the multi-threaded versions of the standard libraries"? Why? One >> would think that the "multi-threaded versions of the standard >> libraries" on such OSes would provide all that OpenSSL needs. I don't >> get it. > > > I think what it means is that "We link against thread-safe versions of the > standard library (which implies there are thread-unsafe versions of the > standard library, which I think might be true on Windows, but I'm pretty > sure there isn't any thread-unsafe version of the standard library on Linux > or OS X) so that, for example, errno doesn't get clobbered by multiple > threads. But we still don't call any threading functions to make OpenSSL > itself threadsafe, so you'll have to provide this boilerplate yourself in > every OpenSSL application you write." Well, *static* libc.a's tend to not be very thread-safe. Solaris < 10 and Linux still today both have a number of process models: - dynamically linked, linked with a threading library - dynamically linked, NOT linked with a threading library - linked with static libc, linked with a threading library - linked with static libc, NOT linked with a threading library and interesting things happen if in the "not linked with a thread library" cases if an object loaded with dlopen() does link with a thread library (even more interesting if the program is statically linked with libc). The dlopen() upgrade to threaded model case happens when you use the name service switch without an nscd daemon or PAM or... A lot of people seem to love static linking. There are problems with dynamic linking, to be sure, but static linking with dynamic dlopen() is insane, and anyways, static linking unfortunately means having to keep track of total dependencies at the program link-edit stage, which is also basically insane (though at least that could be fixed). I'm sure you need to support some of that insanity. > I could also rant about OpenSSL's "homemade" thread local storage (TLS, in > the other sense of the acronym) implementation, rather than using the OS's > API for thread local storage. This can result in memory leaks when a thread > terminates, unless you have complete control over the lifecycle of every > thread and can call a cleanup function before the thread terminates. (And, > as you've noted, in a modern application with many layers of libraries, you > don't have such control.) Oh. Is there any reason not to blow that away, or at least build-time select which to use? >> Finally, what can I do to help? > > Definitely to the extent that you can figure out what the actual behavior is > (although that itself is not easy and usually requires experimentation and > studying the source code in-depth), document your findings on the OpenSSL > wiki: > > http://wiki.opensslfoundation.com/ OK, I'll take a look. > (Though hopefully any such clarifications can be propagated to the "more > official" documentation as well.) > > And I certainly hope that the initialization and thread-safety issues can be > addressed in libcrypto itself, but if they can't or won't, then I think we > (meaning those interested) should write a small add-on library to go along > with OpenSSL, that would include "standard" implementations of the thread > safety callbacks, and perhaps we could convince users of OpenSSL to accept > it as a de-facto standard, to avoid some of the "multiple libraries trying > to set their own thread callbacks" issues. (I've actually been meaning to > write such an "unofficial companion library for OpenSSL" for a while, but > haven't gotten around to it yet.) Great. I was hoping that the response wouldn't be something like "no way, we need these callback setting functions for XYZ reasons" or, worse, "no way". Yes. Nico -- ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
