Ah, the namespacing issue we've discussed
previously.
The module that I manage at CA
is a higher-level library that adds a
similar abstraction layer as yours, but there's now an internal requirement
to make the underlying OpenSSL interface also available to internal
developers. This was originally allowed for Open Source components that
use OpenSSL (OpenLDAP, Apache for example) but has been extended to
internal code that is already using OpenSSL. Our higher-level library uses
extensive reference counting to provide a robust interface to callers. We
of course do an EVP_cleanup in our xxx_lib_release() function. However,
with developers now also able to get at OpenSSL anyone of them could call
EVP_cleanup at anytime. Mandating that EVP_cleanup not be called other
than by my module is ludicrous due to the resource leaks, particularly if our
higher-level library isn't also used in the same
process.
I stongly feel that for OpenSSL to advance to the next
level that usability in enterprise applications has to be treated as a
first-class requirement. I don't see how this can be done
without breaking the current interface, but hey, that's why the version
number is <1.0. What I'd like to see
is:
1. A reference counted startup/shutdown function with
all functions that tear-down global structures controlled by these and otherwise
made unavailable to callers (ie. EVP_cleanup, CRYPTO_cleanup_all_ex_data,
ERR_free_strings, etc.). The init functions such as
xxx_add_all_algorithms() may be left as-is but the associated cleanup
functions are dangerous while public.
2. Namespacing supported. I've implemented this
in my own builds largely using the C preprocessor to "#define
RSA_new CA1_RSA_new" for example with some other changes required in
the perl scripts and some assembler modules. The prefix can be specified
at config time so that, in your case Peter, you could use a prefix of
IBM_. A default OpenSSL build continues to have no prefix. This
means that I no longer have to fear some other OpenSSL build finding its way
into our process at runtime and subverting our FIPS
routines.
An alternative to 2 is to move all functions into a
structure of functions pointers, a la PKCS#11, and have a single function
exported from OpenSSL, eg. OpenSSL_get_functions(), that returns this.
However, I'm guessing that the extra pointer indirection would be considered too
heavy handed. As is evident in the Python world though, trading a few
cycles for increased developer productivity, and in this case determinism and
stability, is a wise move.
I'm happy to contribute the changes, but don't want to
do all of the work unless the OpenSSL core developers agree on the
solution.
Regards,
Steven
From: Peter Waltenberg [mailto:[EMAIL PROTECTED]
Sent: Thursday, 4 August 2005 8:09 AM
To: openssl-dev@openssl.org
Cc: [EMAIL PROTECTED]
Subject: RE: Safety of using OpenSSL from multiple components in the one process
We created new entry points. There's a very shallow layer over the EVP layer which is mostly just a macro away from native OpenSSL.
That was done to satisfy other requirements, abstracting the OpenSSL API's a little so we could isolate IBM apps. from changes in OpenSSL, so we could slide in the FIPS self tests, so we could keep IBM owned code separate from OpenSSL licenced code etc.
I'm not going to claim that it's impossible to solve this in other ways, we just didn't have to try anything else since a solution that mostly worked already was imposed by other considerations in our case.
Our problems have been more with conflicts with other instances of OpenSSL in customer products. IBM libraries are used by customers who integrate them with their code as well.
Peter
Peter Waltenberg
| "Steven Reddie"
<[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/08/2005 10:26 PM
|
|
Hi Peter,
There is a requirement in my case for the OpenSSL API to be accessible to several components. In your case was your higher level component the only direct user of OpenSSL? If not, how did you make the API available to other components? Were the new functions accessed from one DLL with the OpenSSL functions accessed directly from the other DLL, or did you have redirecting stubs in your new DLL?
Steven
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Peter Waltenberg
Sent: Wednesday, 3 August 2005 8:48 PM
To: openssl-dev@openssl.org
Cc: [EMAIL PROTECTED]
Subject: Re: Safety of using OpenSSL from multiple components in the one process
You are correct it is a problem in enterprise applications.
The good news is that it can be resolved, the bad news is that we did that outside the OpenSSL codebase itself.
We created new startup/shutdown functions in another library, added reference counting and load OpenSSL via that.
It might be possible to do something simillar within the existing API's, but brute force did work.
Peter
Peter Waltenberg
| "Steven Reddie"
<[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 03/08/2005 06:42 PM
|
|
Hi All,
This is something that I think I've raised before but don't remember getting resolution on.
OpenSSL maintains various global variables and structures, and there are cleanup functions that must be used to properly release the resources when finished. One example is the OID database managed by the "add_all_algorithms" function and it's associated release function, EVP_cleanup. All is good when the use of OpenSSL is fairly simple, such as a single component using it for the lifetime of the process.
Where things get difficult/dangerous is when multiple seperate components in the one process, with no real knowledge of each other, make use of OpenSSL, and it's even worse if they dynamically load and unload OpenSSL using dlopen/LoadLibrary. With large enterprise applications this is a common situation since different teams develop components that the large product makes us of, and with the increasing use of "plug-in" architectures the dynamic loading/unloading is not uncommon.
There seems to be no way offered by the OpenSSL API for these components to behave well. If they each do a dlopen -> dlsym -> ... -> EVP_cleanup -> dlclose sequence then it seems that they will trample on each other. If they take the extreme opposite and don't call EVP_cleanup then the process will leak until it falls over.
This is a serious issue that I believe impacts the stability and therefore limits the usefulness of OpenSSL in large enterprise applications. Does anyone else have any thoughts on this?
Regards,
Steven
