* Jacques A. Vidrine ([EMAIL PROTECTED]) wrote:
> Prepare for a possibly dumb question. :-)
No, it's a good question.
> How are existing applications generally supposed to take advantage of
> ENGINE? I notice that in crypto/evp/c_all.c, there is an
> OpenBSD-specific hook to enable the `cryptodev' engine. This appears to
> result in (naive, existing) applications using the `cryptodev' engine
> whether they want to or not.
>
> Is there some configuration infrastructure that can be used to affect
> the behavior of existing OpenSSL-based applications with regard to the
> use of ENGINE? If not, should there be? What is the direction here?
If the application makes no ENGINE API calls, then no ENGINEs will be
brought into scope (ie. if you use static-linking, you will only get the
ENGINE core but won't link against ENGINE implementations). The ENGINE
core code maintains an implementation table against a variety of
"categories", check out crypto/engine/tb_*.c. These tables support
sub-classification by "nid" but only some of them make use of them, ie.
cipher, and digest will actually divide the table up according to the
'nid' of the algorithm required, whereas RSA is always RSA and so
there's only one division used - when 'nid' is hard-coded as 1.
Within these tables are some semantics that "just work" - they allow
ENGINE to function in a useful way whilst not causing compatibility
problems for applications that don't care and don't want to know.
However these semantics may not be everyone's "Right Way", so don't feel
embarassed if you find them odd.
Hm, where to start. What follows will be an ad-hoc run-down on this
stuff, <warning>please skip if you're not interested.</you were warned>
--- README.how-engine-implementations-are-registered-and-managed ---
The tables are implemented in crypto/engine/eng_table.c (see the
ENGINE_PILE structure, ENGINE_TABLE is merely an 'nid'-indexed LHASH
table of ENGINE_PILE items). An ENGINE_PILE represents all
implementations of a single algorithm - eg. be it "des_ede3_ecb",
"sha1", or "RSA".
When the "regular" openssl API is about to initialise a context
structure for performing one of these algorithms, it will either use a
supplied "ENGINE" from the caller, or if a NULL is supplied it will
instead request from the core engine code to nominate a "default". Eg.
RSA_new_method() in crypto/rsa/rsa_lib.c uses ENGINE_get_default_RSA(),
and that in turns calls engine_table_select() in crypto/engine/tb_rsa.c.
Unless an ENGINE is "registered" into an ENGINE_TABLE as supporting the
corresponding algorithm, that table remains NULL and so the
engine_table_select() calls trivially return NULL - causing everything
to happen in software. One reason for this is that then no cleanup is
required - the ENGINE_TABLE is never allocated and used unless an ENGINE
is actually registered first. At that same moment, a callback is
registered into the ENGINE_CLEANUP for the same table so that an
application can call ENGINE_cleanup(void) and know it will clean up
*whatever* is required (including, possibly, nothing). If you don't
register ENGINEs, you don't even need to call ENGINE_cleanup() - again,
some slippery coding tricks to ensure backwards compatibility.
So, where are we then - engine_table_select(). This uses the 'nid' to
home in on the required ENGINE_PILE in the ENGINE_TABLE, and this
ENGINE_PILE contains all the possible implementations of the required
algorithm plus a variety of caching/optimising magic. I'll try to
explain as briefly as possible how that behaves, but it'd help first to
forget all about ENGINE_TABLE, ENGINE_PILE, etc. That stuff above was
mentioned in case you find the below a little too hand-wavey - if so,
you can use the stuff above to dig further and see for yourself. Let's
just say that for any given algorithm, there's a table of
implementations stored in the ENGINE code, *OR* the table doesn't exist
because no implementations are registered. That "table" happens to be
an ENGINE_PILE structure if you care to know, but you certainly don't
have to.
Each table of implementations contains a STACK of structural references
to ENGINEs that implement the required algorithm. Many of these
references may not be able to function (ie. ENGINE_init() would fail on
them at run-time), but they exist and were registered by the application
so we store them. In addition to this STACK of implementations, we also
store an ENGINE pointer that will cache a functional reference to a
"default" ENGINE or NULL, and finally we also have a "touchfile" boolean
to prevent us performing a lot of logic if we know the table hasn't been
changed since we last did it.
Whenever you register or unregister an implementation from a table, the
"touchfile" is unset so the next call to engine_table_select() will
cause some logic to be performed to choose a functional reference to an
ENGINE to be returned to the caller (or NULL for no ENGNE). The
"selection logic" in engine_table_select() goes something like this
(whether touchfile is set or not);
- if our cached functional reference is non-NULL, it represents an
ENGINE that is already initialised and so is clearly valid and
working and used as our table's default. We call ENGINE_init() on
the reference to generate a new functional reference and return that
to the caller. Done.
- if our "touchfile" is set and the cached value is actually NULL,
then that is what we return anyway. This is because we don't want to
repeatedly try to initialise implementations that fail to
initialise, at least not unless the list of implementations has
changed. As this list is usually set once-and-for-all when the
application starts up, this means run-time performance should not be
affected once the used tables set their "touchfile"s.
- finally, we loop through our list of implementations and attempt to
ENGINE_init() each of them in turn. The first that succeeds, will be
set as our cached functional reference and a second functional
reference will be returned to the caller.
- in all cases, the "touchfile" is set before returning to the caller.
This may seem a little arbitrary, but the side-effects justify it. They
include;
- If you register a few ENGINEs for any/all algorithms they support,
then at run time you can simply use the RSA, DSA, DH, EVP_CIPHER,
etc APIs and they will automatically use whatever ENGINEs support
the required algorithms and are functioning. Moreover, the order you
add these ENGINEs dictates the order in which they are "tried".
Moreover, there are "set_default" alternatives to the various
"register" ENGINE API functions that will automatically try to
initialise the ENGINE before putting it in the table(s), and if it
initialises, will automatically cache the ENGINE as the default for
future use. The only way to switch defaults from then on is to set
a new default later, or unregister the ENGINE that has become the
default.
- An ENGINE that is functioning and being used for, say, RSA will have
a cached functional reference in the RSA table - as a consequence,
rapidly creating and destroying RSA keys will cause the ENGINE's
functional reference to rapidly oscillate between 1 and 2. If we
didn't cache a functional reference, it would rapidly oscillate
between 0 and 1 which would result in the ENGINE's initialisation
and shutdown processing to get thrashed. As this can include loading
shared-libraries and initialising hardware, it makes sense to ensure
this doesn't happen.
- Applications making no ENGINE API calls to register any ENGINEs will
not be "transparently" using ENGINEs at run-time, so the principle
of least surprise is satisfied.
- Applications wanting "easy acceleration" simply need to ensure that
any ENGINEs they are interested in are registered "for anything they
support", and if the corresponding devices exist and can be
initialised successfully, will be automatically be used for any/all
algorithms they support. Ie. KISS works for KISS requirements.
- Applications wanting fine-tuned control over which ENGINEs are
registered for which algorithms, particularly if they want to
support multiple ENGINEs for different roles, can be as specific as
they want to be. Eg. setting up an external hardware-PRNG, a
distributed-computing driver for RSA, and an on-board chip for
accelerating various DES modes.
And *finally* (I know, about time) you can globally control how all
these implementation tables work using the ENGINE_set_table_flags() API
function. Currently there's only one flag supported,
ENGINE_TABLE_FLAG_NOINIT, which if set will prevent any transparent
attempts to initialise ENGINEs. In this mode of operation, you can
register engines until you're blue in the face, but unless you
explicitly set one of them as a default they serve no purpose except
perhaps to make sure they are stored somewhere (so you can immediately
destroy any handles you had in application code and know
ENGINE_cleanup() will dismiss everything at application termination).
This means no ENGINE gets magically used without the application's
explicit say-so.
Suggestions for improvements in the behaviour or implementation are
welcome, and "diff -u" patches to make such changes are even more
welcome. :-)
Sorry if that's more than you wanted - I think this little splurge had
been a while coming so this seemed as good a time as any. I'll probably
also bookmark this one in the mail archives for future reference. Is
there anything from your point of view that I haven't adequately
explained?
Cheers,
Geoff
--
Geoff Thorpe
[EMAIL PROTECTED]
http://www.openssl.org/
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
