Hey there,
I sense more clarification is needed, and as ENGINE stuff was only very
recently migrated from its own little development branch of CVS into the
main trunk, it's probably overdue for some general discussion anyway. So
here goes a first crack at it - feel free anyone to open this right up.
BTW: this'll probably be a bit of a marathon ...
On Wed, 1 Nov 2000, Ben Laurie wrote:
> Geoff Thorpe wrote:
> >
> > BTW: Right now, all the existing engine implementations typically work
> > immediately without any "setup" beyond what they work out for themselves
> > before, during, or after initialisation.
>
> This is true. But clearly there are other engines where this isn't the
> case. Anyway, I think the idea of driving ENGINE_ctrl()s from the
> configuration isn't so good, because it leads to obscure configuration.
> Probably its better to pass a configuration handle into the
> ENGINE_init() and let it pick up whatever it needs from there itself?
Well the point behind ENGINE, and I can say this on account of the fact
that I started the original hacking of it to solve a need I and others had
identified, was *not* to make it possible to switch one piece of hardware
for another and have things work transparently by changing OpenSSL's
ENGINE at run-time. That is *completely* unrealistic except for one very
specific use-case; namely acceleration-only. It just so happens that this
very case is the one where in theory no extra settings need to be
configured because the ENGINE specific to the accelerator only needs to
*find* the hardware using the accelerator's local installation details and
can start firing off operations immediately. Other cases simply do not so
readily gel with the idea of "switch the ENGINE and go", eg. hardware
key-management, distributed environments, or anything requiring run-time
setup or authorisation.
So, let me explain what the point behind it *was* (lest I sound too
negative :-) - it was so that software that supported the ENGINE API (and
I should qualify that they need to support it *properly*) would not need
to have vendor-specific support hard-coded in to make it possible to
support new or alternative devices. It doesn't remove the need for having
the locally installed device and support software properly configured -
including whatever its concept of "configuration" might be. This idea is
useful not just in apps that are distributed binary-only, but also in apps
that are shipped with source but don't want people having to needlessly
hack it. Eg. an SSL module in Apache should be able to pluck a directive
out of a config file and say "find me this 'ENGINE' and set it as my
default". If you later go in and add a new ENGINE implementation for a new
hardware (or software) unit, and recompile, and you ensure that the
environment of that "unit" is functioning OK in and of itself, then the
SSL module itself should not need to be upgraded or rewritten to get the
hooking to work. If settings are needed by that alternative ENGINE, they
presumably can be found inside the environment in which that new ENGINE
and its device operate - they just don't need to be hacked into OpenSSL or
applications any more.
Any attempt to assume some parameterisation of all ENGINEs in a generic
fashion runs straight up against the fact that it will fail when it
encounters a new kind of ENGINE that wants or needs to do things
differently. Hence me prattling off that hypothetical list of the types of
garbage any given engine might need to for it to "go". The idea was not to
somehow make the ENGINE API an all-singing all-dancing framework where
any/all devices might be supported transparently, and indeed trying to do
so would assume you can generalise the use-cases for all engines and cover
such things (which is simply a counter-intuitive thing to try). The point
was simply that if you add a device/unit and have that device/unit
configured and operating on your system, there's no need to go hunting for
upgrades to your applications for them to use this new device. That's all.
Every device and its ENGINE implementation will need different things for
it to be usable, just as each device itself needs to be installed and
configured on the host system independantly of OpenSSL anyway.
Now, anything new and cool that can be done to make the general class of
device-support slicker and less problematic is great. Particularly if it
centralises a lot of code that would otherwise be duplicated in lots of
ENGINEs. But as someone pointed out, you couldn't simply have a
software-only solution running with the ENGINE API, and expect to plug in
a new PKCS#11 device and have the application switch over transparently.
The ENGINE specific to that device (or PKCS#11) will need to know
addresses, usernames, and whatever else is required to deal with it, and
presumably private keys and what-not will have to be setup and exported in
some kind of PEM equivalent that will be specific to the device and its
support software. If you do try to generalise stuff like this in OpenSSL
itself, you not only open a massive can of worms, you also immediately
create a situation where ENGINEs that work "like this" fit and those that
don't are left where hardware support in OpenSSL used to be - namely,
waiting around for "non-standard" hacks to the library and the
applications that use it.
Example: currently, if someone goes to the effort of installing an nCipher
unit, its support software, creating hardware-managed keys, setting up
smart-card administration, creating and signing hardware-managed certs,
exporting and converting to PEM, and other such things, blah blah blah ...
once all *that* is done, all that should be required is a switch in the
OpenSSL-based software to point to the nCipher ENGINE, and then it should
work "transparently". It should not be necesary after completely
configuring the device itself with keys, smart-cards, etc, to then need to
go and get vendor-specific hacks to the raw OpenSSL library and the
applications that use it. The point of ENGINE is in a nutshell to solve
that second point - no hacking to OpenSSL or its applications needed, it
is not to solve the first step - setting up and configuring everything
that is implicitly specific to the hostsystem and the cryptographic unit
you wish to use.
That's what the "abstraction" is about. In many ways not too disimilar an
aim to PKCS#11 itself, except that by definition the common interface used
here fits in a lot more seamlessly with OpenSSL because it was developed
to do precisely that. It's also a software-friendly approach that places
more "detail" in the hands of hardware developers and implementors.
PKCS#11 is broadly-speaking the other way round, which makes it less
appetising as a solution primarily for software with hardware on the side
as a bonus. An illustration - any common interface that assumes you can
start operations going asynchronously and come back to them later will fit
into OpenSSL rather *badly*. I think maybe some people's perception of
ENGINE is that it should simply mean you can do away with dealing with
crypto units directly and never do anything specific to them. Unless you
live in an acceleration-only world, probably with plug-in-play setup of
the devices too, that is *really* unrealistic right now. We're simply
trying to cut down on the amount of hacking to source code that's needed
after one has finished setting up and configuring such devices.
I'd recommend that if someone really wants to dig further into this and
sees it as an issue that needs to be addressed, that they first get a
couple of hardware units, play around with the ENGINEs that support them,
try and look at the breadth of things the general class of "devices" might
need to operate, and only then try to take stabs at how to abstract all
that. They may find it's a lot less straightforward than they first
envisaged, but the effort would certainly be very welcome. Taking
pot-shots at "general solutions" on mail lists, without doing any direct
experimentation or research as to how that would function in reality, is
great for brainstorming but entirely insufficient on its own.
Cheers,
Geoff
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
