Thor Lancelot Simon wrote:
On Thu, Sep 10, 2009 at 06:10:27PM +0200, Dr. Stephen Henson wrote: > On Wed, Sep 09, 2009, Thor Lancelot Simon wrote: > >> On Sat, Aug 29, 2009 at 05:34:04PM -0400, Steve Marquess wrote: >> >> That this wasn't the obvious approach from the very beginning >> speaks worlds about the limitations of the ENGINE interface.
>>
> The actual story of why FIPS is the way it is is rather different. > I think a few home truths are in order on this and some related > issues.
Ok, I'm going to jump in here. While not a member of the OpenSSL team I do have direct knowledge of their resource and funding situation.
... I've been involved in two FIPS validations of vendor versions of OpenSSL. I think one of them may have been one of the first ones ever done. I am aware of how much work you must have done to get things even into the state they are in today -- though I certainly didn't know it was unfunded.
Yes, that's a key point. The key point, actually. We have had some funding, but the great bulk of that was spent on the test lab fees. I haven't kept track of the uncompensated volunteer effort but it easily totals to well over a man-year. Alas, much of that effort was expended running in circles as we converged on a solution that would satisfy the peculiar requirements of FIPS 140-2. In such circumstances the major challenge was not to implement the *best* solution, but to implement *a* working solution before time and money ran out. We came very close to giving up at several points.
On the other hand, I think at this point it is not unreasonable to talk about how design decisions made elsewhere in OpenSSL in the past (and even in the not-so-distant past) makes this kind of systematic change harder. From my point of view, if I were to set out to do a de novo FIPS compliant version of OpenSSL today, the largest obstacle would be the current ENGINE interface. With a more capable interface to ENGINEs, durable FIPS support would be much easier. I'm sorry if I've offended you by saying so, ...
I think Steve is not so much offended by your observation as incredibly frustrated by the fact that he has wanted to do an elegant and robust FIPS implementation from the very beginning. He's never been given the chance -- by now he's spent a lot of time in total on the FIPS stuff, but never in a block big enough to do the necessary rework in one swell foop. It's always been one fitful tweak after another. I've had to repeatedly tell him, "that would be nice but we have to go for the quick fix before we run out of [time|money]".
There is much in OpenSSL that could be improved. Steve Henson has for years directly related to me a number of architectural issues the team would like to address. The reality is that OpenSSL, unlike Apache or Linux or OpenSSH or Cygwin or almost any other significant open source product you care to think of, has never had any significant corporate backing. None of the team members with salaried jobs are tasked by their employers to spend significant amounts of time on OpenSSL. The ones who free-lance have to get their income where they can find it. OpenSSL is a commercial grade product maintained on a hobby budget. The wonder is that it's as popular and widely used as it is.
Steve Henson, who has no regular paycheck, does a lot of work on OpenSSL but has to live on very erratic income from ad hoc consulting work. He'd much rather work on OpenSSL but that consulting work sometimes leads elsewhere. I know Steve's situation best because I've been involved in most of the consultancy contracting he's done for the past few years. Last year he had a modest but respectable income, but this year he's made far less than he would have stocking shelves at the local supermarket. That sporadic income not only forces him to spend time on non-OpenSSL issues, but also means he can't budget long blocks of time for the sustained concentration that big architectural changes would require.
On what is largely a separate topic, but an important one: You complain that commercial entities use OpenSSL but don't fund it. From my point of view, it is not hard to understand why: it is usually far easier to contribute code or development time than cash, but every time I, at least, have proposed contributing any kind of code to OpenSSL which would involve a structural change, it's been either ignored or shot down by the OpenSSL developers. In several cases developers have privately told me they were working on somethng else better than what I was offering to contribute -- but then that code mysteriously never showed up in the tree.
Not mysterious at all -- for all of them their OpenSSL work has to play second fiddle to making a living. Yes, they should communicate better and so forth, but...
So what happens? Other people responsible for development of vendor versions of OpenSSL at other commercial shops presumably end up in the same situation I do: with trees full of private modifications and enhancements and a great deal of frustration towards OpenSSL itself. At this point, having gone hat-in-hand to my management a number of times asking if I could have my team write (or rewrite) this or that for contribution to OpenSSL, and then having found that OpenSSL wasn't interested or, often, even responsive, I have basically no credibility left
Code contributions are great but it's not as simple as "here's my patch, all you have to do is apply it". I learned that lesson myself on OpenSSL and other projects, that there can be a big difference between the patch that works fine in my specific environment and one that will work on all platforms for all existing usage *and* which doesn't conflict with future plans.
First and foremost, OpenSSL is a security library and many pieces of critical infrastructure rely on it. Code has to be very carefully examined for possible security issues. Let your guard down and apply patches blindly and you get the Debian fiasco.
Then there's the fact that expedient fixes in the past have resulted in spaghetti code and real problems later on because an API wasn't flexible enough to handle future expansion. That's been a big problem that has effectively paralysed future development because some things just can't be done without completely new APIs. EVP and PKCS#11 is one case in point. Having been burnt by this in the past and vowing not to repeat the mistake the team approaches some changes with great trepidation, which given the resource issue means they are sometimes just put off indefinitely.
Finally there is application compatibility, on multiple platforms. If they make a change that breaks lots of existing applications they are the ones that will be lynched, not the patch contributor. Functional changes have to be finessed in a way that causes minimum breakage while also supporting future functionality.
The point here is that code contributions could be more effectively leveraged *if* there were enough team resources to do them justice. As it is they are sometimes effectively ignored for lack of time to properly review them.
... to ask that we contribute _anything_ -- certainly not cash which will presumably be spent on development priorities which share little with ours.
Actually quite the opposite is true -- paying sponsors tend to get exactly what they want, even if it never makes it into the baseline. We consider it a side benefit when paying work results in something that can be merged in the baseline (sometimes the result is not of general utility, sometimes the client wants it to stay proprietary). CMS, PKCS#7 (S/MIME), much of FIPS v1.2, RFC3280 compliance, some PKCS#12, and CryptoAPI ENGINE are all in OpenSSL thanks to commercial sponsors who obtained exactly what they needed, on budget and on schedule, while also contributing to the community.
Also note I'm not talking about donations, this work-for-pay is done under standard commercial terms with non-disclosure agreements, formal proposals and legally binding contracts. In fact after years of ad-hoc subcontracting arrangements the OpenSSL Software Foundation was formed to be the corporate manifestation of OpenSSL for just such commercial support. These guys aren't looking for charity, they're looking for work that will let them improve OpenSSL while still putting food on the table. If we could just get that to a predictable three squares a day for one or two team members then you'd see a huge boost in responsiveness and progress.
This is not a good situation.
Agreed, it's not and what's more I don't think it's a stable situation. Something has to break soon or OpenSSL is at risk of disintegrating into that morass of quasi-neglect you've noted.
-Steve M. -- Steve Marquess OpenSSL Software Foundation, Inc. 1829 Mount Ephraim Road Adamstown, MD 21710 USA +1 877-673-6775 marqu...@opensslfoundation.com ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
