Re: [cryptography] Just how bad is OpenSSL ?
The source code is mostly written to the OpenSSL coding standards, which are seriously different from any other coding standard I've seen (it's not Linux/KR, nor GNU, nor Microsoft, nor Sun/Oracle). Nonconformance with the coding standards in later patches is very common, so it's a mishmash of indentation standards on top of that ... Sorry to dig up an old thread. This seems news worthy for anyone who has spent time staring at the sources. [openssl-users] OpenSSL source reformat, http://openssl.6102.n7.nabble.com/openssl-users-OpenSSL-source-reformat-td55691.html. On Fri, Oct 26, 2012 at 3:38 PM, Andy Isaacson a...@hexapodia.org wrote: On Fri, Oct 26, 2012 at 06:29:47PM +, John Case wrote: So, given what is in the stanford report and then reading this rant about openssl, I am wondering just how bad openssl is ? I've never had to implement it or code with it, so I really have no idea. How long has it been understood that it's a mess (if it is indeed a mess) ? How dangerous is it ? It looks like the rant was published in 2009 Bad is such a subjective measurement. OpenSSL is very very hard for a non-expert to code against. It's hard to figure out what interfaces you should use, what interfaces are well tested, what interfaces are known to be unsafe, and what interfaces are buggy but can be used safely with careful coding. It's fairly easy to accidentally disable security critical codepaths in the process of iterative hmm that doesn't quite work, the docs are unclear, maybe this is a bug in my code or maybe a bug in OpenSSL? that is a normal part of software development. If you need to implement anything even slightly different from what was expected by the authors. The source code is mostly written to the OpenSSL coding standards, which are seriously different from any other coding standard I've seen (it's not Linux/KR, nor GNU, nor Microsoft, nor Sun/Oracle). Nonconformance with the coding standards in later patches is very common, so it's a mishmash of indentation standards on top of that. Naming conventions sometimes indicate that functions are strictly internal and should not be used by applications, but sometimes you have to use an internal API to get a necessary result and other times there are clearly internal APIs in the public namespace. I could go on. Overall, I would say that yes, OpenSSL is a huge mess for application developers. In that sense, it's very bad. On the other hand, it's the most thoroughly reviewed open source crypto implementation, and hasn't had very many security bugs found in the library per se. Its performance is fairly good. In that sense it's still the best option for some use cases. ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Mon, Nov 12, 2012 at 2:32 AM, Jeffrey Walton noloa...@gmail.com wrote:
GCC really should provide a function like SecureZeroMemory that cannot
be optimized away. Its easier than educating every developer about the
optimization issue and telling them to compile with -O0. The Ostrich
Algorithm at its finest.
I'm not disagreeing with you, but I predict with some confidence that
the GCC developers will say it's not necessary because you can write
memset(buffer, 0, length);
asm volatile( : : m (({ struct { char b[length]; } *p = (void
*)buffer; *p; })));
to make it appear that 'buffer' is read from immediately after the
'memset', without further optimization consequences. (Yes, that mess
after m is the documented way to express *how much* memory is read.)
zw
___
cryptography mailing list
cryptography@randombit.net
http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Mon, Nov 5, 2012 at 5:07 AM, Nico Williams n...@cryptonector.com wrote: On Sun, Nov 4, 2012 at 8:42 AM, Ben Laurie b...@links.org wrote: On Sat, Nov 3, 2012 at 12:26 AM, James A. Donald jam...@echeque.com wrote: On Oct 30, 2012 7:50 AM, Ben Laurie b...@links.org wrote: The team has ruled out having the master at github. What is wrong with github? TBH, I wouldn't mind much, but I think the concern is that its not under our control. It's just git, so keep multiple clone repos. You could use an internal one as the master and push updates to the github one if you don't trust github -- use github to serve outsiders. That is exactly the plan. Really, what matters is that you have one master repo and all other official repos be read-only clones of it. As with any master/slave failover/takeover scheme you can always recover from the death of the master by promoting a clone to master status. So why not trust github? Because they've been hacked? But if you keep multiple clones and people keep private clones then you depend on git's use of SHA-1 Merkle hash trees for security. Or, if you want *private* repos, then you must either run your own git servers or pay a github or gitorious. Indeed. Nico -- ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On 2012-11-05 09:31:08 + (+), Ben Laurie wrote:
On Mon, Nov 5, 2012 at 5:07 AM, Nico Williams n...@cryptonector.com wrote:
It's just git, so keep multiple clone repos. You could use an
internal one as the master and push updates to the github one if
you don't trust github -- use github to serve outsiders.
That is exactly the plan.
For what it's worth, that's what we've been doing in the OpenStack
project (with some scripting to auto-close GitHub pull requests with
a message directing the contributor to our code review system). It's
been working quite well and keeps pull load off our overworked
master repository.
--
{ WHOIS( STANL3-ARIN ); WWW( http://fungi.yuggoth.org/ );
FINGER( fu...@yuggoth.org ); IRC( fu...@irc.yuggoth.org#ccl );
PGP( 43495829 ); MUD( kin...@katarsis.mudpy.org:6669 ); }
___
cryptography mailing list
cryptography@randombit.net
http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Sat, Nov 3, 2012 at 12:26 AM, James A. Donald jam...@echeque.com wrote: On Oct 30, 2012 7:50 AM, Ben Laurie b...@links.org wrote: The team has ruled out having the master at github. What is wrong with github? TBH, I wouldn't mind much, but I think the concern is that its not under our control. ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Sun, Nov 4, 2012 at 8:42 AM, Ben Laurie b...@links.org wrote: On Sat, Nov 3, 2012 at 12:26 AM, James A. Donald jam...@echeque.com wrote: On Oct 30, 2012 7:50 AM, Ben Laurie b...@links.org wrote: The team has ruled out having the master at github. What is wrong with github? TBH, I wouldn't mind much, but I think the concern is that its not under our control. It's just git, so keep multiple clone repos. You could use an internal one as the master and push updates to the github one if you don't trust github -- use github to serve outsiders. Really, what matters is that you have one master repo and all other official repos be read-only clones of it. As with any master/slave failover/takeover scheme you can always recover from the death of the master by promoting a clone to master status. So why not trust github? Because they've been hacked? But if you keep multiple clones and people keep private clones then you depend on git's use of SHA-1 Merkle hash trees for security. Or, if you want *private* repos, then you must either run your own git servers or pay a github or gitorious. Nico -- ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Fri, Oct 26, 2012 at 3:38 PM, Andy Isaacson a...@hexapodia.org wrote: On Fri, Oct 26, 2012 at 06:29:47PM +, John Case wrote: So, given what is in the stanford report and then reading this rant about openssl, I am wondering just how bad openssl is ? I've never had to implement it or code with it, so I really have no idea. How long has it been understood that it's a mess (if it is indeed a mess) ? How dangerous is it ? It looks like the rant was published in 2009 Bad is such a subjective measurement. OpenSSL is very very hard for a non-expert to code against. It's hard to figure out what interfaces you should use, what interfaces are well tested, what interfaces are known to be unsafe, and what interfaces are buggy but can be used safely with careful coding. It's fairly easy to accidentally disable security critical codepaths in the process of iterative hmm that doesn't quite work, the docs are unclear, maybe this is a bug in my code or maybe a bug in OpenSSL? that is a normal part of software development. If you need to implement anything even slightly different from what was expected by the authors. OpenSSL gets a lot of critiques for its documentation (or lack thereof), but there is [at least] one exception: the FIPS User Guide and Security Policy. Both are well written and full of technical details. Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 11:29 AM, Thierry Moreau thierry.mor...@connotech.com wrote: Solar Designer wrote: On Mon, Oct 29, 2012 at 04:06:58PM -0400, Jeffrey Walton wrote: The OpenSSL cleanse() function will likely fail on BIOs created from storage and memory mapped files when used on SSDs due to write leveling and on-controller compression. If write leveling goes away, it looks like cleanse() will still fail due to compression. Hence the need for random, non-compressible data. Not overwriting the same location may also happen due to journaling filesystems. Isn't memory-space cleanse() isolated from file system specifics except for the swap space? Is the SSD technology used for swap state in any of the OS distributions? Its a standard upgrade option from the Apple Store: 128GB Solid State Drive [Add $100.00] 256GB Solid State Drive [Add $400.00] 512GB Solid State Drive [Add $900.00] SSDs are an option for Dell and Gateway (and likely many other manufactures). I also know a few folks who have swapped out their SATA drives. All Windows machines, though. Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: I was recently reading the most dangerous code in the world article at stanford: https://crypto.stanford.edu/~dabo/pubs/abstracts/ssl-client-bugs.html and found the hackernews discussion: http://news.ycombinator.com/item?id=4695350 (interesting discussion and argument about curl library and how often it is badly deployed) And the hackernews discussion led me to OpenSSL is written by monkeys: http://www.peereboom.us/assl/assl/html/openssl.html So, given what is in the stanford report and then reading this rant about openssl, I am wondering just how bad openssl is ? I've never had to implement it or code with it, so I really have no idea. How long has it been understood that it's a mess (if it is indeed a mess) ? How dangerous is it ? If your engineering process includes an SDLC, then OpenSSL has at least two other issues: 1) It can't clean compile with nominal warnings 2) It lacks platform security integration For (1), a clean compile is often a security gate, and I don't like the choice of the Ostrich Algorithm. I wrote to NIST a while back and asked that a clean compile be added as a CMVP requirement, but nothing every came from it. The problem is that I cannot change the source files, so I can't fix the problems flagged by the compiler warning system, dynamic analysis, or static analysis. OpenSSL compiles clean with the following set of flags (under gcc): -Wall -pedantic -DPEDANTIC -Wno-long-long -Wsign-compare -Wmissing-prototypes -Wshadow -Wformat -Werror -DCRYPTO_MDEBUG_ALL -DCRYPTO_MDEBUG_ABORT -DREF_CHECK -DOPENSSL_NO_DEPRECATED you can find this in Configure, in the variable $gcc_devteam_warn. If you can make a case for more/less/different flags it should compile clean under, I'd be happy to fix it (at least in non-frozen branches). I have no idea what you mean by nominal warnings. For example, I know that OpenSSL uses uninitialized data (and *not* in the PRNG path). A bug report was filed and closed won't fix because the members of interest were initialized. I also know the library suffers conversion problems and truncation problems. Pointer? For completeness, here are the GCC switches of interest: * C Code: -Wall -Wextra -Wconversion -Wstrict–overflow -Wformat=2 -Wformat-security Ah, OK, you supply them. I don't have any argument with these, I suspect, but observe that all of them are newer than OpenSSL is. If people want issues like this fixed, they need to raise them :-) * C++ Code (includes C Code warnings): -Woverloaded-virtual -Wreorder -Wsign-promo -Wnon-virtual-dtor * Objective C Code (includes C Code warnings): -Wstrict-selector-match -Wundeclared-selector For item (2), we have some good defenses provided by the platform but they are not used - ASLR, DEP, Stack Canaries, Safe Memory functions etc. Considering how often the library handles untrusted input (the remote host's data should always be considered untrusted) and how often there are parser problems, I would expect these to be mandatory for high integrity software. For completeness, here are the switches for GCC/LD (use as available): * -fPIE/-pie (or –fPIC/-shared) * -fstack-protector-all * FORTIFY_SOURCES=2 * -z,relro, -z,now Again, I wrote to NIST and asked that the CMVP program include platform security integration since I cannot change it after validation. For what its worth, its not just OpenSSL that has these problems. Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. I'm not sure I agree some defenses are pointless. For example, attackers are very clever at building exploits such as ROP gadgets. ASLR and DEP are two of the better defenses we have in this case when a program failed its initial mission of no bugs. I'm not convinced a second line of defense is pointless. And I am aware of userland and kernel leaking addresses at times - I'm just not willing to throw the baby out with the bath water. Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 11:09 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. I'm not sure I agree some defenses are pointless. Nor would I, which is why its lucky its not what I said. For example, attackers are very clever at building exploits such as ROP gadgets. ASLR and DEP are two of the better defenses we have in this case when a program failed its initial mission of no bugs. I'm not convinced a second line of defense is pointless. And I am aware of userland and kernel leaking addresses at times - I'm just not willing to throw the baby out with the bath water. Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 11:17 AM, Peter Gutmann pgut...@cs.auckland.ac.nz wrote: Ben Laurie b...@links.org writes: Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I think that's a bit of an extreme comment on FIPS 140. For one thing it makes for a great measure of how desperate a vendor is to get onto the US government procurement gravy train, so it does have some value. How can it be a great measure of that when OpenSSL has FIPS 140? ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
Ben Laurie b...@links.org writes: On Tue, Oct 30, 2012 at 11:17 AM, Peter Gutmann pgut...@cs.auckland.ac.nz wrote: Ben Laurie b...@links.org writes: Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I think that's a bit of an extreme comment on FIPS 140. For one thing it makes for a great measure of how desperate a vendor is to get onto the US government procurement gravy train, so it does have some value. How can it be a great measure of that when OpenSSL has FIPS 140? It's a perfect measure, it shows how desperate some vendors were to sell OpenSSL (or OpenSSL-using products) to the USG. Peter. ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
So: 1. What is the process by which you get OpenSSL contributors to notice a serious issue and apply a patch? 2. What are the criteria for applying a patch? Is it just 'whatever interests the devs'? It seems that publishing an exploit works, but is that necessary? 3. It's 2012 -- why the is OpenSSL running its own ticket tracker and source control servers??? (RT is a disaster.) 4. What does it take to become an OpenSSL volunteer? Matt On Oct 30, 2012, at 10:12 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 11:58 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. AFAICS that is not rejected, it is ignored. There's a difference. Also, your patch appears to be reversed. Or your spelling is terrible :-) Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. Also not rejected. Now, I agree that having patches ignored isn't so great either, but the problem is: * RT doesn't actually work, the guy who allegedly maintains our infrastructure doesn't, and the team can't agree what to do about it (not that its tried very hard). * OpenSSL is mostly maintained by volunteers, who may not have felt particularly inspired by your patches, or may just have missed them. * When people are paid, they're generally paid to do specific things, not to trawl through RT (if they even could) looking for patches to adopt. I'm sure someone could pay for that if they want to, though. * CVS is a shit tool, too, making it hard to deal with patches - we've even agreed as a team to move off it, but see above about infrastructure :-) I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. It always amuses me when bigcorp pays to have a patch made, but somehow manages to fail to understand that the guy applying the patch has to eat, too. Plus, ISTR the IP situation is none too clear on all of these. This reminds me of the first attempt to FIPSify OpenSSL, where there was zero budget for the developer - just money for test labs and the like (what do you mean you want money to work on it? I thought it was free software!). The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. I'll grant you that your very small patches could be considered help, and it is a little unfortunate they they were ignored, but like I say, RT is a shit tool, at least as implemented at OpenSSL, as is CVS (I notice you didn't supply the needed 4 patches, just a single one) and no-one's paying anyone to pick patches up from it, particularly. The rest of your help appears to be specifying flags you'd like to be used and expecting us to do the work for you. Which I actually might, I find that kind of thing therapeutic, but you get my point. I think the project would welcome help - but it needs to be useful help :-) ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 2:21 PM, Matthew Green matthewdgr...@gmail.com wrote: So: 1. What is the process by which you get OpenSSL contributors to notice a serious issue and apply a patch? I wouldn't know, I haven't tried :-) In my case, just ask (me, that is, not some mailing list). If the issue is serious, I will likely apply the patch. 2. What are the criteria for applying a patch? Is it just 'whatever interests the devs'? It seems that publishing an exploit works, but is that necessary? I think it can be taken as read that the devs are interested in the security and stability of OpenSSL. 3. It's 2012 -- why the is OpenSSL running its own ticket tracker and source control servers??? (RT is a disaster.) Damn good question. Probably because we don't have a volunteer to move everything somewhere else and keep it running. 4. What does it take to become an OpenSSL volunteer? :-) Like most (good) open source projects: sustained contribution. Matt On Oct 30, 2012, at 10:12 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 11:58 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. AFAICS that is not rejected, it is ignored. There's a difference. Also, your patch appears to be reversed. Or your spelling is terrible :-) Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. Also not rejected. Now, I agree that having patches ignored isn't so great either, but the problem is: * RT doesn't actually work, the guy who allegedly maintains our infrastructure doesn't, and the team can't agree what to do about it (not that its tried very hard). * OpenSSL is mostly maintained by volunteers, who may not have felt particularly inspired by your patches, or may just have missed them. * When people are paid, they're generally paid to do specific things, not to trawl through RT (if they even could) looking for patches to adopt. I'm sure someone could pay for that if they want to, though. * CVS is a shit tool, too, making it hard to deal with patches - we've even agreed as a team to move off it, but see above about infrastructure :-) I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. It always amuses me when bigcorp pays to have a patch made, but somehow manages to fail to understand that the guy applying the patch has to eat, too. Plus, ISTR the IP situation is none too clear on all of these. This reminds me of the first attempt to FIPSify OpenSSL, where there was zero budget for the developer - just money for test labs and the like (what do you mean you want money to work on it? I thought it was free software!). The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. I'll grant you that your very small patches could be considered help, and it is a little unfortunate they they were ignored, but like I say, RT is a shit tool, at least as implemented at OpenSSL, as is CVS (I notice you didn't supply the needed 4 patches, just a single one) and no-one's paying anyone to pick patches up from it, particularly. The rest of your help appears to be specifying flags you'd like to be used and expecting us to do the work for you. Which I actually might, I find that kind of thing therapeutic, but you get my point. I think the project would welcome help - but it needs to be useful help :-) ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
I strongly suggest you move to git ASAP. It's not hard, though some history can be lost in the move using off-the-shelf conversion tools. (MIT Kerberos recently moved from SVN to git, and before that, from CVS to SVN, and they seem to have done a lot of manual cleanup to avoid some losses of history. You might want to talk to them if this is a problem for you, though, frankly, I think it shouldn't be, after all you can still keep CVS around for archeology...) That would be a great first step towards making contributions easier, since then patches can be posted in the form of git branches, pull requests, and formatted patches e-mailed or attached to RT. And refreshing older patches would be much easier too. Nico -- ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 2:31 PM, Nico Williams n...@cryptonector.com wrote: I strongly suggest you move to git ASAP. It's not hard, though some history can be lost in the move using off-the-shelf conversion tools. (MIT Kerberos recently moved from SVN to git, and before that, from CVS to SVN, and they seem to have done a lot of manual cleanup to avoid some losses of history. You might want to talk to them if this is a problem for you, though, frankly, I think it shouldn't be, after all you can still keep CVS around for archeology...) That would be a great first step towards making contributions easier, since then patches can be posted in the form of git branches, pull requests, and formatted patches e-mailed or attached to RT. And refreshing older patches would be much easier too. This is exactly what we've agreed to do. Well, no particular agreement around RT yet, but the git part. ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
I would be happy to volunteer to move everything to Github. But it really is really, really easy to do, and the maintenance required is minimal. That or git+redmine or git+JIRA would be my suggestion. On Tue, Oct 30, 2012 at 3:28 PM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 2:21 PM, Matthew Green matthewdgr...@gmail.com wrote: So: 1. What is the process by which you get OpenSSL contributors to notice a serious issue and apply a patch? I wouldn't know, I haven't tried :-) In my case, just ask (me, that is, not some mailing list). If the issue is serious, I will likely apply the patch. 2. What are the criteria for applying a patch? Is it just 'whatever interests the devs'? It seems that publishing an exploit works, but is that necessary? I think it can be taken as read that the devs are interested in the security and stability of OpenSSL. 3. It's 2012 -- why the is OpenSSL running its own ticket tracker and source control servers??? (RT is a disaster.) Damn good question. Probably because we don't have a volunteer to move everything somewhere else and keep it running. 4. What does it take to become an OpenSSL volunteer? :-) Like most (good) open source projects: sustained contribution. Matt On Oct 30, 2012, at 10:12 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 11:58 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. AFAICS that is not rejected, it is ignored. There's a difference. Also, your patch appears to be reversed. Or your spelling is terrible :-) Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. Also not rejected. Now, I agree that having patches ignored isn't so great either, but the problem is: * RT doesn't actually work, the guy who allegedly maintains our infrastructure doesn't, and the team can't agree what to do about it (not that its tried very hard). * OpenSSL is mostly maintained by volunteers, who may not have felt particularly inspired by your patches, or may just have missed them. * When people are paid, they're generally paid to do specific things, not to trawl through RT (if they even could) looking for patches to adopt. I'm sure someone could pay for that if they want to, though. * CVS is a shit tool, too, making it hard to deal with patches - we've even agreed as a team to move off it, but see above about infrastructure :-) I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. It always amuses me when bigcorp pays to have a patch made, but somehow manages to fail to understand that the guy applying the patch has to eat, too. Plus, ISTR the IP situation is none too clear on all of these. This reminds me of the first attempt to FIPSify OpenSSL, where there was zero budget for the developer - just money for test labs and the like (what do you mean you want money to work on it? I thought it was free software!). The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. I'll grant you that your very small patches could be considered help, and it is a little unfortunate they they were ignored, but like I say, RT is a shit tool, at least as implemented at OpenSSL, as is CVS (I notice you didn't supply the needed 4 patches, just a single one) and no-one's paying anyone to pick patches up from it, particularly. The rest of your help appears to be specifying flags you'd like to be used and expecting us to do the work for you. Which I actually might, I find that kind of thing therapeutic, but you get my point. I think the project would welcome help - but it needs to be useful help :-) ___ cryptography mailing list
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 2:39 PM, Patrick Mylund Nielsen cryptogra...@patrickmylund.com wrote: I would be happy to volunteer to move everything to Github. But it really is really, really easy to do, and the maintenance required is minimal. That or git+redmine or git+JIRA would be my suggestion. The team has ruled out having the master at github. On Tue, Oct 30, 2012 at 3:28 PM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 2:21 PM, Matthew Green matthewdgr...@gmail.com wrote: So: 1. What is the process by which you get OpenSSL contributors to notice a serious issue and apply a patch? I wouldn't know, I haven't tried :-) In my case, just ask (me, that is, not some mailing list). If the issue is serious, I will likely apply the patch. 2. What are the criteria for applying a patch? Is it just 'whatever interests the devs'? It seems that publishing an exploit works, but is that necessary? I think it can be taken as read that the devs are interested in the security and stability of OpenSSL. 3. It's 2012 -- why the is OpenSSL running its own ticket tracker and source control servers??? (RT is a disaster.) Damn good question. Probably because we don't have a volunteer to move everything somewhere else and keep it running. 4. What does it take to become an OpenSSL volunteer? :-) Like most (good) open source projects: sustained contribution. Matt On Oct 30, 2012, at 10:12 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 11:58 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. AFAICS that is not rejected, it is ignored. There's a difference. Also, your patch appears to be reversed. Or your spelling is terrible :-) Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. Also not rejected. Now, I agree that having patches ignored isn't so great either, but the problem is: * RT doesn't actually work, the guy who allegedly maintains our infrastructure doesn't, and the team can't agree what to do about it (not that its tried very hard). * OpenSSL is mostly maintained by volunteers, who may not have felt particularly inspired by your patches, or may just have missed them. * When people are paid, they're generally paid to do specific things, not to trawl through RT (if they even could) looking for patches to adopt. I'm sure someone could pay for that if they want to, though. * CVS is a shit tool, too, making it hard to deal with patches - we've even agreed as a team to move off it, but see above about infrastructure :-) I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. It always amuses me when bigcorp pays to have a patch made, but somehow manages to fail to understand that the guy applying the patch has to eat, too. Plus, ISTR the IP situation is none too clear on all of these. This reminds me of the first attempt to FIPSify OpenSSL, where there was zero budget for the developer - just money for test labs and the like (what do you mean you want money to work on it? I thought it was free software!). The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. I'll grant you that your very small patches could be considered help, and it is a little unfortunate they they were ignored, but like I say, RT is a shit tool, at least as implemented at OpenSSL, as is CVS (I notice you didn't supply the needed 4 patches, just a single one) and no-one's paying anyone to pick patches up from it, particularly. The rest of your help appears to be specifying flags you'd like to be used and expecting us to do the work for you. Which I actually might, I find that kind of thing therapeutic, but you get my point.
Re: [cryptography] Just how bad is OpenSSL ?
Thank god... On Oct 30, 2012 7:50 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 2:39 PM, Patrick Mylund Nielsen cryptogra...@patrickmylund.com wrote: I would be happy to volunteer to move everything to Github. But it really is really, really easy to do, and the maintenance required is minimal. That or git+redmine or git+JIRA would be my suggestion. The team has ruled out having the master at github. On Tue, Oct 30, 2012 at 3:28 PM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 2:21 PM, Matthew Green matthewdgr...@gmail.com wrote: So: 1. What is the process by which you get OpenSSL contributors to notice a serious issue and apply a patch? I wouldn't know, I haven't tried :-) In my case, just ask (me, that is, not some mailing list). If the issue is serious, I will likely apply the patch. 2. What are the criteria for applying a patch? Is it just 'whatever interests the devs'? It seems that publishing an exploit works, but is that necessary? I think it can be taken as read that the devs are interested in the security and stability of OpenSSL. 3. It's 2012 -- why the is OpenSSL running its own ticket tracker and source control servers??? (RT is a disaster.) Damn good question. Probably because we don't have a volunteer to move everything somewhere else and keep it running. 4. What does it take to become an OpenSSL volunteer? :-) Like most (good) open source projects: sustained contribution. Matt On Oct 30, 2012, at 10:12 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 11:58 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. AFAICS that is not rejected, it is ignored. There's a difference. Also, your patch appears to be reversed. Or your spelling is terrible :-) Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. Also not rejected. Now, I agree that having patches ignored isn't so great either, but the problem is: * RT doesn't actually work, the guy who allegedly maintains our infrastructure doesn't, and the team can't agree what to do about it (not that its tried very hard). * OpenSSL is mostly maintained by volunteers, who may not have felt particularly inspired by your patches, or may just have missed them. * When people are paid, they're generally paid to do specific things, not to trawl through RT (if they even could) looking for patches to adopt. I'm sure someone could pay for that if they want to, though. * CVS is a shit tool, too, making it hard to deal with patches - we've even agreed as a team to move off it, but see above about infrastructure :-) I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. It always amuses me when bigcorp pays to have a patch made, but somehow manages to fail to understand that the guy applying the patch has to eat, too. Plus, ISTR the IP situation is none too clear on all of these. This reminds me of the first attempt to FIPSify OpenSSL, where there was zero budget for the developer - just money for test labs and the like (what do you mean you want money to work on it? I thought it was free software!). The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. I'll grant you that your very small patches could be considered help, and it is a little unfortunate they they were ignored, but like I say, RT is a shit tool, at least as implemented at OpenSSL, as is CVS (I notice you didn't supply the needed 4 patches, just a single one) and no-one's paying anyone to pick patches up from it,
Re: [cryptography] Just how bad is OpenSSL ?
Hopefully somebody's doing some kind of integrity check pre-release no matter where it's hosted... :) In either case, happy to help if it is manhours you need, and I'm sure others on this list are as well. On Tue, Oct 30, 2012 at 3:51 PM, Aaron Grattafiori aa...@digitalinfinity.net wrote: Thank god... On Oct 30, 2012 7:50 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 2:39 PM, Patrick Mylund Nielsen cryptogra...@patrickmylund.com wrote: I would be happy to volunteer to move everything to Github. But it really is really, really easy to do, and the maintenance required is minimal. That or git+redmine or git+JIRA would be my suggestion. The team has ruled out having the master at github. On Tue, Oct 30, 2012 at 3:28 PM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 2:21 PM, Matthew Green matthewdgr...@gmail.com wrote: So: 1. What is the process by which you get OpenSSL contributors to notice a serious issue and apply a patch? I wouldn't know, I haven't tried :-) In my case, just ask (me, that is, not some mailing list). If the issue is serious, I will likely apply the patch. 2. What are the criteria for applying a patch? Is it just 'whatever interests the devs'? It seems that publishing an exploit works, but is that necessary? I think it can be taken as read that the devs are interested in the security and stability of OpenSSL. 3. It's 2012 -- why the is OpenSSL running its own ticket tracker and source control servers??? (RT is a disaster.) Damn good question. Probably because we don't have a volunteer to move everything somewhere else and keep it running. 4. What does it take to become an OpenSSL volunteer? :-) Like most (good) open source projects: sustained contribution. Matt On Oct 30, 2012, at 10:12 AM, Ben Laurie b...@links.org wrote: On Tue, Oct 30, 2012 at 11:58 AM, Jeffrey Walton noloa...@gmail.com wrote: On Tue, Oct 30, 2012 at 5:03 AM, Ben Laurie b...@links.org wrote: On Mon, Oct 29, 2012 at 10:34 PM, Jeffrey Walton noloa...@gmail.com wrote: On Fri, Oct 26, 2012 at 2:29 PM, John Case c...@sdf.org wrote: [SNIP] Apparently you think the best way to get a secure platform is to apply pressure through pointless security standards. I'd suggest your efforts might be better spent supplying patches instead. Or, y'know, talking to the authors of the s/w in question. You never know, they might care. Ah, OK. My bad. I've tried supplying patches and filing bug report/enhancement requests. Here was a gentle patch for spelling corrections in a README - rejected. http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2401. AFAICS that is not rejected, it is ignored. There's a difference. Also, your patch appears to be reversed. Or your spelling is terrible :-) Here was a patch for Xcode awareness - rejected (is it fair to say when its sites for years without acknowledgement?). http://rt.openssl.org/Ticket/Display.html?user=guestpass=guestid=2402. Also not rejected. Now, I agree that having patches ignored isn't so great either, but the problem is: * RT doesn't actually work, the guy who allegedly maintains our infrastructure doesn't, and the team can't agree what to do about it (not that its tried very hard). * OpenSSL is mostly maintained by volunteers, who may not have felt particularly inspired by your patches, or may just have missed them. * When people are paid, they're generally paid to do specific things, not to trawl through RT (if they even could) looking for patches to adopt. I'm sure someone could pay for that if they want to, though. * CVS is a shit tool, too, making it hard to deal with patches - we've even agreed as a team to move off it, but see above about infrastructure :-) I can't locate a bug report on the use of the uninitialized data. Perhaps I had the discussion on the developer's mailing list (I know I'm not imagining it, so my apologies). I am also aware that patches existed for some time for CCM mode, GCM mode, and SRP. In the case of GCM, IBM supplied the patches 5 or 10 years earlier. None were acted upon. It always amuses me when bigcorp pays to have a patch made, but somehow manages to fail to understand that the guy applying the patch has to eat, too. Plus, ISTR the IP situation is none too clear on all of these. This reminds me of the first attempt to FIPSify OpenSSL, where there was zero budget for the developer - just money for test labs and the like (what do you mean you want money to work on it? I thought it was free software!). The project does not appear to want outside help. If I am drawing the wrong conclusion, please forgive me. I'll grant you that your very small patches could
Re: [cryptography] Just how bad is OpenSSL ?
Solar Designer wrote: On Tue, Oct 30, 2012 at 11:29:17AM -0400, Thierry Moreau wrote: Isn't memory-space cleanse() isolated from file system specifics except for the swap space? Normally yes, but the swap space may be in a file (rather than a disk partition), or the swap partition may be in a virtual machine, which may reside in a file. Is the SSD technology used for swap state in any of the OS distributions? It depends on how the OS is installed. Plenty of installs have swap on SSD. Assuming that cleanse() as to deal only with L1 CPU cache, L2 CPU cache, main memory, and swap space, I considered a periodical swap space sanitation operation to be useful: add a new swap space partition, remove an existing one, sanitize the removed one (low-level, below file system), put it back into the available set of partitions. I did not experiment in practice. But that partition sanitation strategy ought to be part of an open HSM type of project. What kind of HSM is that where you expect to need swap at all? Just disable swap, unless you're using an OS that can't live without swap. I don't know. The intended HSM is Linux-based with a selected set of software components for its mission: server-side packages that would be on the closed HSM's host are candidates for the open HSM context. Then it's just a matter of the shortest route to finish: route a) secure the swap, route b) monitor software components for maximum memory usage vs physical mem plus make a memory exhaustion fault analysis. Alexander -- - Thierry Moreau CONNOTECH Experts-conseils inc. 9130 Place de Montgolfier Montreal, QC, Canada H2M 2A1 Tel. +1-514-385-5691 ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Oct 30, 2012, at 9:11 AM, Thierry Moreau thierry.mor...@connotech.com wrote: Then it's just a matter of the shortest route to finish: route a) secure the swap, route b) monitor software components for maximum memory usage vs physical mem plus make a memory exhaustion fault analysis. Errr, isn't the shortest route c) don't use swap in that system? You are not *forced* to use swap in Linux: I have plenty of Linux instances where it is not turned on. Noting that it is humorous that people are attributing this to bad OpenSSL, not bad understanding of the places where OpenSSL runs --Paul Hoffman ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Tue, Oct 30, 2012 at 12:10 PM, Paul Hoffman paul.hoff...@vpnc.org wrote: On Oct 30, 2012, at 9:11 AM, Thierry Moreau thierry.mor...@connotech.com wrote: Then it's just a matter of the shortest route to finish: route a) secure the swap, route b) monitor software components for maximum memory usage vs physical mem plus make a memory exhaustion fault analysis. Errr, isn't the shortest route c) don't use swap in that system? You are not *forced* to use swap in Linux: I have plenty of Linux instances where it is not turned on. Noting that it is humorous that people are attributing this to bad OpenSSL, not bad understanding of the places where OpenSSL runs I'm not sure anyone is blaming negative platform interactions on OpenSSL (I did not get that impression). It is what it is. A comment in the source code on occasion warning about the negative interaction would be nice though. +1 if its properly formatted, too. Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Sun, Oct 28, 2012 at 3:01 PM, Solar Designer so...@openwall.com wrote:
On Sat, Oct 27, 2012 at 06:47:05PM -0700, Patrick Pelletier wrote:
For the most part, I would say that OpenSSL is not badly written, just
badly documented. I am not a cryptography expert (just a smart,
experienced programmer, trying to use TLS) so I'm not in a particularly
good position to judge the cryptographic merits of OpenSSL. For the
most part, it seems sound, although a couple of things have given me
pause. One is OPENSSL_cleanse, which overwrites memory with random
data. Every other cryptography library I've seen seems to be happy with
just overwriting sensitive memory with zero. I haven't seen any
explanation of why OpenSSL believes that overwriting with random data is
better than overwriting with zero. In the absence of an explanation, it
feels a bit cargo-cultish.
There is explanation. CHANGES:
*) New function OPENSSL_cleanse(), which is used to cleanse a section of
memory from it's contents. This is done with a counter that will
place alternating values in each byte. This can be used to solve
two issues: 1) the removal of calls to memset() by highly optimizing
compilers, and 2) cleansing with other values than 0, since those can
be read through on certain media, for example a swap space on disk.
[Geoff Thorpe]
crypto/mem.c (two places):
/* Create a dependency on the value of 'cleanse_ctr' so our memory
* sanitisation function can't be optimised out. NB: We only do
* this for 2Kb so the overhead doesn't bother us. */
if(ret (num 2048))
{ extern unsigned char cleanse_ctr;
((unsigned char *)ret)[0] = cleanse_ctr;
}
The OPENSSL_cleanse() function is such that the memory is overwritten
with the counter values, whereas the counter is incremented in ways
dependent on the pointer to the memory region, and it is used to
pre-initialize regions being allocated by OpenSSL's CRYPTO_malloc().
This creates a non-trivial inter-dependency between the memory allocator
and the memory sanitization functions, which presumably a compiler won't
recognize as something that can be optimized out with no effect on
operation of correctly written programs.
Frankly, I do find this specific approach questionable, but the
rationale is there, and the compiler optimization problem that this
weird approach is trying to solve is real.
Out of curiosity, how would you do it? Perhaps cleanse() should go in
a separate source file compiled with no optimizations (-O0)?
Microsoft recognized the optimization problem some time ago, and
introduced SecureZeroMemory in Windows 2000. The C/C++ committee
recognizes optimizations (confer 'restrict' keyword), but doe not
offer a keyword to ensure program statements are not removed. Ditto
for Posix.
The vagueness in the definition of the 'volatile' qualifier has led to
different interpretations of its meaning (Microsoft, GCC, et al), so
assigning to a volatile global pointer will usually work, but its an
abuse of the keyword:
static volatile void* g_zeroize = NULL;
...
void wiper(void* buff, size_t len) {
memset(buff, 0x00, len);
g_zeroize = buff;
}
GCC uses volatile strictly for memory mapped hardware
(http://gcc.gnu.org/ml/gcc-help/2012-03/msg00251.html), so its an
abuse that the value is changed by software. Microsoft's
interpretation includes changes to memory by the hardware, OS, and
other threads (http://msdn.microsoft.com/en-us/library/12a04hfd(v=vs.90).aspx).
The OpenSSL cleanse() function will likely fail on BIOs created from
storage and memory mapped files when used on SSDs due to write
leveling and on-controller compression. If write leveling goes away,
it looks like cleanse() will still fail due to compression. Hence the
need for random, non-compressible data.
Jeff
___
cryptography mailing list
cryptography@randombit.net
http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
I am wondering just how bad openssl is ? While one can find various software engineer faults, I think that main issue is not that it is bad, it is that OpenSSL is written for cryptographic experts not standard software developers. The unfortunate thing is that most of the time the latter have no other choice but to use OpenSSL; in a perfect world they would be isolated from it by workflow-specific APIs that prevent them from shooting themselves in the foot too easily. Von On Oct 26, 2012, at 2:29 PM, John Case wrote: I was recently reading the most dangerous code in the world article at stanford: https://crypto.stanford.edu/~dabo/pubs/abstracts/ssl-client-bugs.html and found the hackernews discussion: http://news.ycombinator.com/item?id=4695350 (interesting discussion and argument about curl library and how often it is badly deployed) And the hackernews discussion led me to OpenSSL is written by monkeys: http://www.peereboom.us/assl/assl/html/openssl.html So, given what is in the stanford report and then reading this rant about openssl, I am wondering just how bad openssl is ? I've never had to implement it or code with it, so I really have no idea. How long has it been understood that it's a mess (if it is indeed a mess) ? How dangerous is it ? It looks like the rant was published in 2009 ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Mon, Oct 29, 2012 at 04:06:58PM -0400, Jeffrey Walton wrote: On Sun, Oct 28, 2012 at 3:01 PM, Solar Designer so...@openwall.com wrote: The OPENSSL_cleanse() function is such that the memory is overwritten with the counter values, whereas the counter is incremented in ways dependent on the pointer to the memory region, and it is used to pre-initialize regions being allocated by OpenSSL's CRYPTO_malloc(). This creates a non-trivial inter-dependency between the memory allocator and the memory sanitization functions, which presumably a compiler won't recognize as something that can be optimized out with no effect on operation of correctly written programs. Frankly, I do find this specific approach questionable, but the rationale is there, and the compiler optimization problem that this weird approach is trying to solve is real. Out of curiosity, how would you do it? The approach I am using lately is to run a quick self-test right after performing the actual crypto task using the same code. I also make it likely (to the extent possible from C source) that both calls are made from the same scope (same initial stack pointer value). As long as the crypto and the self-test are together complicated enough that the compiler does not do any of it at compile time, this works. However, this approach would not be usable by many of OpenSSL's interfaces, which may be too performance-critical, or in some cases they may be too trivial (e.g., if some function merely copies data then its self-test would be fairly likely to be optimized out). Perhaps cleanse() should go in a separate source file compiled with no optimizations (-O0)? It is already in a separate source file. I think it is compiled with the same optimization as the rest of OpenSSL now. The counter updates and the dependency on the memory allocator in the current OPENSSL_cleanse() make it likely that as long as there's just one instance of OPENSSL_cleanse() code in the entire program then neither the writes inside this function nor calls to it are optimized out. So this inter-dependency does play a useful role. (The writes can't be optimized out because there's just one instance of the code, and hopefully at least one of its uses potentially needs the writes as far as the compiler is aware. The calls can't be optimized out because they not only make the writes, but also update the counter, which is sort of made use of via the memory allocator.) However, with possible link time optimization there could be multiple instances of OPENSSL_cleanse() (e.g., some of them could be inlined), and the writes could then be optimized out of some of the instances. GCC uses volatile strictly for memory mapped hardware (http://gcc.gnu.org/ml/gcc-help/2012-03/msg00251.html), so its an abuse that the value is changed by software. offtopic This thread on gcc-help talks about two possible uses of volatile: for memory-mapped I/O (correct) and for communication between threads (wrong unless memory barriers are also used). However, these are not the only possible uses. volatile is also useful for communication within one process (one thread), typically when the main program wants to check whether one of its signal handlers has been called. The signal handler updates a variable and returns. The main program reads the volatile variable from inside a loop and becomes aware of the event. Without the volatile keyword, it would potentially be reading a cached copy of the variable (usually a register), so it would not learn of the event until much later (likely until exiting/re-entering the function), if at all. /offtopic The OpenSSL cleanse() function will likely fail on BIOs created from storage and memory mapped files when used on SSDs due to write leveling and on-controller compression. If write leveling goes away, it looks like cleanse() will still fail due to compression. Hence the need for random, non-compressible data. Not overwriting the same location may also happen due to journaling filesystems. I think OPENSSL_cleanse() is generally meant to be called shortly after the data had been written to this memory location (and made use of), before the data would likely hit external media. If we're lucky (as expected), only the overwritten data (such as the trivial pattern produced by OPENSSL_cleanse() currently) would potentially hit external media, and then it almost does not matter whether it's all zeroes or some pattern. (Well, it may matter for reasons other than confidentiality of the original data in that memory location - e.g., leaking the fact that a certain OpenSSL interface has been used at least a certain number of times.) Alexander ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
Hi Alexander, Sorry to go offlist. On Mon, Oct 29, 2012 at 7:31 PM, Solar Designer so...@openwall.com wrote: On Mon, Oct 29, 2012 at 04:06:58PM -0400, Jeffrey Walton wrote: On Sun, Oct 28, 2012 at 3:01 PM, Solar Designer so...@openwall.com wrote: [SNIP, SNIP, SNIP] GCC uses volatile strictly for memory mapped hardware (http://gcc.gnu.org/ml/gcc-help/2012-03/msg00251.html), so its an abuse that the value is changed by software. offtopic This thread on gcc-help talks about two possible uses of volatile: for memory-mapped I/O (correct) and for communication between threads (wrong unless memory barriers are also used). However, these are not the only possible uses. volatile is also useful for communication within one process (one thread), typically when the main program wants to check whether one of its signal handlers has been called. The signal handler updates a variable and returns. The main program reads the volatile variable from inside a loop and becomes aware of the event. Without the volatile keyword, it would potentially be reading a cached copy of the variable (usually a register), so it would not learn of the event until much later (likely until exiting/re-entering the function), if at all. /offtopic According to GCC's interpretation, I believe this is an abuse since volatile only applies to memory mapped hardware. The OpenSSL cleanse() function will likely fail on BIOs created from storage and memory mapped files when used on SSDs due to write leveling and on-controller compression. If write leveling goes away, it looks like cleanse() will still fail due to compression. Hence the need for random, non-compressible data. Not overwriting the same location may also happen due to journaling filesystems. This is a tough problem, and I have not figured out how to solve it short of using an encrypted drive (not available on all hardware, such as Mac Books). Risk Acceptance to the rescue! Jeff ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Sat, Oct 27, 2012 at 06:47:05PM -0700, Patrick Pelletier wrote:
For the most part, I would say that OpenSSL is not badly written, just
badly documented. I am not a cryptography expert (just a smart,
experienced programmer, trying to use TLS) so I'm not in a particularly
good position to judge the cryptographic merits of OpenSSL. For the
most part, it seems sound, although a couple of things have given me
pause. One is OPENSSL_cleanse, which overwrites memory with random
data. Every other cryptography library I've seen seems to be happy with
just overwriting sensitive memory with zero. I haven't seen any
explanation of why OpenSSL believes that overwriting with random data is
better than overwriting with zero. In the absence of an explanation, it
feels a bit cargo-cultish.
There is explanation. CHANGES:
*) New function OPENSSL_cleanse(), which is used to cleanse a section of
memory from it's contents. This is done with a counter that will
place alternating values in each byte. This can be used to solve
two issues: 1) the removal of calls to memset() by highly optimizing
compilers, and 2) cleansing with other values than 0, since those can
be read through on certain media, for example a swap space on disk.
[Geoff Thorpe]
crypto/mem.c (two places):
/* Create a dependency on the value of 'cleanse_ctr' so our memory
* sanitisation function can't be optimised out. NB: We only do
* this for 2Kb so the overhead doesn't bother us. */
if(ret (num 2048))
{ extern unsigned char cleanse_ctr;
((unsigned char *)ret)[0] = cleanse_ctr;
}
The OPENSSL_cleanse() function is such that the memory is overwritten
with the counter values, whereas the counter is incremented in ways
dependent on the pointer to the memory region, and it is used to
pre-initialize regions being allocated by OpenSSL's CRYPTO_malloc().
This creates a non-trivial inter-dependency between the memory allocator
and the memory sanitization functions, which presumably a compiler won't
recognize as something that can be optimized out with no effect on
operation of correctly written programs.
Frankly, I do find this specific approach questionable, but the
rationale is there, and the compiler optimization problem that this
weird approach is trying to solve is real.
Alexander
___
cryptography mailing list
cryptography@randombit.net
http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On 10/26/12 11:29 AM, John Case wrote: So, given what is in the stanford report and then reading this rant about openssl, I am wondering just how bad openssl is ? I've never had to implement it or code with it, so I really have no idea. I think that OpenSSL is written by monkeys is a bit sensationalist and unfair, but I do think it's fair to say that OpenSSL is user-hostile, and is written by experts for their own use, not for others to come along and use. The biggest problem with OpenSSL is that it is really, really poorly documented. There is no big picture, gentle introduction documentation that ships with OpenSSL. Most of the OpenSSL functions are documented in man pages, but some important functions are not documented. And the man pages often don't explain enough about the functions, like why would I want to use this function instead of that one, when they both sound like they do the same thing? Also, the naming of the man pages is user-hostile. The worst one is that there is a man page named rand. The other OpenSSL man pages tell me to see rand(3). But, of course, man 3 rand gets me the man page for the C standard library function, rand(). I eventually figured out I needed to do man 3ssl rand to get the man page for OpenSSL's random number functions. As far as I can tell, there is only one decent book that has been written about OpenSSL: the O'Reilly book Network Security with OpenSSL. It is ten years old now, which is not as bad as it sounds, because OpenSSL hasn't changed that much since then. Nevertheless, it obviously doesn't cover anything that's been added to OpenSSL in the past decade, and I had to modify some of the code examples because current versions of OpenSSL are better about using const in function prototypes, for instance. I also got some good laughs, like when the book talked about this brand-new algorithm called AES that OpenSSL had just recently added support for. Between reading the O'Reilly book, the man pages, the OpenSSL mailing list, random things people have written on the web, and of course, the source code (a lot!), I've been able to use OpenSSL, but it's been very slow and frustrating, and I curse the name of OpenSSL every day. Besides the poor documentation, the other thing about OpenSSL is that it is definitely not batteries included. Now, I'm not expecting it to be some high-level https library like curl. I intentionally wanted a low-level TLS library, because my job was to encapsulate a custom messaging protocol in TLS. But still, OpenSSL is lacking in some things I would hope a low-level TLS library would have. For instance, it doesn't have everything you need to validate certificates, and you need to write some of that code yourself. One of the most glaring things OpenSSL is missing out-of-the-box is thread safety. If you want OpenSSL to be threadsafe, you have to supply your own callback. This is different than the approach taken by many other libraries (for example, libevent, or GnuTLS) which supply threading implementations for pthreads and Win32, and only require callbacks if you are on a more exotic operating system. Not only is it annoying to have to supply this boilerplate code, but my bigger concern is that it makes it very tricky to use OpenSSL from more than one library in the same process. Assuming that each library wants to be threadsafe, and wants to hide its use of OpenSSL as an implementation detail (rather than telling the user you need to supply OpenSSL thread callbacks), then that means each library needs to set the OpenSSL threading callbacks to point at its own implementation. But since the OpenSSL threading callbacks are global, only one of them is going to win. Now, it's possible that everything will be okay, because whoever wins will just end up providing thread safety to everyone. But, it seems kind of sketchy and scary, and I can imagine bad things happening, like if one library is initialized (and thus sets the OpenSSL callbacks) after another library is already using OpenSSL. (One of the things I'm getting at in the previous paragraph is that OpenSSL seems to be written with the intention that it's only going to be used by the main program, which a single person or organization is going to write. OpenSSL doesn't seem to have in mind the possibility that it will be used by multiple, higher-level libraries which don't know anything about each other, but which are all used by a single program in a single address space.) So, I think that OpenSSL could be made better by two things: (1) better documentation, and (2) a small helper library that sits on top of it and provides thread-safety callbacks, and other missing helper functions like certificate validation. But, the helper library should be small, simple, and low-level enough that it would be possible to convince the high-level libraries (such as curl, libevent, etc.) to link against
Re: [cryptography] Just how bad is OpenSSL ?
On 28/10/12 12:47 PM, Patrick Pelletier wrote: Just a slow sunday morning so I thought I'd dive in on one point. For the rest, nodding. The other thing that bugged me a bit was in the infamous rand(3ssl) man page: 3. The state should be very large. If the RNG is being used to generate 4096 bit RSA keys, 2 2048 bit random strings are required (at a minimum). If your RNG state only has 128 bits, you are obviously limiting the search space to 128 bits, not 2048. I'm probably getting a little carried away on this last point but it does indicate that it may not be a bad idea to keep quite a lot of RNG state. It should be easier to break a cipher than guess the RNG seed data. This seems to contradict the advice given on Linux's random(4) man page: The amount of seed material required to generate a crypto‐ graphic key equals the effective key size of the key. For example, a 3072-bit RSA or Diffie-Hellman private key has an effective key size of 128 bits This is comparing apples to oranges, which needs to be done with care and understanding. Go to keylength.net and poke around at the algorithms used; there is no better lesson in this, and it's worth 10 minutes of play. In essence the 3072 relates to asymmetric (apples) algorithms, whereas the 128 relates to symmetric (oranges). We can approximate the strengths of these by converting the 3072 down to 128 but this is an approximation (albeit backed up by research). It is designed to allow us to more happily pair the algorithms together for a balanced result, not to make decisions of how many bits of entropy are required for each algorithm. (it requires about 2^128 oper‐ ations to break) so a key generator only needs 128 bits (16 bytes) of seed material from /dev/random. I do not believe either of those statements follow. There are plenty of algorithms out there that require X random input bits, but only deliver Y bits of security when Y is some number less than X and Y is some apples-and-oranges comparison standard. It simply doesn't follow that all algorithms are efficient, and we don't even have a definition of efficiency at this level. While some safety margin above that minimum is reasonable, as a guard against flaws in the CPRNG algorithm, no crypto‐ graphic primitive available today can hope to promise more than 256 bits of security, so if any program reads more than 256 bits (32 bytes) from the kernel random pool per invoca‐ tion, or per reasonable reseed interval (not less than one minute), that should be taken as a sign that its cryptography is not skilfully implemented. I don't believe the Linux pages are skilfully written :) iang ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
Re: [cryptography] Just how bad is OpenSSL ?
On Fri, Oct 26, 2012 at 06:29:47PM +, John Case wrote: So, given what is in the stanford report and then reading this rant about openssl, I am wondering just how bad openssl is ? I've never had to implement it or code with it, so I really have no idea. How long has it been understood that it's a mess (if it is indeed a mess) ? How dangerous is it ? It looks like the rant was published in 2009 Bad is such a subjective measurement. OpenSSL is very very hard for a non-expert to code against. It's hard to figure out what interfaces you should use, what interfaces are well tested, what interfaces are known to be unsafe, and what interfaces are buggy but can be used safely with careful coding. It's fairly easy to accidentally disable security critical codepaths in the process of iterative hmm that doesn't quite work, the docs are unclear, maybe this is a bug in my code or maybe a bug in OpenSSL? that is a normal part of software development. If you need to implement anything even slightly different from what was expected by the authors. The source code is mostly written to the OpenSSL coding standards, which are seriously different from any other coding standard I've seen (it's not Linux/KR, nor GNU, nor Microsoft, nor Sun/Oracle). Nonconformance with the coding standards in later patches is very common, so it's a mishmash of indentation standards on top of that. Naming conventions sometimes indicate that functions are strictly internal and should not be used by applications, but sometimes you have to use an internal API to get a necessary result and other times there are clearly internal APIs in the public namespace. I could go on. Overall, I would say that yes, OpenSSL is a huge mess for application developers. In that sense, it's very bad. On the other hand, it's the most thoroughly reviewed open source crypto implementation, and hasn't had very many security bugs found in the library per se. Its performance is fairly good. In that sense it's still the best option for some use cases. -andy ___ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
