Re: [openssl-dev] Work on a new RNG for OpenSSL
On 28 June 2017 at 03:41, Theodore Ts'o wrote: > On Wed, Jun 28, 2017 at 11:41:11AM +1000, Peter Waltenberg wrote: > > And FYI. On systems not backed with hardware RNG's /dev/random is > > extremely slow. 1-2 bytes/second is a DOS attack on it's own without any > > other effort required. > > Please, stop suggesting the use /dev/random. The right answer is > /dev/urandom or getrandom(2). > a) On Linux. b) If its the right answer, why is there a difference between /dev/random and /dev/urandom? > > - Ted > -- > openssl-dev mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev > -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
Cory asked: > When you say “the linked article”, do you mean the PCWorld one? My apologies I meant the one Ted referred to soon after. Pauli -- Oracle Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia -Original Message- From: Cory Benfield [mailto:c...@lukasa.co.uk] Sent: Wednesday, 28 June 2017 5:15 PM To: openssl-dev@openssl.org Subject: Re: [openssl-dev] Work on a new RNG for OpenSSL > On 28 Jun 2017, at 04:00, Paul Dale wrote: > > > Peter Waltenberg wrote: >> The next question you should be asking is: does our proposed design mitigate >> known issues ?. >> For example this: >> http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html > > Using the OS RNG won't fix the lack of boot time randomness unless there is a > HRNG present. > > For VMs, John's suggestion that /dev/hwrng should be installed is reasonable. > > For embedded devices, a HRNG is often not possible. Here getrandom() (or > /dev/random since old kernels are common) should be used. Often /dev/urandom > is used instead and the linked article is the result. There are possible > mitigations that some manufacturers include (usually with downsides). When you say “the linked article”, do you mean the PCWorld one? Because that article doesn’t provide any suggestion that /dev/urandom has anything to do with it. It is at least as likely that the SSH key is hard-coded into the machine image. The flaw here is not “using /dev/urandom”, it’s “exposing your router’s SSH access on the external side of the router”, plus the standard level of poor configuration done by shovelware router manufacturers. Cory -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
Debian also screwed up here at one point and the SSH keys for Debian installs came from a very small subset of keys. This CLASS of problem is common and it's something you need to make efforts to avoid. And again, it is something you need to address as far as you can because you simply can't rely on the users of your software to be able to do better. Seeding is a hard problem as is using the seed material correctly. The overall objective is security, security requires instance unique keys, keys that aren't trivially guessed. Quite a few of the suggestions made so far would compromise that. It's a very different problem from generating good pseudo-random sequences and by it's nature doesn't lend itself well to clean and elegant solutions. Peter From: Cory Benfield To: openssl-dev@openssl.org Date: 28/06/2017 17:15 Subject:Re: [openssl-dev] Work on a new RNG for OpenSSL Sent by:"openssl-dev" > On 28 Jun 2017, at 04:00, Paul Dale wrote: > > > Peter Waltenberg wrote: >> The next question you should be asking is: does our proposed design mitigate known issues ?. >> For example this: >> http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html > > Using the OS RNG won't fix the lack of boot time randomness unless there is a HRNG present. > > For VMs, John's suggestion that /dev/hwrng should be installed is reasonable. > > For embedded devices, a HRNG is often not possible. Here getrandom() (or /dev/random since old kernels are common) should be used. Often /dev/urandom is used instead and the linked article is the result. There are possible mitigations that some manufacturers include (usually with downsides). When you say “the linked article”, do you mean the PCWorld one? Because that article doesn’t provide any suggestion that /dev/urandom has anything to do with it. It is at least as likely that the SSH key is hard-coded into the machine image. The flaw here is not “using /dev/urandom”, it’s “exposing your router’s SSH access on the external side of the router”, plus the standard level of poor configuration done by shovelware router manufacturers. Cory -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
On 26 June 2017 at 18:10, Salz, Rich via openssl-dev < openssl-dev@openssl.org> wrote: > > Pseudorandomness of the output has been a design goal/requirement only > > in SHA-3 family. Any prior hash function’s exhibition of this property is > > coincidental. > > > > Therefore I suggest using SHA3 instead. > > Is pseudorandomness a requirement? Or is it the "50% chance of a bitflip"? > You are asking if a pseudorandom number generator requires pseudorandomness? -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
Defence in depth seems prudent: independent sources with agglomeration and whitening. As Kurt noted, [on modern OSes,] it is really unclear what sources are available to us that are not already being used by the kernel. I would turn to hardware. Since OpenSSL already has assembly-level optimization for various CPU types, accessing instructions like RDSEED and RDRAND (when available) doesn’t sound too hard. Mix their output into the seed. It can only improve the result. So, [on these same modern OSes,] what benefit do we really get from using multiple "independent" sources? They are unlikely to actually be independent if the kernel is consuming them as well and we consume the kernel. Depends on what you mean by “independent”. A completely different mechanism – probably not. A mechanism whose output bits/bytes are not (tractably) correlated? Probably yes. We shouldn't trust the user to provide entropy. Definitely. No. We shouldn’t trust the user to provide all entropy – but should welcome user’s contribution to the entropy pool. smime.p7s Description: S/MIME cryptographic signature -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
On Wed, Jun 28, 2017 at 12:01:29PM -0500, Benjamin Kaduk via openssl-dev wrote: > > I'm not sure what you mean by "draining the kernel's entropy pools". > That is, if you are adhering to the belief that taking random bits out > of a generator removes entropy from it that must be replenished, does > that not apply also to any generator/pool we write for ourselves? Or > maybe you just refer to the behavior of linux /dev/random, in which case > I would point out Ted (the author/maintainer of linux /dev/random)'s > suggestion to just use (getrandom or) /dev/random and tacit agreement > that the behavior of reducing the entropy count on reads from > /dev/random is not really needed anymore. Replace all /dev/random with /dev/urandom. > At boot time *all* pools are empty. FreeBSD has a random seed file on > disk to be loaded on next boot that helps with this (I didn't check > linux), It depends on the distro, but they should all be doing this. On systems using systemd that file is probably /var/lib/systemd/random-seed. Kurt -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
On 06/26/2017 11:28 PM, Paul Dale wrote: > Given the variety of RNGs available, would an EVP RNG interface make sense? > With a safe default in place (and no weak generators provided), the decision > can be left to the user. > A side benefit is that the unit tests could implement a simple fully > deterministic generator and the code for producing known sequences removed > from the code base. There are some benefits to this idea, as you note, but it does not seem like a clear "immediate win" to me. Maybe this is just some emotional response that has not fully absorbed "no weak generators provided", as I can't really articulate any reason to oppose it other than "randomness is so low-level that we should just provide it and not options for it". > > Defence in depth seems prudent: independent sources with agglomeration and > whitening. As Kurt noted, [on modern OSes,] it is really unclear what sources are available to us that are not already being used by the kernel. Rich had commented about the dragonfly (kernel) implementation "wow, is it really that easy?". To large extent, yes, a secure RNG can present as being that simple/easy -- if you're writing it in the kernel! The kernel has easy and direct access to lots of interrupt-driven entropy sources, any hardware generators present, etc., as well as rdrand/etc. It doesn't have to worry about fork-safety or syscall overhead, and can basically just implement the raw crypto needed for whitening/mixing/stretching. So, [on these same modern OSes,] what benefit do we really get from using multiple "independent" sources? They are unlikely to actually be independent if the kernel is consuming them as well and we consume the kernel. Now, of course OpenSSL runs on OSes that do not provide a modern kernel RNG and we will need some solution for them, which will likely look as you describe. I'm just not convinced there is much value in duplicating what the kernel is doing in the cases that the kernel does it well. > > We shouldn't trust the user to provide entropy. I've seen what is typically > provided. Uninitialised buffers aren't random. User inputs (mouse and > keyboard) likewise aren't very good. That both are still being suggested is > frustrating. I've seen worse suggestions, some to the effect that > "time(NULL) ^ getpid()" is too good and just time() is enough. Definitely. But, as we're not the kernel, finding good sources of real randomness as a generic userspace process is quite hard. > > As for specific questions and comments: > > John Denker wrote: >> If you trust the ambient OS to provide a seed, why not >> trust it for everything, and not bother to implement an >> openssl-specfic RNG at all? > I can think of a few possibilities: Ah, preemptive replies to my comments above, excellent. > * Diversifying the sources provides resistance to compromise of individual > sources. Although a full kernel compromise is unrecoverable, a kernel bug > that leaked the internal pools in a read only manner isn't unforeseeable. It is not unforseeable, sure, but so are lots of things. Spewing the contents of the openssl process-local randomness pool on the network isn't unforseeable, either; do we have any reason to think there is substantially more risk from one unknown than the other? > * Not all operating systems have good RNGs. Sure, and we need to support the ones that don't have good RNGs. But on the ones that do, what do we gain from duplicating the effort? > > * Draining the kernel's entropy pools is unfriendly behaviour, other > processes will typically want some randomness too. > > * At boot time the kernel pools are empty (low or no quality). This > compounds when several things require seeding. I'm not sure what you mean by "draining the kernel's entropy pools". That is, if you are adhering to the belief that taking random bits out of a generator removes entropy from it that must be replenished, does that not apply also to any generator/pool we write for ourselves? Or maybe you just refer to the behavior of linux /dev/random, in which case I would point out Ted (the author/maintainer of linux /dev/random)'s suggestion to just use (getrandom or) /dev/random and tacit agreement that the behavior of reducing the entropy count on reads from /dev/random is not really needed anymore. At boot time *all* pools are empty. FreeBSD has a random seed file on disk to be loaded on next boot that helps with this (I didn't check linux), and openssl has/can use ~/.rnd or similar, but those are not immune from compromise out-of-band. In order to be properly confident of good randomness, new randomness needs to be collected from the environment and added to the pool, and the kernel is in a much better position to do so (and know when it has enough!) than we are. > * Performance is also a consideration, although with a gradual collection > strategy this should be less of a concern. Except at start up. Given that we're going to be i
Re: [openssl-dev] Work on a new RNG for OpenSSL
Thanks for the quick reply. It sounds reasonable to make the default choice depending on the os environment. For me it is not a religious question what OpenSSL's default choice should be. I trust that you will find a sensible solution. And if OpenSSL supports both methods I can always make my own choice if I need to. Regards, Matthias On 28.06.2017 16:46, Matt Caswell wrote: > > On 28/06/17 15:42, Matthias St. Pierre wrote: >> Hello Matt, >> >> I am not quite sure what your current favourite solution for the upcoming >> default OpenSSL random generator is. Are you favouring >> >> - a DRBG (following SP800-90Ar1) which is using the OS RNG as entropy >> source for (re-)seeding or >> >> - simply passing all generate requests over to the OS RNG? >> >> It looks like you made two votes for the first and one vote for the second >> variant (see below). Could you please clarify your preference? > Both :-) > > i.e. both should be available as an option. > > I don't think we will necessary be able to do the latter on all > platforms that we support. > > As for which of the two is the default: where it is available - the > latter. Where it isn't the former. > > Matt -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
On 28/06/17 15:42, Matthias St. Pierre wrote: > Hello Matt, > > I am not quite sure what your current favourite solution for the upcoming > default OpenSSL random generator is. Are you favouring > > - a DRBG (following SP800-90Ar1) which is using the OS RNG as entropy source > for (re-)seeding or > > - simply passing all generate requests over to the OS RNG? > > It looks like you made two votes for the first and one vote for the second > variant (see below). Could you please clarify your preference? Both :-) i.e. both should be available as an option. I don't think we will necessary be able to do the latter on all platforms that we support. As for which of the two is the default: where it is available - the latter. Where it isn't the former. Matt > > Regards, > > Matthias St. Pierre > > > Vote 1: > > On 27.06.2017 09:28, Matt Caswell wrote: >> On 26/06/17 21:18, Kurt Roeckx wrote: “Recommendation for Random Number Generation Using Deterministic Random Bit Generators” http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf That design may look complicated, but if you think you can leave out some of the blocks in their diagram, proceed with caution. Every one of those blocks is there for a reason. >>> SP800-90A (or revision 1) can clearly be used as reference on how >>> to implement it, even if we don't use an approved algorithm from >>> it. And I really think we should look at that document when >>> implementing it. >>> >>> There should probably also be an option to use an RNG that >>> conforms to it. >> I am strongly in favour of this approach. We should be led by standards. >> > > Vote 2: (comment on PR #3789: WIP: Add DRBG random method) > > https://github.com/openssl/openssl/pull/3789#issuecomment-311494544 > > > Vote 3: > > > On 28.06.2017 11:29, Matt Caswell wrote: >> On 27/06/17 19:50, Benjamin Kaduk wrote: >>> On 06/27/2017 02:28 AM, Matt Caswell wrote: On 26/06/17 21:18, Kurt Roeckx wrote: > I think it should by default be provided by the OS, and I don't > think any OS is documenting how much randomness it can provide. > I also agree that, by default, using the OS provided source makes a lot of sense. >>> Do you mean having openssl just pass through to >>> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed >>> our own thing? >> I meant the former. >> >> Matt > -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
Hello Matt, I am not quite sure what your current favourite solution for the upcoming default OpenSSL random generator is. Are you favouring - a DRBG (following SP800-90Ar1) which is using the OS RNG as entropy source for (re-)seeding or - simply passing all generate requests over to the OS RNG? It looks like you made two votes for the first and one vote for the second variant (see below). Could you please clarify your preference? Regards, Matthias St. Pierre Vote 1: On 27.06.2017 09:28, Matt Caswell wrote: > On 26/06/17 21:18, Kurt Roeckx wrote: >>> “Recommendation for Random Number Generation Using Deterministic Random >>> Bit Generators” >>> http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf >>> >>> That design may look complicated, but if you think you can >>> leave out some of the blocks in their diagram, proceed with >>> caution. Every one of those blocks is there for a reason. >> SP800-90A (or revision 1) can clearly be used as reference on how >> to implement it, even if we don't use an approved algorithm from >> it. And I really think we should look at that document when >> implementing it. >> >> There should probably also be an option to use an RNG that >> conforms to it. > I am strongly in favour of this approach. We should be led by standards. > Vote 2: (comment on PR #3789: WIP: Add DRBG random method) https://github.com/openssl/openssl/pull/3789#issuecomment-311494544 Vote 3: On 28.06.2017 11:29, Matt Caswell wrote: > On 27/06/17 19:50, Benjamin Kaduk wrote: >> On 06/27/2017 02:28 AM, Matt Caswell wrote: >>> On 26/06/17 21:18, Kurt Roeckx wrote: >>> I think it should by default be provided by the OS, and I don't think any OS is documenting how much randomness it can provide. >>> I also agree that, by default, using the OS provided source makes a lot >>> of sense. >>> >> Do you mean having openssl just pass through to >> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed >> our own thing? > I meant the former. > > Matt -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
On Wed, Jun 28, 2017 at 08:15:20AM +0100, Cory Benfield wrote: > When you say “the linked article”, do you mean the PCWorld one? > Because that article doesn’t provide any suggestion that > /dev/urandom has anything to do with it. It is at least as likely > that the SSH key is hard-coded into the machine image. The flaw here > is not “using /dev/urandom”, it’s “exposing your router’s SSH access > on the external side of the router”, plus the standard level of poor > configuration done by shovelware router manufacturers. The PC World article was clearly caused by hard-coding the RSA private key in the flashed firmware and sharing it across fast numbers of routers. The reference which I believe Paul Dale was looking for dates back to five years ago, in the "Minding your P's and Q's paper", published in Usenix Security: https://factorable.net/ The researchers gave me early access to the paper in early July and I spent most of July 4th on the rooftop of the Museum of Science rooftop, reworking Linux's random driver during the day (so we could watch the fireworks from an advantageous position that evening). We had patches released before the P's and Q's paper was published, and these fixes are in 3.6 and newer kernels. There may still be issues with silly programs which initialize symmetric keys the first time the system boots, instead of doing it on an on-demand basis the first time someone tries connecting to the SSH port (when there would be plenty of entropy mixed in). And of course, things have gotten more challenging because systemd starts the sshd service much more quickly, and if you are on an embedded system which does not have a high resolution clock, and very little entropy when the box is first booted (for example, if the user plugs in the router without having first attached any of the LAN or WAN cables), it may still be a problem on some systems. Unfortunately, though, this ends up being a system design problem. There only so much you can do just at the kernel level, or just at the openssl level. My recommendation, again, is to use getrandom(2) if it is available, and if it isn't (on kernels older than 3.17) fall back to using /dev/urandom. On kernels older than 3.17, you're probably totally screwed because the kernel is very likely going to be missing large numbers of security fixes, so if someone is unfortunate enough to be using a kernel which is older than 3.17, they're probably totally screwed anyway. (The only exception to this rule is if you're getting monthly security updates from your cell phone provider, but if you're not using an Google-supported android releases, and you're using a ARM SOC with a 3.10 BSP kernel, you might as well give up now. And if the leader of the free world, is using such a device to tweet messages in lieu of press conferences --- again, you've got bigger problems. :-) If you are using a system which is 3.17 or newer, but your libc doesn't have support for getrandom(2), I would recommend trying to call getrandom(2) directly using the syscall(2) interface. This will mean hardcoding Linux's syscall number for different architectures, which is painful and ugly, but that's going to be the best way to block until you are sure the CRNG has been initialized, in case you do have silly programs trying to generate long-term secrets a second or so after the box is first powered on. Cheers, - Ted P.S. Short summry of kernel versions from an upstream perspective 3.17 -- first kernel version with getrandom(2) 3.18 -- Long Term Stable kernel 4.1 -- Long Term Stable kernel 4.4 -- Long Term Stable kernel 4.8 -- first kernel version using a ChaCha20 CRNG 4.9 -- Long Term Stable kernel 4.12 -- Next Long Term Stable kernel (to be released in a week or two) (Long Term Stable kernels are ones which get upstream security fixes backported by Greg K-H and his team. Just because your kernel has a LTS version may not mean you are getting the latest bug fixes. SOC vendors are notorious for not bothering to update their kernels with the latest bug fixes, and many embedded and mobile device vendors aren't bothering to keep up as well.) -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
On 27/06/17 19:50, Benjamin Kaduk wrote: > On 06/27/2017 02:28 AM, Matt Caswell wrote: >> >> On 26/06/17 21:18, Kurt Roeckx wrote: >> >>> I think it should by default be provided by the OS, and I don't >>> think any OS is documenting how much randomness it can provide. >>> >> I also agree that, by default, using the OS provided source makes a lot >> of sense. >> > > Do you mean having openssl just pass through to > getrandom()/read()-from-'/dev/random'/etc. or just using those to seed > our own thing? I meant the former. Matt -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl-dev] Work on a new RNG for OpenSSL
> On 28 Jun 2017, at 04:00, Paul Dale wrote: > > > Peter Waltenberg wrote: >> The next question you should be asking is: does our proposed design mitigate >> known issues ?. >> For example this: >> http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html > > Using the OS RNG won't fix the lack of boot time randomness unless there is a > HRNG present. > > For VMs, John's suggestion that /dev/hwrng should be installed is reasonable. > > For embedded devices, a HRNG is often not possible. Here getrandom() (or > /dev/random since old kernels are common) should be used. Often /dev/urandom > is used instead and the linked article is the result. There are possible > mitigations that some manufacturers include (usually with downsides). When you say “the linked article”, do you mean the PCWorld one? Because that article doesn’t provide any suggestion that /dev/urandom has anything to do with it. It is at least as likely that the SSH key is hard-coded into the machine image. The flaw here is not “using /dev/urandom”, it’s “exposing your router’s SSH access on the external side of the router”, plus the standard level of poor configuration done by shovelware router manufacturers. Cory -- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
