Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

>> Even opaque objects usually have some public interface. I think exposing 
>> RAND_add_ex()
>> would be a good idea for 1.1..1, and it’s likely to serve as an acceptable 
>> “live forever” API.
>
>  That’s my point.  API decisions live forever. 

Point well taken. Nonetheless…

> Suppose we move around the DRBG’s so that they are per-thread, or
> per-SSL_CTX or per-SSL object?  Will that API still work?  Or will we
> need a A “RAND_ex_ex” function? 

The *API* probably still would work. The *implementation* of it (which is 
supposed to stay under the hood) may need to change.

Offhand, I don’t see why the user-level API call would have to be different 
depending on whether the relevant DRBG is thread-local, or SSL-object-local. 
The only difficulty would be if you want to have *both*…

But my point is – this (“reseeding assistant”) is a needed capability. So it 
should be available to the users rather sooner than later…

> We don’t have even consensus on when and how to reseed.

Luckily, for the interface(s) I’m asking for it’s simple – reseed upon explicit 
request. ;-)
I understand the concerns for the reseeding in general.


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

➢ Even opaque objects usually have some public interface. I think exposing 
RAND_add_ex() would be a good idea for 1.1..1, and it’s likely to serve as an 
acceptable “live forever” API.

That’s my point.  API decisions live forever.  Suppose we move around the 
DRBG’s so that they are per-thread, or per-SSL_CTX or per-SSL object?  Will 
that API still work?  Or will we need a A “RAND_ex_ex” function?  We don’t have 
even consensus on when and how to reseed.


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Thu, Aug 24, 2017 at 05:47:55PM +, Salz, Rich via openssl-dev wrote:
> 
> >This is why I want to add things like that by default in the
> >additional data.
>   
> On reseed or generate?

Mostly on generate, but I see little point in not doing it on
reseed.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Thu, Aug 24, 2017 at 05:32:15PM +, Salz, Rich via openssl-dev wrote:
> 
> >An idea that I had was to default to reseed on fork if we know we
> >have a working syscall.

... to get entropy

>   
> Or /dev device too?

The point is that you can't be sure that /dev is still going to be
available, we need to be sure we still have a source available.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

>>I personally see no harm if these RNG objects are made available to 
>> applications that need/use them
>
> But decisions like this live forever.   It is therefore VERY important to 
> spend time thinking about what
> becomes part of the public API and what remains hidden so that we can change 
> things later when we
> have a better understanding.
 
No argument here, especially since I can’t claim to understand all the 
implications of exposing RNG (yet ;).
But it is clear that at least some applications need to work with an RNG more 
closely than others, and would want more control over its behavior. 

>   This concept is new to OpenSSL :)

:-) :-)

> But we just put out a 1.1.0 release that made things opaque, so it’s more 
> fresh in the minds of at
> least some of the dev team.

In general, making things opaque is a good idea, RNG included. But:

> Personally, since DRBG is new in 1.1.1 I would be quite happy if we didn’t 
> expose anything public
> until the release after that.

Even opaque objects usually have some public interface. I think exposing 
RAND_add_ex() would be a good idea for 1.1..1, and it’s likely to serve as an 
acceptable “live forever” API.


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

>This is why I want to add things like that by default in the
>additional data.
  
On reseed or generate?

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

>I personally see no harm if these RNG objects are made available to 
> applications that need/use them

But decisions like this live forever.   It is therefore VERY important to spend 
time thinking about what becomes part of the public API and what remains hidden 
so that we can change things later when we have a better understanding.

This concept is new to OpenSSL :)  But we just put out a 1.1.0 release that 
made things opaque, so it’s more fresh in the minds of at least some of the dev 
team.

Personally, since DRBG is new in 1.1.1 I would be quite happy if we didn’t 
expose anything public until the release after that.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

>An idea that I had was to default to reseed on fork if we know we
>have a working syscall.
  
Or /dev device too?

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

On 8/24/17, 09:45, "openssl-dev on behalf of Steffen Nurpmeso" 
 wrote:
 >> …  But I think a new API, 
 >> RAND_add_ex() that took a flag 
 >> RAND_ADD_GLOBAL, RAND_ADD_LOCAL, RAND_ADD_PRIVATE,
 >> RAND_LOCAL_PRIVATE indicating which to seed.   
 >
 > If you say it and continue like that then let me wonder why the
>  objects as such cannot be accessed like
>
> RAND_object_get(SPECIAL_CONSTANT_OR_NULL), or
> RAND_object_create(SPECIAL_CONSTANT_OR_NULL), plus
> RAND_object_bytes() and RAND_object_add().  And all the current
> could be macros or inline functions to the corresponding new
> object based functions.

My take and motivation is this:

- OpenSSL is designed to run on all kinds of architectures, CPU, and OS. Some 
have better entropy sources then others.
- The current blog essentially says that regardless of how many RNGs you 
defined – if the first one provides “enough” entropy, other RNGs won’t be asked 
to contribute. Some applications may not want to rely on one RNG for tasks they 
consider critical.
- Applications that need better entropy guarantees may want to contribute to 
the entropy pool by mixing in entropy from sources other than the default RNG. 
- NIST SP 800-90A Rev 1 understood this need and defined interface “Additional 
input” that can be applied to both reseeding and generation. Since not every 
application has such need, NIST made this interface optional. 
- OpenSSL is not developed with one specific application in mind, therefore it 
make sense to implement and provide that optional interface, allowing so 
applications to mix in randomness from whatever other sources they trust and 
have available.

I personally see no harm if these RNG objects are made available to 
applications that need/use them. But my first priority is being able to add to 
the RNG entropy pool more randomness from a source I trust.

 


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Thu, Aug 24, 2017 at 08:07:54AM +1000, Peter Waltenberg wrote:
> The bad case I'm aware of is the fork() one as it's critical that the RNG 
> state diverge on fork(). Without that you can get some very nasty 
> behaviour in things like TLS servers. Some of which have a thread pool + 
> fork() model to handle increasing load.
> 
> While ideally you'd do a complete reseed, just different state in each RNG 
> is a LOT better than nothing, and even PID + whatever else you can 
> scrounge up will help a lot. Even the high res counters available on most 
> current CPU's would help there because forking multiple processes isn't 
> quite synchronous.

This is why I want to add things like that by default in the
additional data.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Wed, Aug 23, 2017 at 05:12:56PM -0400, Paul Kehrer wrote:
> On August 19, 2017 at 2:48:19 AM, Salz, Rich via openssl-dev (
> openssl-dev@openssl.org) wrote:
> 
> 
> I think the safest thing is for us to not change the default. Programs that
> know they are going to fork can do the right/safe thing. It would be nicer
> if we could automatically always do the right thing, but I don’t think it’s
> possible.
> 
> 
> It appears the current position is that since there will be edge cases
> where a reseed would fail (thus either halting the RNG or silently not
> reseeding it) that we should not attempt to reseed? I would argue it is
> better to attempt to reseed and document that edge cases may need to reseed
> themselves. This dramatically narrows the window from "everybody needs to
> do it" to "users in certain scenarios that are becoming rarer by the day
> need to do it". Given that backwards compatibility is a concern maybe
> failure to reseed on fork should only drop an error on the child process's
> error queue though? That behavior could potentially be a separate flag that
> OpenSSL uses by default (OPENSSL_TRY_TO_INIT_ATFORK), and then
> OPENSSL_INIT_ATFORK can be more strict about reseed failures if desired.

An idea that I had was to default to reseed on fork if we know we
have a working syscall.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Steffen Nurpmeso]

"Blumenthal, Uri - 0553 - MITLL"  wrote:
 |>So I guess you want an interface that can both add things to the
 |>   "entropy" pool, and to the "additional data" pool? It shouldn't
 |>be that hard, I'll try to come up with some proposal soon.
 |  
 |I’d say the interface that  Rich Salz proposed would be good enough:
 |
 |> …  But I think a new API, RAND_add_ex() that took a flag that had \
 |> values like
 |> RAND_ADD_GLOBAL, RAND_ADD_LOCAL, RAND_ADD_PRIVATE,
 |> RAND_LOCAL_PRIVATE indicating which to seed. Thoughts?
 |
 |It exposes what’s necessary, but nothing more. Another benefit – malicious \
 |input would not compromise the entropy pool.
 |
 |> We should think carefully about what API’s we are exposing, and might \
 |> want to wait for 1.1.2
 |
 |Nothing wrong with thinking about what API to expose, and how. Since \
 |1.1.1 is what’s currently being shaped – there’s no reason to postpone \
 |that thinking. Especially since the RNG/DRBG work is being done on \
 |1.1.1 now, and this is a part of it.

If you say it and continue like that then let me wonder why the
objects as such cannot be accessed like
RAND_object_get(SPECIAL_CONSTANT_OR_NULL), or
RAND_object_create(SPECIAL_CONSTANT_OR_NULL), plus
RAND_object_bytes() and RAND_object_add().  And all the current
could be macros or inline functions to the corresponding new
object based functions.
But i am not writing a multithreaded server with the need for
performance comparisons, RAND_bytes() is good enough for us.
(And since OpenSSL is optional we still have to do some searching
around to find a PRNG anyway, with a builtin arc4 fallback.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

Okay:  https://github.com/openssl/openssl/pull/4239



-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Peter Waltenberg]

The bad case I'm aware of is the fork() one as it's critical that the RNG 
state diverge on fork(). Without that you can get some very nasty 
behaviour in things like TLS servers. Some of which have a thread pool + 
fork() model to handle increasing load.

While ideally you'd do a complete reseed, just different state in each RNG 
is a LOT better than nothing, and even PID + whatever else you can 
scrounge up will help a lot. Even the high res counters available on most 
current CPU's would help there because forking multiple processes isn't 
quite synchronous.

I don't think 'telling the user to fix it' is a particularly good option 
in these cases as in general the user will be even less capable of dealing 
with this than OpenSSL. By all means warn the users that behaviour may not 
be ideal, but do your best first.

Peter





From:   Paul Kehrer <paul.l.keh...@gmail.com>
To: "openssl-dev@openssl.org" <openssl-dev@openssl.org>
Date:   24/08/2017 07:13
Subject:    Re: [openssl-dev] Work on a new RNG for OpenSSL
Sent by:"openssl-dev" <openssl-dev-boun...@openssl.org>



On August 19, 2017 at 2:48:19 AM, Salz, Rich via openssl-dev (
openssl-dev@openssl.org) wrote:

I think the safest thing is for us to not change the default. Programs 
that know they are going to fork can do the right/safe thing. It would be 
nicer if we could automatically always do the right thing, but I don’t 
think it’s possible. 


It appears the current position is that since there will be edge cases 
where a reseed would fail (thus either halting the RNG or silently not 
reseeding it) that we should not attempt to reseed? I would argue it is 
better to attempt to reseed and document that edge cases may need to 
reseed themselves. This dramatically narrows the window from "everybody 
needs to do it" to "users in certain scenarios that are becoming rarer by 
the day need to do it". Given that backwards compatibility is a concern 
maybe failure to reseed on fork should only drop an error on the child 
process's error queue though? That behavior could potentially be a 
separate flag that OpenSSL uses by default (OPENSSL_TRY_TO_INIT_ATFORK), 
and then OPENSSL_INIT_ATFORK can be more strict about reseed failures if 
desired.

-Paul-- 
openssl-dev mailing list
To unsubscribe: 
https://urldefense.proofpoint.com/v2/url?u=https-3A__mta.openssl.org_mailman_listinfo_openssl-2Ddev=DwICAg=jf_iaSHvJObTbx-siA1ZOg=K53ZTnW2gq2IjM1tbpz7kYoHgvTfJ_aR8s4bK_o2xzY=fF3EsvxEsVa_Gqvi75hhE7fVtPzma348fsGdbb_GMLc=OhHmJAPRZokV7vWQaJy75wCYUxEWJLfAf79YJH6Mhao=
 





-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Kehrer]

On August 19, 2017 at 2:48:19 AM, Salz, Rich via openssl-dev (
openssl-dev@openssl.org) wrote:


I think the safest thing is for us to not change the default. Programs that
know they are going to fork can do the right/safe thing. It would be nicer
if we could automatically always do the right thing, but I don’t think it’s
possible.


It appears the current position is that since there will be edge cases
where a reseed would fail (thus either halting the RNG or silently not
reseeding it) that we should not attempt to reseed? I would argue it is
better to attempt to reseed and document that edge cases may need to reseed
themselves. This dramatically narrows the window from "everybody needs to
do it" to "users in certain scenarios that are becoming rarer by the day
need to do it". Given that backwards compatibility is a concern maybe
failure to reseed on fork should only drop an error on the child process's
error queue though? That behavior could potentially be a separate flag that
OpenSSL uses by default (OPENSSL_TRY_TO_INIT_ATFORK), and then
OPENSSL_INIT_ATFORK can be more strict about reseed failures if desired.

-Paul
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

>So I guess you want an interface that can both add things to the
>   "entropy" pool, and to the "additional data" pool? It shouldn't
>be that hard, I'll try to come up with some proposal soon.
  
I’d say the interface that  Rich Salz proposed would be good enough:

> …  But I think a new API, RAND_add_ex() that took a flag that had values like
> RAND_ADD_GLOBAL, RAND_ADD_LOCAL, RAND_ADD_PRIVATE,
> RAND_LOCAL_PRIVATE indicating which to seed. Thoughts?

It exposes what’s necessary, but nothing more. Another benefit – malicious 
input would not compromise the entropy pool.

> We should think carefully about what API’s we are exposing, and might want to 
> wait for 1.1.2

Nothing wrong with thinking about what API to expose, and how. Since 1.1.1 is 
what’s currently being shaped – there’s no reason to postpone that thinking. 
Especially since the RNG/DRBG work is being done on 1.1.1 now, and this is a 
part of it.


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

Forgot to add that the adversary would have to compromise not only Intel but 
also AMD CPUs. Not sure about ARM - but if it implements RDRAND then it must be 
compromised too, otherwise the enemy victory wouldn be incomplete. ;-)
And think of the chips powering mobile devices...

Regards,
Uri

Sent from my iPhone

> On Aug 21, 2017, at 20:06, Paul Dale  wrote:
> 
> Uri wrote:
>>>   It might also use things like RDRAND / RDSEED which we don't trust.
>> ...
>> From cryptography point of view, it cannot hurt, but may help a lot
> 
> There is a scenario where it does hurt: 
> https://www.lvh.io/posts/2013/10/thoughts-on-rdrand-in-linux.html
> 
> This attack wouldn't be difficult to implement given all the out of order 
> execution and look ahead that CPUs do.   It requires a compromised RDRAND 
> instruction changing the behaviour of a subsequent XOR into a copy.  Not only 
> would it not be producing random bits but it would remove any randomness from 
> the bits you already have.
> 
> 
> Pauli
> -- 
> Oracle
> Dr Paul Dale | Cryptographer | Network Security & Encryption 
> Phone +61 7 3031 7217
> Oracle Australia
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

My offhand knee-jerk reaction would be that if you have a CPU compromised to 
that extent - the battle has been lost with no reason to continue.

Going into more details, I checked that post, and disagree with the author (and 
I'm in a good company, as Linus seems to hold the same opinion). According to 
the author (as I understood him), not only RDRAND must be compromised (i.e., 
produce bad output), but the CPU would have to switch to a "special" processing 
mode when RDRAND instruction is detected. And this malicious CPU would have to 
understand how RNG is implemented in Linux, Mac, Windows, FreeBSD (at what 
point it calls RDRAND and what instructions follow)... And it's betting on 
RDRAND being the last in the chain of RNG inputs (otherwise it would wipe out 
previous entropy, but would in turn be replaced by something good from the next 
source). Also, usually mixing = XOR, but some prefer arithmetic addition - so 
RDRAND would be crafted to replace any subsequent operation to copy. Plus, it 
would have to track a bunch of the following instructions because it's common 
but not certain that the RDRAND output is mixed in immediately (in the very 
next instruction)...

To summarize, I don't have enough tin-foil for a hat of that size.

But regardless, a good solution IMHO is to offer an interface to mix in (add to 
the pool) whatever data the user wants as an additional input (in terms of SP 
800-90A that defined "additional input" to mix during reseeding and 
generation). That data could be an output of RDRAND, of an external RNG 
(including hardware TRNG), or... It would be up to the user to pick the 
additional source that he trusts. Also, nobody forces the user to employ that 
interface at all (just like on a decent OS nobody now is forced to use 
RAND_add()) - but it would be there for those who need/want it. 

Regards,
Uri

Sent from my iPhone

> On Aug 21, 2017, at 20:06, Paul Dale  wrote:
> 
> Uri wrote:
>>>  It might also use things like RDRAND / RDSEED which we don't trust.
>> ...
>> From cryptography point of view, it cannot hurt, but may help a lot
> 
> There is a scenario where it does hurt: 
> https://www.lvh.io/posts/2013/10/thoughts-on-rdrand-in-linux.html
> 
> This attack wouldn't be difficult to implement given all the out of order 
> execution and look ahead that CPUs do.   It requires a compromised RDRAND 
> instruction changing the behaviour of a subsequent XOR into a copy.  Not only 
> would it not be producing random bits but it would remove any randomness from 
> the bits you already have.
> 
> 
> Pauli
> -- 
> Oracle
> Dr Paul Dale | Cryptographer | Network Security & Encryption 
> Phone +61 7 3031 7217
> Oracle Australia
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Dale]

Uri wrote:
>>It might also use things like RDRAND / RDSEED which we don't trust.
> ...
>  From cryptography point of view, it cannot hurt, but may help a lot

There is a scenario where it does hurt: 
https://www.lvh.io/posts/2013/10/thoughts-on-rdrand-in-linux.html

This attack wouldn't be difficult to implement given all the out of order 
execution and look ahead that CPUs do.   It requires a compromised RDRAND 
instruction changing the behaviour of a subsequent XOR into a copy.  Not only 
would it not be producing random bits but it would remove any randomness from 
the bits you already have.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

We should think carefully about what API’s we are exposing, and might want to 
wait for 1.1.2

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Mon, Aug 21, 2017 at 06:12:16PM +0200, Kurt Roeckx wrote:
> So I guess you want an interface that can both add things to the
> "entropy" pool, and to the "additional data" pool? It shouldn't
> be that hard, I'll try to come up with some proposal soon.

I was thinking about adding 2 callbacks. One that is called when we
want to have entropy, the other that is called when we can use
additional data. The first would only be called by the global
DRBGs, the second by all DRBGs.

The DRBG now actually already uses callbacks to get entropy (and
a nonce), but none of that is exposed. So it would require
additional callbacks for additional data. We should probably just
add functions to set the defaults or something.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

>I at least have a plan to add additional data, but probably not in
>the current idea was probably not the way you would like to see it.

:-)  

>My idea was to query at least various sources that we don't
>attribute any entropy to, like getpid(), gettimeofday(),
>clock_gettime(), the TSC, ...

>From my point of view – adding these doesn’t add a whole lot, but it doesn’t 
>hurt. IMHO – add away. ;-)

>It might also use things like RDRAND / RDSEED which we don't trust.

Some don’t trust these, some think that they would add a good amount of 
entropy. I for one would certainly like to see the output of these mixed in. 
>From cryptography point of view, it cannot hurt, but may help a lot. Consider 
it as a lottery ticket you don’t have to pay for. ;-)

>So I guess you want an interface that can both add things to the
>"entropy" pool, and to the "additional data" pool?

That is correct. Especially because some of us have “real” nice/fancy hardware 
RNG (TRNG) available, and some like to mix in the output from RNGs on hardware 
tokens - maybe not as impressive as a “real” fancy TRNG, but as they say, every 
bit helps – in this case literally.

> It shouldn't be that hard, I'll try to come up with some proposal soon.

Thank you!!


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Mon, Aug 21, 2017 at 03:56:29PM +, Blumenthal, Uri - 0553 - MITLL wrote:
> >> P.S. I wonder if it's feasible to have a configuration parameter that 
> >> would allow me to tell the TLS code to invoke RAND_add_ex() before 
> >> generating session keys?
> >
> > Either you accept that NIST SP 90A is right, or you just bypass it 
> > completely.  We’re in the first camp.  
> 
> You mean NIST SP 800-90A, released Jan 2012 and withdrawn Jun 2015? With Rev 
> 1 *draft* currently available (released Jun 2015)?  ;-)
> 
> I’m glad you agree that “it is right”, because in our argument it supports my 
> side over yours. Let’s go through the 90A Rev 1 draft 
> http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-90Ar1.pdf:
> 
> Page 11 Section 7 provides a functional model of a DRBG (Figure 1), clearly 
> showing “additional input” for both the Reseed Function and the Generate 
> Function.  The text says “… and may include additional optional sources, 
> including … additional input.”

I at least have a plan to add additional data, but probably not in
the current idea was probably not the way you would like to see it.
My idea was to query at least various sources that we don't
attribute any entropy to, like getpid(), gettimeofday(),
clock_gettime(), the TSC, ... It might also use things like RDRAND
/ RDSEED which we don't trust.

So I guess you want an interface that can both add things to the
"entropy" pool, and to the "additional data" pool? It shouldn't
be that hard, I'll try to come up with some proposal soon.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

>> P.S. I wonder if it's feasible to have a configuration parameter that would 
>> allow me to tell the TLS code to invoke RAND_add_ex() before generating 
>> session keys?
>
> Either you accept that NIST SP 90A is right, or you just bypass it 
> completely.  We’re in the first camp.  

You mean NIST SP 800-90A, released Jan 2012 and withdrawn Jun 2015? With Rev 1 
*draft* currently available (released Jun 2015)?  ;-)

I’m glad you agree that “it is right”, because in our argument it supports my 
side over yours. Let’s go through the 90A Rev 1 draft 
http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-90Ar1.pdf:

Page 11 Section 7 provides a functional model of a DRBG (Figure 1), clearly 
showing “additional input” for both the Reseed Function and the Generate 
Function.  The text says “… and may include additional optional sources, 
including … additional input.”

Page 12 Sec 7.2: “Additional input may also be provided during reseeding and 
when pseudorandom bits are requested.”

Page 22 Sec 8.7.2: “Additional input may optionally be provided to the reseed 
and generate functions during requests. The additional input may be obtained 
from inside or outside a cryptographic module, and may include secret or public 
information. ... Additional input is optional for both the DRBG and the 
consuming application, and the ability to enter additional input may or may not 
be included in an implementation.”

Same place: “The use of additional input may be a means of providing more 
entropy for the DRBG internal state that will increase assurance that the 
entropy requirements are met. If the additional input is kept secret and has 
sufficient entropy, the input can provide more assurance when recovering from 
the compromise of the entropy input, the seed or one or more DRBG internal 
states.”

The last quote explains exactly why people (myself included) may want that 
interface. The draft does not *mandate* making this interface available, but 
blesses it nonetheless. I’m one of those who falls into the category of “need 
to increase assurance that the entropy requirements are met”.

In other words, the standard clearly does allow the implementer to *include* 
additional input. The standard also allows the implementer to “weasel out” of 
it, but I cannot imagine why a security-cautious implementer would want to. 



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

Offhand, I'd say it's a perfect solution. It allows me to mix in additional 
randomness when I want to the RNG that I think may need it. Exactly what I 
need. 

Thanks! 

P.S. I wonder if it's feasible to have a configuration parameter that would 
allow me to tell the TLS code to invoke RAND_add_ex() before generating session 
keys?

Regards,
Uri

Sent from my iPhone

> On Aug 18, 2017, at 19:42, Salz, Rich via openssl-dev 
>  wrote:
> 
> ➢ But I’d like the development team to comment on (and ideally – accept) my 
> request to add RAND_add() method to the RNG that is used in generation of 
> private keys.
> 
> Well, I’ve been thinking about this for a bit, since you first raised it.  I 
> am still not sure of the need.  And as the blog post says, we’re not 
> convinced that the current DRBG arrangement is something that will never 
> change.  But I think a new API, RAND_add_ex that took a flag that had values 
> like RAND_ADD_GLOBAL, RAND_ADD_LOCAL, RAND_ADD_PRIVATE, RAND_LOCAL_PRIVATE 
> indicating which to seed. Thoughts?
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Steffen Nurpmeso]

"Salz, Rich"  wrote:
 |Is this new RNG object available to user programs, or do they need
 |to reinvent the wheel even though they definitely link against the
 |OpenSSL library?
 |
 |You don’t have to re-invent the wheel, but you might have to modify \
 |the source ☺  Did you read the blog posting?  What wasn’t clear?

Ok.
Yes, and the linked thread on the crypto ML.
Nothing, sir.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

Is this new RNG object available to user programs, or do they need
to reinvent the wheel even though they definitely link against the
OpenSSL library?


You don’t have to re-invent the wheel, but you might have to modify the source 
☺  Did you read the blog posting?  What wasn’t clear?

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Steffen Nurpmeso]

"Salz, Rich via openssl-dev"  wrote:
 |➢ But I’d like the development team to comment on (and ideally – accept) \
 |my request to add RAND_add() method to the RNG that is used in generation \
 |of private keys.
 |
 |Well, I’ve been thinking about this for a bit, since you first raised \
 |it.  I am still not sure of the need.  And as the blog post says, we’re \
 |not convinced that the current DRBG arrangement is something that will \
 |never change.  But I think a new API, RAND_add_ex that took a flag \
 |that had values like RAND_ADD_GLOBAL, RAND_ADD_LOCAL, RAND_ADD_PRIVATE, \
 |RAND_LOCAL_PRIVATE indicating which to seed. Thoughts?

Is this new RNG object available to user programs, or do they need
to reinvent the wheel even though they definitely link against the
OpenSSL library?

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

➢ But I’d like the development team to comment on (and ideally – accept) my 
request to add RAND_add() method to the RNG that is used in generation of 
private keys.

Well, I’ve been thinking about this for a bit, since you first raised it.  I am 
still not sure of the need.  And as the blog post says, we’re not convinced 
that the current DRBG arrangement is something that will never change.  But I 
think a new API, RAND_add_ex that took a flag that had values like 
RAND_ADD_GLOBAL, RAND_ADD_LOCAL, RAND_ADD_PRIVATE, RAND_LOCAL_PRIVATE 
indicating which to seed. Thoughts?

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

 
The problem with /dev/urandom will go away sooner or later. All major
platforms either have a CPRNG syscall for years or introduced one
recently. BSD has getentropy(2) for a while, Windows has
CryptGenRandom() and Linux has getrandom(2) since Kernel 3.2 and glibc 2.15.


Agreed.

And when we can make –with-rand-seed=syscall the default, then it will be  a 
happier place ☺

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

It seems to me this all depends on the order of things you do to
create a daemon. You could make sure the RNG is inited, chroot,
and then fork for instance. And I suspect there are actually
programs that do it in that order.


Yes.

I think the safest thing is for us to not change the default.  Programs that 
know they are going to fork can do the right/safe thing.  It would be nicer if 
we could automatically always do the right thing, but I don’t think it’s 
possible.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

All the items discussed are important.

But I’d like the development team to comment on (and ideally – accept) my 
request to add RAND_add() method to the RNG that is used in generation of 
private keys.
Reason/justification: to be able to improve the available randomness by mixing 
in output from hardware-based TRNG(s).
--
Regards,
Uri Blumenthal



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Fri, Aug 18, 2017 at 09:22:48AM +0200, Tomas Mraz wrote:
> On Thu, 2017-08-17 at 22:11 +0200, Kurt Roeckx wrote:
> > On Thu, Aug 17, 2017 at 02:34:49PM +0200, Tomas Mraz wrote:
> > > On Thu, 2017-08-17 at 12:22 +, Salz, Rich via openssl-dev
> > > wrote:
> > > > I understand the concern.  The issue I am wrestling with is
> > > > strict
> > > > compatibility with the existing code.  Does anyone really *want*
> > > > the
> > > > RNG’s to not reseed on fork?  It’s hard to imagine, but maybe
> > > > somewhere someone is.  And then it’s not about just reseeding,
> > > > but
> > > > what about when (if) we add other things, like whether or not the
> > > > secure arena gets zero’d in a child?
> > > > 
> > > > So let me phrase it this way:  does anyone object to changing the
> > > > default so NO_ATFORK must be used to avoid the reseeding and
> > > > other
> > > > things we might add later?
> > > 
> > > I can hardly see anyone would be broken if the default is to reseed
> > > RNG on fork. However that might not be true for other atfork
> > > functionalities so perhaps there is a need to make each of these
> > > future
> > > atfork functions configurable and either on or off by default
> > > individually and not as a whole.
> > 
> > There might be cases where after fork you're not able to get to
> > /dev/urandom anymore.
> 
> I do not think so. Which particular cases do you have on mind? Yes,
> after fork+exec you could for example switch SELinux domain and won't
> be able to access something but immediately after fork it should not be
> so. Also perhaps the reseeding after fork can be made less strict in
> regards to failures reading /dev/urandom or so.

It seems to me this all depends on the order of things you do to
create a daemon. You could make sure the RNG is inited, chroot,
and then fork for instance. And I suspect there are actually
programs that do it in that order.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Tomas Mraz]

On Thu, 2017-08-17 at 22:11 +0200, Kurt Roeckx wrote:
> On Thu, Aug 17, 2017 at 02:34:49PM +0200, Tomas Mraz wrote:
> > On Thu, 2017-08-17 at 12:22 +, Salz, Rich via openssl-dev
> > wrote:
> > > I understand the concern.  The issue I am wrestling with is
> > > strict
> > > compatibility with the existing code.  Does anyone really *want*
> > > the
> > > RNG’s to not reseed on fork?  It’s hard to imagine, but maybe
> > > somewhere someone is.  And then it’s not about just reseeding,
> > > but
> > > what about when (if) we add other things, like whether or not the
> > > secure arena gets zero’d in a child?
> > > 
> > > So let me phrase it this way:  does anyone object to changing the
> > > default so NO_ATFORK must be used to avoid the reseeding and
> > > other
> > > things we might add later?
> > 
> > I can hardly see anyone would be broken if the default is to reseed
> > RNG on fork. However that might not be true for other atfork
> > functionalities so perhaps there is a need to make each of these
> > future
> > atfork functions configurable and either on or off by default
> > individually and not as a whole.
> 
> There might be cases where after fork you're not able to get to
> /dev/urandom anymore.

I do not think so. Which particular cases do you have on mind? Yes,
after fork+exec you could for example switch SELinux domain and won't
be able to access something but immediately after fork it should not be
so. Also perhaps the reseeding after fork can be made less strict in
regards to failures reading /dev/urandom or so.

-- 
Tomáš Mráz
Red Hat

No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]

 * Google and NSA associates, this message is none of your business.
 * Please leave it alone, and consider whether your actions are
 * authorized by the contract with Red Hat, or by the US constitution.
 * If you feel you're being encouraged to disregard the limits built
 * into them, remember Edward Snowden and Wikileaks.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Thu, Aug 17, 2017 at 02:34:49PM +0200, Tomas Mraz wrote:
> On Thu, 2017-08-17 at 12:22 +, Salz, Rich via openssl-dev wrote:
> > I understand the concern.  The issue I am wrestling with is strict
> > compatibility with the existing code.  Does anyone really *want* the
> > RNG’s to not reseed on fork?  It’s hard to imagine, but maybe
> > somewhere someone is.  And then it’s not about just reseeding, but
> > what about when (if) we add other things, like whether or not the
> > secure arena gets zero’d in a child?
> > 
> > So let me phrase it this way:  does anyone object to changing the
> > default so NO_ATFORK must be used to avoid the reseeding and other
> > things we might add later?
> 
> I can hardly see anyone would be broken if the default is to reseed
> RNG on fork. However that might not be true for other atfork
> functionalities so perhaps there is a need to make each of these future
> atfork functions configurable and either on or off by default
> individually and not as a whole.

There might be cases where after fork you're not able to get to
/dev/urandom anymore.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

I have only one issue with the current design: the apparent absence of 
RAND_add() interface for the "private" RNG.

I request that it is added (no pun intended, really :).

Regards,
Uri

Sent from my iPhone

> On Aug 17, 2017, at 09:18, Salz, Rich via openssl-dev 
>  wrote:
> 
> \
>> somewhere someone is.  And then it’s not about just reseeding, but
>> what about when (if) we add other things, like whether or not the
>> secure arena gets zero’d in a child?
> 
>It's difficult to think of what circumstances this might break existing
>code? What scenario did you have in mind?
> 
> As excerpted above in my post.
> 
> -- 
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

\
> somewhere someone is.  And then it’s not about just reseeding, but
> what about when (if) we add other things, like whether or not the
> secure arena gets zero’d in a child?

It's difficult to think of what circumstances this might break existing
code? What scenario did you have in mind?

As excerpted above in my post.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Tomas Mraz]

On Thu, 2017-08-17 at 12:22 +, Salz, Rich via openssl-dev wrote:
> I understand the concern.  The issue I am wrestling with is strict
> compatibility with the existing code.  Does anyone really *want* the
> RNG’s to not reseed on fork?  It’s hard to imagine, but maybe
> somewhere someone is.  And then it’s not about just reseeding, but
> what about when (if) we add other things, like whether or not the
> secure arena gets zero’d in a child?
> 
> So let me phrase it this way:  does anyone object to changing the
> default so NO_ATFORK must be used to avoid the reseeding and other
> things we might add later?

I can hardly see anyone would be broken if the default is to reseed
RNG on fork. However that might not be true for other atfork
functionalities so perhaps there is a need to make each of these future
atfork functions configurable and either on or off by default
individually and not as a whole.


> By the way I noticed that openssl_init_fork_handlers() is not
> guarded by
> RUN_ONCE(). This should be fixed, too.
> 
> Yeah, I’ll fix that; thanks.
> 
-- 
Tomáš Mráz
Red Hat

No matter how far down the wrong road you've gone, turn back.
  Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]

 * Google and NSA associates, this message is none of your business.
 * Please leave it alone, and consider whether your actions are
 * authorized by the contract with Red Hat, or by the US constitution.
 * If you feel you're being encouraged to disregard the limits built
 * into them, remember Edward Snowden and Wikileaks.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matt Caswell]

On 17/08/17 13:22, Salz, Rich via openssl-dev wrote:
> I understand the concern.  The issue I am wrestling with is strict
> compatibility with the existing code.  Does anyone really *want* the
> RNG’s to not reseed on fork?  It’s hard to imagine, but maybe
> somewhere someone is.  And then it’s not about just reseeding, but
> what about when (if) we add other things, like whether or not the
> secure arena gets zero’d in a child?
> 
> So let me phrase it this way:  does anyone object to changing the
> default so NO_ATFORK must be used to avoid the reseeding and other
> things we might add later?

It's difficult to think of what circumstances this might break existing
code? What scenario did you have in mind? Even if it does break
something obscure, I think this is a case where security-by-default
takes precedence.

Matt
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Kehrer]

Great news and congratulations to everyone on landing this work.

I see that the RNG is now capable of automatically reseeding itself on
fork, which will be a huge win for applications that aren't rigorous
about doing so themselves (read: most of them). However, it appears
that OPENSSL_INIT_ATFORK is not set as an option when OpenSSL calls
OPENSSL_init_crypto. Would it be possible to make this default? This
would be a large improvement in terms of protecting applications
linking against OpenSSL.

-Paul Kehrer (reaperhulk)

On Mon, Aug 14, 2017 at 10:45 AM, Salz, Rich via openssl-dev
 wrote:
> Thanks everyone for the discussion (mainly in June) about this.  There’s a
> blog post describing what we’ve done for the 1.1.1 release:
> https://www.openssl.org/blog/blog/2017/08/12/random/
>
>
>
>
> --
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

And this is a very good answer. Perhaps this guidance deserves being documented 
somewhere besides this mailing list? Something along the lines of 
It is documented in the RAND_add manpage.

➢ I’m not sure I agree here. What effort are you talking about? In order to 
select an order in which available sources are queried, the developers had to 
think (hopefully :). Those thought could be documented in a few lines of text. 

And what would be the point?  Why should someone trust the documentation, which 
can get out of date, more than the source?

➢ So while the team clearly has the right to make changes (especially 
before the interface became public ;), but I’d rather that such changes  are 
guided by an informed consent from the public (such as yours truly ;). 

We aren’t cutting off any avenues of participation.  Email discussion and pull 
requests are always welcome.  But yes, the barrier to useful participation is 
that someone has to first read and understand the source.

➢ I think it is *imperative* for a user to be able to RAND_add() to the DRBG 
that gnerates private keys. I cannot emphasize enough how critical this is.

I am curious to know your justification for this.  It seems to me that if 
you accept the DRBG document, which we do, then the way we do the seeding is 
fine.  If you don’t accept the document, then modify the source.


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

   >>> Modify the source :)
>>   
>>Very bad answer. 
   >
   >And also a wrong one.  Your application can always call RAND_add().  
Sorry for mistake.
 
And this is a very good answer. Perhaps this guidance deserves being documented 
somewhere besides this mailing list? Something along the lines of 

“RNG Seed Sources can be set by --with-rand-seed. “os” is the default source. 
Sources are tried until enough bits of randomness have been collected. If you 
want to mix data from a particular source into the seed, but don’t want to make 
that source exclusive – use RAND_add() method.”
   
 > This is a mostly volunteer open source project. 

Yeah, I realize and appreciate that.

> We are unlikely to commit to something that requires so much effort

I’m not sure I agree here. What effort are you talking about? In order to 
select an order in which available sources are queried, the developers had to 
think (hopefully :). Those thought could be documented in a few lines of text. 

> when, frankly, most of the consumers aren’t interested, or qualified, to make 
> an assessment.

So they’ll be happy with the default. Fine with me. ;-)

>  I am sorry if that sounds obnoxious or conceited.  It shouldn’t; there are 
> many things that I know I’m not qualified to comment on :)  And also, we 
> reserve the right to make changes.

No offense taken. But you “freeze” interface to and behavior of ciphers and 
cipher modes, for example. This (how you seed RNG that provides keys to those) 
is at least equally important. It’s not a minute detail that nobody should care 
about or nose in.

So while the team clearly has the right to make changes (especially before the 
interface became public ;), but I’d rather that such changes  are guided by an 
informed consent from the public (such as yours truly ;). 

 >   I expect that the FIPS project, just starting, will be of interest to you. 
   
Thank you – indeed it is of  interest. (Though I see FIPS more as a curse than 
as a blessing ;-).
 
One important thing I missed earlier:

>  We also added a separate global DRBG for private key generation and added 
> API’s to use it.
> This object isn’t reachable directly, but it is used by the new BN_priv_rand 
> and BN_priv_rand_range API’s.
> Those API’s, in turn, are used by all private-key generating functions.

I think it is *imperative* for a user to be able to RAND_add() to the DRBG that 
gnerates private keys. I cannot emphasize enough how critical this is.



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

>>> 3. What should I do if I want a given source to be used in addition to the 
>>> other sources, regardless of whether openssl thinks it got “enough bits” of 
>>> randomness or not?

>> Modify the source :)

>Very bad answer. 

And also a wrong one.  Your application can always call RAND_add().  Sorry for 
mistake.

> I have no problem reading the source code. I do have a problem with (a) 
> important decisions like this not “formalized” and documented, and (b) 
> mechanisms to tune the RNG seeding not provided and clearly and 
> comprehensively documented.
   
This is a mostly volunteer open source project.  We are unlikely to commit to 
something that requires so much effort when, frankly, most of the consumers 
aren’t interested, or qualified, to make an assessment.  I am sorry if that 
sounds obnoxious or conceited.  It shouldn’t; there are many things that I know 
I’m not qualified to comment on :)  And also, we reserve the right to make 
changes.

I expect that the FIPS project, just starting, will be of interest to you. 

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

>> 1. What’s the default if “with-rand-seed” was not provided? All of the 
>> listed supported types? None of them? Some of them…?
>  
>   As the first bullet says, it’s “os”.   

OK, thanks.

> As for the second part of your question, it is deliberately not answered.   
> If you care, you’ll have to read the source.  (It’s clean and easy to do so, 
> now.)  We’re not documenting everything.

I think it’s a bad approach to not document this important behavior. It has to 
be security-analyzed, then frozen & published.

>>2. What is the order in which the seed sources are tried (both when 
>>“with-random-seed” was and was not given)? 
>
> Read the source.

Likewise. It has to be published, and the developers need to figure out the 
right way here and commit to it (no pun intended).

>> 3. What should I do if I want a given source to be used in addition to the 
>> other sources, regardless of whether openssl thinks it got “enough bits” of 
>> randomness or not?
>
> Modify the source :)

Very bad answer. 

When randomness is involved, adding more of diverse sources can only improve 
the outcome. Therefore there must be a way to tell OpenSSL to *not* stop when 
it got what it believe to be “enough” but mix in more from other sources. And 
that mechanism must *not* be an individual hack – but a standard reviewed 
maintained access method.

> For a few reasons, we’re deliberately not documenting all the details.  
> Interested parties will have to read the source.

I have no problem reading the source code. I do have a problem with (a) 
important decisions like this not “formalized” and documented, and (b) 
mechanisms to tune the RNG seeding not provided and clearly and comprehensively 
documented.

I urge the developers to reconsider.



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> 1. What’s the default if “with-rand-seed” was not provided? All of the listed 
> supported types? None of them? Some of them…?

As the first bullet says, it’s “os”.   As for the second part of your question, 
it is deliberately not answered.   If you care, you’ll have to read the source. 
 (It’s clean and easy to do so, now.)  We’re not documenting everything.

>2. What is the order in which the seed sources are tried (both when 
>“with-random-seed” was and was not given)? 

Read the source.

> 3. What should I do if I want a given source to be used in addition to the 
> other sources, regardless of whether openssl thinks it got “enough bits” of 
> randomness or not?

Modify the source :)

For a few reasons, we’re deliberately not documenting all the details.  
Interested parties will have to read the source.



-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

Thanks everyone for the discussion (mainly in June) about this.  There’s a blog 
post describing what we’ve done for the 1.1.1 release: 
https://www.openssl.org/blog/blog/2017/08/12/random/

 

Nice. But some important things could be made clearer.

 

We added a new configuration parameter, --with-rand-seed, which takes a 
comma-separated list of values for seed sources. Each method is tried in turn, 
stopping when enough bits of randomness have been collected.

 
What’s the default if “with-rand-seed” was not provided? All of the listed 
supported types? None of them? Some of them…?
What is the order in which the seed sources are tried (both when 
“with-random-seed” was and was not given)? 
What should I do if I want a given source to be used in addition to the other 
sources, regardless of whether openssl thinks it got “enough bits” of 
randomness or not?



smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

Thanks everyone for the discussion (mainly in June) about this.  There’s a blog 
post describing what we’ve done for the 1.1.1 release: 
https://www.openssl.org/blog/blog/2017/08/12/random/

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[John Denker via openssl-dev]

Executive summary:

As has been said many times before, what we need (but do not have)
is /one/ source of randomness that never blocks and never returns
bits that are guessable by the adversary.

In favorable cases, using getrandom(,,0) \*\ is appropriate
for openssl.  There are problems with that, but switching to
getrandom(,,GRND_RANDOM) \**\ would not solve the problems.

\*\  Reading /dev/urandom is almost the same.
\**\ Reading /dev/random is essentially the same.

In cases where getrandom() is not good enough, the problems tend
to be highly platform-dependent.  Many of these problems would be
quite difficult for openssl to detect (much less solve).  Some
platforms are not secure and cannot be made secure.

On 06/27/2017 01:40 PM, Theodore Ts'o wrote:

>> My recommendation for Linux is to use getrandom(2) the flags field set
>> to zero.
>> [...] /dev/urandom (which has the same performance characteristics as the
>> getrandom system call)


Similarly, on 06/29/2017 04:03 AM, Dimitry Andric gave what might
be considered the usually-correct answer for the wrong reasons:

> In short, almost everybody should use /dev/urandom

OK.  There's also getrandom().

> and /dev/random is kept alive for old programs.

[...]

> The Linux random(4) manpage says:
> 
>The /dev/random device is a legacy interface which  dates  back
>to a time where the cryptographic primitives used in the imple‐
>mentation of /dev/urandom were not  widely  trusted.   It  will
>return random bytes only within the estimated number of bits of
>fresh  noise  in  the  entropy  pool,  blocking  if  necessary.
>/dev/random is suitable for applications that need high quality
>randomness, and can afford indeterminate delays.

That's what the manpage says ... but does anybody believe it?

On 06/27/2017 06:22 PM, Ted told us not to trust what it says in the man
pages.

Oddly enough, all the advice given above (including the list traffic
and the man pages) is flatly contradicted by what it says in the most
up-to-date kernel source, namely:

>>>  /dev/random is suitable for use when very high
>>>  quality randomness is desired (for example, for key generation

Reference:
  
https://git.kernel.org/pub/scm/linux/kernel/git/tytso/random.git/tree/drivers/char/random.c?id=e2682130931f#n111

All it all, it's hardly surprising that users are confused.

==

When it was introduced, the random / urandom split was advertised as
a way of solving certain problems with the old approach.  To block or
not to block, that is the question.  The problems didn't actually
get solved, just shifted.  The split requires users (rather than the
RNG designers) to deal with the problems.  The fact that the recently-
introduced getrandom(2) call has flags such as GRND_RANDOM and
GRND_NONBLOCK means that users are still on the hook for problems
they almost certainly cannot understand, much less solve.

The conclusion remains the same:  What we need (but do not have) is
/one/ source of randomness that never blocks and never returns bits
that are guessable by the adversary.

==

In fact there are profound distinctions between an ideal HRNG and
an ideal PRNG.  AFAICT neither one exists in the real world, in the
same sense that ideal spheres and planes do not exist, but still
the idealizations are meaningful and helpful.

It seems likely that /dev/random was intended, at the time of the
split, to serve as approximate HRNG, while /dev/urandom was intended
to be a PRNG of some kind.  Using terms like "legacy interface" is
an astonishing mischaracterization of the distinction.



Similarly, it is strange to talk about

> a time where the cryptographic primitives used in the imple‐
>mentation of /dev/urandom were not  widely  trusted

In fact, 
 ++ Improper seeding is, and has always been, the #1 threat to
  both /dev/random and /dev/urandom.
 ++ Compromise of the internal state is a threat to /dev/urandom.
  It is better to prevent this than to try to cure it.
  If the PRNG is compromised, probably a lot of other things are too.
 ++ Lousy architectural design is always a threat.
 ++ Coding errors are always a threat.
 ++ etc.
 ++ etc.
 -- Cryptanalytic attack against the outputs is way, Way, WAY
  down on the list, and always has been, assuming the crypto
  primitives are halfway decent, assuming the architecture and
  implementation are sound.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Dimitry Andric]

On 29 Jun 2017, at 06:03, Ben Laurie  wrote:
> 
> On 28 June 2017 at 03:41, Theodore Ts'o  wrote:
> On Wed, Jun 28, 2017 at 11:41:11AM +1000, Peter Waltenberg wrote:
> > And FYI. On systems not backed with hardware RNG's /dev/random is
> > extremely slow. 1-2 bytes/second is a DOS attack on it's own without any
> > other effort required.
> 
> Please, stop suggesting the use /dev/random.  The right answer is
> /dev/urandom or getrandom(2).
> 
> a) On Linux.
> 
> b) If its the right answer, why is there a difference between /dev/random and 
> /dev/urandom?

The Linux random(4) manpage says:

   The /dev/random device is a legacy interface which  dates  back
   to a time where the cryptographic primitives used in the imple‐
   mentation of /dev/urandom were not  widely  trusted.   It  will
   return random bytes only within the estimated number of bits of
   fresh  noise  in  the  entropy  pool,  blocking  if  necessary.
   /dev/random is suitable for applications that need high quality
   randomness, and can afford indeterminate delays.

and:

   Usage
   The /dev/random interface is considered a legacy interface, and
   /dev/urandom is preferred and sufficient in all use cases, with
   the exception of applications which require  randomness  during
   early  boot  time; for these applications, getrandom(2) must be
   used instead, because it will block until the entropy  pool  is
   initialized.

In short, almost everybody should use /dev/urandom, and /dev/random is
kept alive for old programs.

-Dimitry



signature.asc
Description: Message signed with OpenPGP
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Ben Laurie]

On 28 June 2017 at 03:41, Theodore Ts'o  wrote:

> On Wed, Jun 28, 2017 at 11:41:11AM +1000, Peter Waltenberg wrote:
> > And FYI. On systems not backed with hardware RNG's /dev/random is
> > extremely slow. 1-2 bytes/second is a DOS attack on it's own without any
> > other effort required.
>
> Please, stop suggesting the use /dev/random.  The right answer is
> /dev/urandom or getrandom(2).
>

a) On Linux.

b) If its the right answer, why is there a difference between /dev/random
and /dev/urandom?


>
> - Ted
> --
> openssl-dev mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
>
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Dale]

Cory asked:
> When you say “the linked article”, do you mean the PCWorld one?

My apologies I meant the one Ted referred to soon after.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia


-Original Message-
From: Cory Benfield [mailto:c...@lukasa.co.uk] 
Sent: Wednesday, 28 June 2017 5:15 PM
To: openssl-dev@openssl.org
Subject: Re: [openssl-dev] Work on a new RNG for OpenSSL


> On 28 Jun 2017, at 04:00, Paul Dale <paul.d...@oracle.com> wrote:
> 
> 
> Peter Waltenberg wrote:
>> The next question you should be asking is: does our proposed design mitigate 
>> known issues ?. 
>> For example this:
>> http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html
> 
> Using the OS RNG won't fix the lack of boot time randomness unless there is a 
> HRNG present.
> 
> For VMs, John's suggestion that /dev/hwrng should be installed is reasonable.
> 
> For embedded devices, a HRNG is often not possible.  Here getrandom() (or 
> /dev/random since old kernels are common) should be used.  Often /dev/urandom 
> is used instead and the linked article is the result.  There are possible 
> mitigations that some manufacturers include (usually with downsides).

When you say “the linked article”, do you mean the PCWorld one? Because that 
article doesn’t provide any suggestion that /dev/urandom has anything to do 
with it. It is at least as likely that the SSH key is hard-coded into the 
machine image. The flaw here is not “using /dev/urandom”, it’s “exposing your 
router’s SSH access on the external side of the router”, plus the standard 
level of poor configuration done by shovelware router manufacturers.

Cory

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Peter Waltenberg]

Debian also screwed up here at one point and the SSH keys for Debian 
installs came from a very small subset of keys. This CLASS of problem is 
common and it's something you need to make efforts to avoid. And again, it 
is something you need to address as far as you can because you simply 
can't rely on the users of your software to be able to do better.

Seeding is a hard problem as is using the seed material correctly.

The overall objective is security, security requires instance unique keys, 
keys that aren't trivially guessed. Quite a few of the suggestions made so 
far would compromise that. It's a very different problem from generating 
good pseudo-random sequences and by it's nature doesn't lend itself well 
to clean and elegant solutions. 

Peter 






From:   Cory Benfield <c...@lukasa.co.uk>
To: openssl-dev@openssl.org
Date:   28/06/2017 17:15
Subject:        Re: [openssl-dev] Work on a new RNG for OpenSSL
Sent by:"openssl-dev" <openssl-dev-boun...@openssl.org>




> On 28 Jun 2017, at 04:00, Paul Dale <paul.d...@oracle.com> wrote:
> 
> 
> Peter Waltenberg wrote:
>> The next question you should be asking is: does our proposed design 
mitigate known issues ?. 
>> For example this:
>> 
http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html

> 
> Using the OS RNG won't fix the lack of boot time randomness unless there 
is a HRNG present.
> 
> For VMs, John's suggestion that /dev/hwrng should be installed is 
reasonable.
> 
> For embedded devices, a HRNG is often not possible.  Here getrandom() 
(or /dev/random since old kernels are common) should be used.  Often 
/dev/urandom is used instead and the linked article is the result.  There 
are possible mitigations that some manufacturers include (usually with 
downsides).

When you say “the linked article”, do you mean the PCWorld one? Because 
that article doesn’t provide any suggestion that /dev/urandom has anything 
to do with it. It is at least as likely that the SSH key is hard-coded 
into the machine image. The flaw here is not “using /dev/urandom”, it’s 
“exposing your router’s SSH access on the external side of the router”, 
plus the standard level of poor configuration done by shovelware router 
manufacturers.

Cory

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev




-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Ben Laurie]

On 26 June 2017 at 18:10, Salz, Rich via openssl-dev <
openssl-dev@openssl.org> wrote:

> > Pseudorandomness of the output has been a design goal/requirement only
> > in SHA-3 family. Any prior hash function’s exhibition of this property is
> > coincidental.
> >
> > Therefore I suggest using SHA3 instead.
>
> Is pseudorandomness a requirement?  Or is it the "50% chance of a bitflip"?
>

You are asking if a pseudorandom number generator requires pseudorandomness?
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Blumenthal, Uri - 0553 - MITLL]

Defence in depth seems prudent: independent sources with agglomeration and 
whitening.

As Kurt noted, [on modern OSes,] it is really unclear what sources are 
available to us that are not already being used by the kernel.

 

I would turn to hardware. Since OpenSSL already has assembly-level optimization 
for various CPU types, accessing instructions like RDSEED and RDRAND (when 
available) doesn’t sound too hard. Mix their output into the seed. It can only 
improve the result.

 

So, [on these same modern OSes,] what benefit do we really get from using 
multiple "independent" sources?  They are unlikely to actually be independent 
if the kernel is consuming them as well and we consume the kernel.

 

Depends on what you mean by “independent”. A completely different mechanism – 
probably not. A mechanism whose output bits/bytes are not (tractably) 
correlated? Probably yes.



We shouldn't trust the user to provide entropy. 
 
Definitely. 

 

No.  We shouldn’t trust the user to provide all entropy – but should welcome 
user’s contribution to the entropy pool.




smime.p7s
Description: S/MIME cryptographic signature
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Wed, Jun 28, 2017 at 12:01:29PM -0500, Benjamin Kaduk via openssl-dev wrote:
> 
> I'm not sure what you mean by "draining the kernel's entropy pools". 
> That is, if you are adhering to the belief that taking random bits out
> of a generator removes entropy from it that must be replenished, does
> that not apply also to any generator/pool we write for ourselves?  Or
> maybe you just refer to the behavior of linux /dev/random, in which case
> I would point out Ted (the author/maintainer of linux /dev/random)'s
> suggestion to just use (getrandom or) /dev/random and tacit agreement
> that the behavior of reducing the entropy count on reads from
> /dev/random is not really needed anymore.

Replace all /dev/random with /dev/urandom.

> At boot time *all* pools are empty.  FreeBSD has a random seed file on
> disk to be loaded on next boot that helps with this (I didn't check
> linux),

It depends on the distro, but they should all be doing this. On
systems using systemd that file is probably
/var/lib/systemd/random-seed.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Benjamin Kaduk via openssl-dev]

On 06/26/2017 11:28 PM, Paul Dale wrote:
> Given the variety of RNGs available, would an EVP RNG interface make sense?  
> With a safe default in place (and no weak generators provided), the decision 
> can be left to the user.
> A side benefit is that the unit tests could implement a simple fully 
> deterministic generator and the code for producing known sequences removed 
> from the code base.

There are some benefits to this idea, as you note, but it does not seem
like a clear "immediate win" to me.  Maybe this is just some emotional
response that has not fully absorbed "no weak generators provided", as I
can't really articulate any reason to oppose it other than "randomness
is so low-level that we should just provide it and not options for it".

>
> Defence in depth seems prudent: independent sources with agglomeration and 
> whitening.

As Kurt noted, [on modern OSes,] it is really unclear what sources are
available to us that are not already being used by the kernel.  Rich had
commented about the dragonfly (kernel) implementation "wow, is it really
that easy?".  To large extent, yes, a secure RNG can present as being
that simple/easy -- if you're writing it in the kernel!  The kernel has
easy and direct access to lots of interrupt-driven entropy sources, any
hardware generators present, etc., as well as rdrand/etc.  It doesn't
have to worry about fork-safety or syscall overhead, and can basically
just implement the raw crypto needed for whitening/mixing/stretching.

So, [on these same modern OSes,] what benefit do we really get from
using multiple "independent" sources?  They are unlikely to actually be
independent if the kernel is consuming them as well and we consume the
kernel.

Now, of course OpenSSL runs on OSes that do not provide a modern kernel
RNG and we will need some solution for them, which will likely look as
you describe.  I'm just not convinced there is much value in duplicating
what the kernel is doing in the cases that the kernel does it well.

>
> We shouldn't trust the user to provide entropy.  I've seen what is typically 
> provided.  Uninitialised buffers aren't random.  User inputs (mouse and 
> keyboard) likewise aren't very good.  That both are still being suggested is 
> frustrating.  I've seen worse suggestions, some to the effect that 
> "time(NULL) ^ getpid()" is too good and just time() is enough.

Definitely.  But, as we're not the kernel, finding good sources of real
randomness as a generic userspace process is quite hard.

>
> As for specific questions and comments:
>
> John Denker wrote:
>> If you trust the ambient OS to provide a seed, why not
>> trust it for everything, and not bother to implement an
>> openssl-specfic RNG at all?
> I can think of a few possibilities:

Ah, preemptive replies to my comments above, excellent.

> * Diversifying the sources provides resistance to compromise of individual 
> sources.  Although a full kernel compromise is unrecoverable, a kernel bug 
> that leaked the internal pools in a read only manner isn't unforeseeable.

It is not unforseeable, sure, but so are lots of things.  Spewing the
contents of the openssl process-local randomness pool on the network
isn't unforseeable, either; do we have any reason to think there is
substantially more risk from one unknown than the other?

> * Not all operating systems have good RNGs.

Sure, and we need to support the ones that don't have good RNGs.
But on the ones that do, what do we gain from duplicating the effort?

>
> * Draining the kernel's entropy pools is unfriendly behaviour, other 
> processes will typically want some randomness too.
>
> * At boot time the kernel pools are empty (low or no quality).  This 
> compounds when several things require seeding.

I'm not sure what you mean by "draining the kernel's entropy pools". 
That is, if you are adhering to the belief that taking random bits out
of a generator removes entropy from it that must be replenished, does
that not apply also to any generator/pool we write for ourselves?  Or
maybe you just refer to the behavior of linux /dev/random, in which case
I would point out Ted (the author/maintainer of linux /dev/random)'s
suggestion to just use (getrandom or) /dev/random and tacit agreement
that the behavior of reducing the entropy count on reads from
/dev/random is not really needed anymore.

At boot time *all* pools are empty.  FreeBSD has a random seed file on
disk to be loaded on next boot that helps with this (I didn't check
linux), and openssl has/can use ~/.rnd or similar, but those are not
immune from compromise out-of-band.  In order to be properly confident
of good randomness, new randomness needs to be collected from the
environment and added to the pool, and the kernel is in a much better
position to do so (and know when it has enough!) than we are.

> * Performance is also a consideration, although with a gradual collection 
> strategy this should be less of a concern.  Except at start up.

Given that we're going to be 

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matthias St. Pierre]

Thanks for the quick reply. It sounds reasonable to make the default choice 
depending on the os environment.
For me it is not a religious question what OpenSSL's default choice should be. 
I trust that you will find a
sensible solution. And if OpenSSL supports both methods I can always make my 
own choice if I need to.

Regards,

Matthias


On 28.06.2017 16:46, Matt Caswell wrote:
>
> On 28/06/17 15:42, Matthias St. Pierre wrote:
>> Hello Matt,
>>
>> I am not quite sure what your current favourite solution for the upcoming 
>> default OpenSSL random generator is. Are you favouring
>>
>> - a DRBG  (following SP800-90Ar1) which is using the OS RNG as entropy 
>> source for (re-)seeding  or
>>
>> - simply passing all generate requests over to the OS RNG?
>>
>> It looks like you made two votes for the first and one vote for the second 
>> variant (see below). Could you please clarify your preference?
> Both :-)
>
> i.e. both should be available as an option.
>
> I don't think we will necessary be able to do the latter on all
> platforms that we support.
>
> As for which of the two is the default: where it is available - the
> latter. Where it isn't the former.
>
> Matt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matt Caswell]

On 28/06/17 15:42, Matthias St. Pierre wrote:
> Hello Matt,
> 
> I am not quite sure what your current favourite solution for the upcoming 
> default OpenSSL random generator is. Are you favouring
> 
> - a DRBG  (following SP800-90Ar1) which is using the OS RNG as entropy source 
> for (re-)seeding  or
> 
> - simply passing all generate requests over to the OS RNG?
> 
> It looks like you made two votes for the first and one vote for the second 
> variant (see below). Could you please clarify your preference?

Both :-)

i.e. both should be available as an option.

I don't think we will necessary be able to do the latter on all
platforms that we support.

As for which of the two is the default: where it is available - the
latter. Where it isn't the former.

Matt



> 
> Regards,
> 
> Matthias St. Pierre
> 
> 
> Vote 1:
> 
> On 27.06.2017 09:28, Matt Caswell wrote:
>> On 26/06/17 21:18, Kurt Roeckx wrote:
   “Recommendation for Random Number Generation Using Deterministic Random 
 Bit Generators”
   http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf

 That design may look complicated, but if you think you can
 leave out some of the blocks in their diagram, proceed with
 caution.  Every one of those blocks is there for a reason.
>>> SP800-90A (or revision 1) can clearly be used as reference on how
>>> to implement it, even if we don't use an approved algorithm from
>>> it. And I really think we should look at that document when
>>> implementing it.
>>>
>>> There should probably also be an option to use an RNG that
>>> conforms to it.
>> I am strongly in favour of this approach. We should be led by standards.
>>
> 
> Vote 2:   (comment on PR #3789: WIP: Add DRBG random method)
> 
> https://github.com/openssl/openssl/pull/3789#issuecomment-311494544
> 
> 
> Vote 3:
> 
> 
> On 28.06.2017 11:29, Matt Caswell wrote:
>> On 27/06/17 19:50, Benjamin Kaduk wrote:
>>> On 06/27/2017 02:28 AM, Matt Caswell wrote:
 On 26/06/17 21:18, Kurt Roeckx wrote:

> I think it should by default be provided by the OS, and I don't
> think any OS is documenting how much randomness it can provide.
>
 I also agree that, by default, using the OS provided source makes a lot
 of sense.

>>> Do you mean having openssl just pass through to
>>> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
>>> our own thing?
>> I meant the former.
>>
>> Matt
> 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matthias St. Pierre]

Hello Matt,

I am not quite sure what your current favourite solution for the upcoming 
default OpenSSL random generator is. Are you favouring

- a DRBG  (following SP800-90Ar1) which is using the OS RNG as entropy source 
for (re-)seeding  or

- simply passing all generate requests over to the OS RNG?

It looks like you made two votes for the first and one vote for the second 
variant (see below). Could you please clarify your preference?

Regards,

Matthias St. Pierre


Vote 1:

On 27.06.2017 09:28, Matt Caswell wrote:
> On 26/06/17 21:18, Kurt Roeckx wrote:
>>>   “Recommendation for Random Number Generation Using Deterministic Random 
>>> Bit Generators”
>>>   http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
>>>
>>> That design may look complicated, but if you think you can
>>> leave out some of the blocks in their diagram, proceed with
>>> caution.  Every one of those blocks is there for a reason.
>> SP800-90A (or revision 1) can clearly be used as reference on how
>> to implement it, even if we don't use an approved algorithm from
>> it. And I really think we should look at that document when
>> implementing it.
>>
>> There should probably also be an option to use an RNG that
>> conforms to it.
> I am strongly in favour of this approach. We should be led by standards.
>

Vote 2:   (comment on PR #3789: WIP: Add DRBG random method)

https://github.com/openssl/openssl/pull/3789#issuecomment-311494544


Vote 3:


On 28.06.2017 11:29, Matt Caswell wrote:
> On 27/06/17 19:50, Benjamin Kaduk wrote:
>> On 06/27/2017 02:28 AM, Matt Caswell wrote:
>>> On 26/06/17 21:18, Kurt Roeckx wrote:
>>>
 I think it should by default be provided by the OS, and I don't
 think any OS is documenting how much randomness it can provide.

>>> I also agree that, by default, using the OS provided source makes a lot
>>> of sense.
>>>
>> Do you mean having openssl just pass through to
>> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
>> our own thing?
> I meant the former.
>
> Matt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Theodore Ts'o]

On Wed, Jun 28, 2017 at 08:15:20AM +0100, Cory Benfield wrote:
> When you say “the linked article”, do you mean the PCWorld one?
> Because that article doesn’t provide any suggestion that
> /dev/urandom has anything to do with it. It is at least as likely
> that the SSH key is hard-coded into the machine image. The flaw here
> is not “using /dev/urandom”, it’s “exposing your router’s SSH access
> on the external side of the router”, plus the standard level of poor
> configuration done by shovelware router manufacturers.

The PC World article was clearly caused by hard-coding the RSA private
key in the flashed firmware and sharing it across fast numbers of routers.

The reference which I believe Paul Dale was looking for dates back to
five years ago, in the "Minding your P's and Q's paper", published in
Usenix Security:

 https://factorable.net/

The researchers gave me early access to the paper in early July and I
spent most of July 4th on the rooftop of the Museum of Science
rooftop, reworking Linux's random driver during the day (so we could
watch the fireworks from an advantageous position that evening).  We
had patches released before the P's and Q's paper was published, and
these fixes are in 3.6 and newer kernels.

There may still be issues with silly programs which initialize
symmetric keys the first time the system boots, instead of doing it on
an on-demand basis the first time someone tries connecting to the SSH
port (when there would be plenty of entropy mixed in).

And of course, things have gotten more challenging because systemd
starts the sshd service much more quickly, and if you are on an
embedded system which does not have a high resolution clock, and very
little entropy when the box is first booted (for example, if the user
plugs in the router without having first attached any of the LAN or
WAN cables), it may still be a problem on some systems.

Unfortunately, though, this ends up being a system design problem.
There only so much you can do just at the kernel level, or just at the
openssl level.

My recommendation, again, is to use getrandom(2) if it is available,
and if it isn't (on kernels older than 3.17) fall back to using
/dev/urandom.  On kernels older than 3.17, you're probably totally
screwed because the kernel is very likely going to be missing large
numbers of security fixes, so if someone is unfortunate enough to be
using a kernel which is older than 3.17, they're probably totally
screwed anyway.  (The only exception to this rule is if you're getting
monthly security updates from your cell phone provider, but if you're
not using an Google-supported android releases, and you're using a ARM
SOC with a 3.10 BSP kernel, you might as well give up now.  And if the
leader of the free world, is using such a device to tweet messages in
lieu of press conferences --- again, you've got bigger problems. :-)

If you are using a system which is 3.17 or newer, but your libc
doesn't have support for getrandom(2), I would recommend trying to
call getrandom(2) directly using the syscall(2) interface.  This will
mean hardcoding Linux's syscall number for different architectures,
which is painful and ugly, but that's going to be the best way to
block until you are sure the CRNG has been initialized, in case you do
have silly programs trying to generate long-term secrets a second or
so after the box is first powered on.

Cheers,

- Ted

P.S.  Short summry of kernel versions from an upstream perspective

3.17  -- first kernel version with getrandom(2)
3.18  -- Long Term Stable kernel
4.1 -- Long Term Stable kernel
4.4 -- Long Term Stable kernel
4.8 -- first kernel version using a ChaCha20 CRNG
4.9 -- Long Term Stable kernel
4.12 -- Next Long Term Stable kernel (to be released in a week or two)

(Long Term Stable kernels are ones which get upstream security fixes
backported by Greg K-H and his team.  Just because your kernel has a
LTS version may not mean you are getting the latest bug fixes.  SOC
vendors are notorious for not bothering to update their kernels with
the latest bug fixes, and many embedded and mobile device vendors
aren't bothering to keep up as well.)
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matt Caswell]

On 27/06/17 19:50, Benjamin Kaduk wrote:
> On 06/27/2017 02:28 AM, Matt Caswell wrote:
>>
>> On 26/06/17 21:18, Kurt Roeckx wrote:
>>
>>> I think it should by default be provided by the OS, and I don't
>>> think any OS is documenting how much randomness it can provide.
>>>
>> I also agree that, by default, using the OS provided source makes a lot
>> of sense.
>>
> 
> Do you mean having openssl just pass through to
> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
> our own thing?

I meant the former.

Matt
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Cory Benfield]

> On 28 Jun 2017, at 04:00, Paul Dale  wrote:
> 
> 
> Peter Waltenberg wrote:
>> The next question you should be asking is: does our proposed design mitigate 
>> known issues ?. 
>> For example this:
>> http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html
> 
> Using the OS RNG won't fix the lack of boot time randomness unless there is a 
> HRNG present.
> 
> For VMs, John's suggestion that /dev/hwrng should be installed is reasonable.
> 
> For embedded devices, a HRNG is often not possible.  Here getrandom() (or 
> /dev/random since old kernels are common) should be used.  Often /dev/urandom 
> is used instead and the linked article is the result.  There are possible 
> mitigations that some manufacturers include (usually with downsides).

When you say “the linked article”, do you mean the PCWorld one? Because that 
article doesn’t provide any suggestion that /dev/urandom has anything to do 
with it. It is at least as likely that the SSH key is hard-coded into the 
machine image. The flaw here is not “using /dev/urandom”, it’s “exposing your 
router’s SSH access on the external side of the router”, plus the standard 
level of poor configuration done by shovelware router manufacturers.

Cory

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Dale]

Peter Waltenberg wrote:
> The next question you should be asking is: does our proposed design mitigate 
> known issues ?. 
> For example this:
> http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html

Using the OS RNG won't fix the lack of boot time randomness unless there is a 
HRNG present.

For VMs, John's suggestion that /dev/hwrng should be installed is reasonable.

For embedded devices, a HRNG is often not possible.  Here getrandom() (or 
/dev/random since old kernels are common) should be used.  Often /dev/urandom 
is used instead and the linked article is the result.  There are possible 
mitigations that some manufacturers include (usually with downsides).

The question is should a security toolkit try to do a good job regardless?

I've seen more cases than I care to count where long term cryptographic 
material is generated on first boot out of the factory.  I've even seen some 
cases where this was done during the factory test.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Peter Waltenberg]

If the desired outcome is security you must generate instance unique keys 
and elegant software design alone is simply not enough to achieve that. 

And I didn't say solve below I said mitigate. 
You can't solve the problem of someone using already created keys in 
multiple VM's. 
But you can and should reduce the chances that someone will create them 
from a fresh keygen because that simply can't be mitigated anywhere else 
but in your code.

Simillar issues exist with fork(), and again, you should make efforts to 
mitigate that risk because the user can't.

Magic fairy dust like (/dev/hwrng) undoubtedly helps where it exists, but 
you still have to apply it correctly to achieve the desired outcome.

Peter



From:   John Denker via openssl-dev <openssl-dev@openssl.org>
To: "openssl-dev@openssl.org" <openssl-dev@openssl.org>
Date:   28/06/2017 12:19
Subject:    Re: [openssl-dev] Work on a new RNG for OpenSSL
Sent by:"openssl-dev" <openssl-dev-boun...@openssl.org>



On 06/27/2017 06:41 PM, Peter Waltenberg wrote:

> Consider most of the worlds compute is now done on VM's where images are 

> cloned, duplicated and restarted as a matter of course. Not vastly 
> different from an embedded system where the clock powers up as 00:00 
> 1-Jan, 1970 on each image. If you can trust the OS to come up with 
unique 
> state each time you can rely solely on the OS RNG - well provided you 
> reseed often enough anyway, i.e. before key generation. That's also why 
> seeding a chain of PRNG's once at startup is probably not sufficient 
here.

That is approximately the last thing openssl should be
fussing over.  There is a set of problems there, with a
set of solutions, none of which openssl has any say over.

===>  The VM setup should provide a virtual /dev/hwrng  <===

Trying to secure a virtual machine without a virtual hwrng
(or the equivalent) is next to impossible.  There may be
workarounds, but they tend to be exceedingly locale-specific,
and teaching openssl to try to discover them would be a
tremendous waste of resources.

So stop trying to operate without /dev/hwrng already.

It reminds me of the old Smith & Dale shtick:
  -- Doctor, doctor, it hurts when I do *this*.
  -- So don't do that.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev





-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Theodore Ts'o]

On Wed, Jun 28, 2017 at 11:41:11AM +1000, Peter Waltenberg wrote:
> And FYI. On systems not backed with hardware RNG's /dev/random is 
> extremely slow. 1-2 bytes/second is a DOS attack on it's own without any 
> other effort required.

Please, stop suggesting the use /dev/random.  The right answer is
/dev/urandom or getrandom(2).

- Ted
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Theodore Ts'o]

On Wed, Jun 28, 2017 at 01:50:49AM +, Salz, Rich wrote:
> That's interesting info Ted, thanks. But we don't currently know
> anything about which kernel or glibc version we're building for;
> maybe that has to change.  (We barely, and rarely, look at the
> toolchain version.)  And we need to be able to build a version which
> runs across a whole mess of kernels and x86 CPU's.

The performance numbers for /dev/urandom and getrandom pre-4.9 should
be largely consistent until you get back to when we were using MD5
instead of SHA-1 for the CRNG.  When you get back to the days when
/dev/urandom used MD5, the performance will be (obviously) faster, and
given the way we use the cryptographic hash algorithms, the currently
known attacks to MD5 and SHA won't be an issue.

The much *LARGER* problem is that those ancient kernels are almost
certainly filled with unpatched security holes (e.g., the kind that
might be exploitable remotely using a malformed ICMP packet.)  Why
should the attacker bother with some complex cryptographic attack with
a simple buffer overrun will get them all that they need, much more
easily?

If you are using an older version of glibc, that doesn't have
getrandom(2) support, you can either attempt to call getrandom(2)
directly via the syscall interface, or you can just fall back to
opening /dev/urandom and reading from it.  As long as you do proper
error checking to catch things like the fd exhaustion attack, you
should be fine from a security perspective, and the performance of
reading from /dev/urandom should be no different from using
getrandom(2), assuming you keep the fd to /dev/urandom open during the
life of the process.

If you have a pre-3.17 kernel that doesn't have getrandom(2) support,
the same logic applies above.  You won't have a performance problem,
and the potential security concerns (IMHO, of a purely certificational
nature) of /dev/random from that era will be the *least* of your
problems.

Cheers,

- Ted
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Dale]

Ben wrote:
> On 06/27/2017 07:24 PM, Paul Dale wrote:
>> The hierarchy of RNGs will overcome some of the
>> performance concerns.  Only the root needs to call getrandom().
>> I do agree that having a DRBG at the root level is a good idea though.
 
> Just to check my understanding, the claim is that adding more layers of 
> hashing and/or encryption will still be faster than a larger number of 
> syscalls?

I'm not sure if it will be faster or not, although it seems likely.  The kernel 
will have to do the same cryptographic operations so using it adds a syscall 
overhead.  If the kernel is doing different cryptographic operations, then it 
could be faster.
 
However, I'm more interested in separation of the random sources.  I'd prefer 
to not be sharing my RNG with others if possible.  A compromise is unlikely but 
if one happens it would be nice to limit the damage.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia

From: Benjamin Kaduk [mailto:bka...@akamai.com] 
Sent: Wednesday, 28 June 2017 11:22 AM
To: openssl-dev@openssl.org; Paul Dale <paul.d...@oracle.com>
Subject: Re: [openssl-dev] Work on a new RNG for OpenSSL


-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[John Denker via openssl-dev]

On 06/27/2017 06:41 PM, Peter Waltenberg wrote:

> Consider most of the worlds compute is now done on VM's where images are 
> cloned, duplicated and restarted as a matter of course. Not vastly 
> different from an embedded system where the clock powers up as 00:00 
> 1-Jan, 1970 on each image. If you can trust the OS to come up with unique 
> state each time you can rely solely on the OS RNG - well provided you 
> reseed often enough anyway, i.e. before key generation. That's also why 
> seeding a chain of PRNG's once at startup is probably not sufficient here.

That is approximately the last thing openssl should be
fussing over.  There is a set of problems there, with a
set of solutions, none of which openssl has any say over.

===>  The VM setup should provide a virtual /dev/hwrng  <===

Trying to secure a virtual machine without a virtual hwrng
(or the equivalent) is next to impossible.  There may be
workarounds, but they tend to be exceedingly locale-specific,
and teaching openssl to try to discover them would be a
tremendous waste of resources.

So stop trying to operate without /dev/hwrng already.

It reminds me of the old Smith & Dale shtick:
  -- Doctor, doctor, it hurts when I do *this*.
  -- So don't do that.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Theodore Ts'o]

On Tue, Jun 27, 2017 at 04:12:48PM -0500, Benjamin Kaduk wrote:
> 
> While you're here, would you mind confirming/denying the claim I read
> that the reason the linux /dev/random tracks an entropy estimate and
> blocks when it gets too low is to preserve backward security in the face
> of attacks against SHA1?

The explanation is a bit more complicated than that.  Linux was the
first OS to have a built-in random number generator suitable for use
in cryptographic applications.  Linux's /dev/random dates back to the
days when we still had export control (this is why a cryptographic
hash was used instead of an encryption algorithm), and when we didn't
have nearly as much understanding about cryptoanalysis as we do now.

As a result, just about all of the RNG's from that era (such the
original PGP's RNG, designed by Colin Plumb, and with whom I consulted
extensively when I designed Linux's /dev/random system) were much more
focused on collecting environmental noise and using cryptographic
primitives for mixing.  We were much less comfortable "back in the
day" in being willing to trust in the strength of any cryptographic
primitives.  Given the weaknesses that were found in MD4 and MD5
(which date from that era), I'd say that our caution was justified.

The original entropy estimation and accounting philosophy comes from
these historical reasons.  The reason why it is still preserved is
mainly because the shift from trusting in cryptographic primitives
versus entropy estimates was gradual, and by the time we had reached
general consensus, just about everyone was using /dev/urandom anyway.
And there was no real point in changing /dev/random.

In addition, as it turns out, we still need an entropy estimate
anyway.  First of all, you still need to decide when you have gathered
enough environmental "noise" that you are willing to consider the CRNG
to be fully initialized.  Using some kind of entropy estimate is good
way to do this which allows us to have some kind of standard which is
portable across wide variety of CPU architecture and hardware
configurations.

Secondly, if you are using certain types of hardware random number
generators, they may be expensive to use them continuously, so using
the entropy accounting to control how often to read from the hwrng can
be useful.  This also allows us to assign a "derating percentage" if
we don't want to invest complete trust in the quality of the output
from said hardware RNG.

In any case, at this point what I recommend is that people just use
getrandom(2) and be happy.  Getrandom will block until CRNG is
initialized (which is only really an issue during early boot on most
systems), and eliminates a lot of the other potential pitfalls with
using /dev/urandom (for example, it's not vulnerable to a file
descriptor exhaustion attack combined with the universal problem that
most userspace application programmers don't bother to check error
returns).

If your libc doesn't support getrandom(2) for whatever reason, open
and read from /dev/urandom instead.  Unless you are running during
early boot (and doing something really silly like generating long term
public keys a few seconds after the box is powered on, fresh from the
factory), you should be fine so long as do proper error checking of
all of your system calls.

Cheers,

- Ted
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> Just to check my understanding, the claim is that adding more layers of 
> hashing and/or encryption will still be faster than a larger number of 
> syscalls?

In terms of overall system performance?  Yes.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

--
Senior Architect, Akamai Technologies
Member, OpenSSL Dev Team
IM: richs...@jabber.at Twitter: RichSalz

From: Benjamin Kaduk via openssl-dev [mailto:openssl-dev@openssl.org]
Sent: Tuesday, June 27, 2017 9:22 PM
To: openssl-dev@openssl.org; Paul Dale <paul.d...@oracle.com>
Subject: Re: [openssl-dev] Work on a new RNG for OpenSSL

On 06/27/2017 07:24 PM, Paul Dale wrote:

The hierarchy of RNGs will overcome some of the performance concerns.  Only the 
root needs to call getrandom().
I do agree that having a DRBG at the root level is a good idea though.


Just to check my understanding, the claim is that adding more layers of hashing 
and/or encryption will still be faster than a larger number of syscalls?

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

That's interesting info Ted, thanks. But we don't currently know anything about 
which kernel or glibc version we're building for; maybe that has to change.  
(We barely, and rarely, look at the toolchain version.)  And we need to be able 
to build a version which runs across a whole mess of kernels and x86 CPU's.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Peter Waltenberg]

The next question you should be asking is: does our proposed design 
mitigate known issues ?. 
For example this:

http://www.pcworld.com/article/2886432/tens-of-thousands-of-home-routers-at-risk-with-duplicate-ssh-keys.html

Consider most of the worlds compute is now done on VM's where images are 
cloned, duplicated and restarted as a matter of course. Not vastly 
different from an embedded system where the clock powers up as 00:00 
1-Jan, 1970 on each image. If you can trust the OS to come up with unique 
state each time you can rely solely on the OS RNG - well provided you 
reseed often enough anyway, i.e. before key generation. That's also why 
seeding a chain of PRNG's once at startup is probably not sufficient here.

And FYI. On systems not backed with hardware RNG's /dev/random is 
extremely slow. 1-2 bytes/second is a DOS attack on it's own without any 
other effort required.

This isn't solely a matter of good software design. And yes, I know, hard 
problem. If it wasn't a hard problem you probably wouldn't be dealing with 
it now.


Peter




From:   Benjamin Kaduk via openssl-dev <openssl-dev@openssl.org>
To: openssl-dev@openssl.org, Kurt Roeckx <k...@roeckx.be>, John Denker 
<s...@av8n.com>
Date:   28/06/2017 09:38
Subject:        Re: [openssl-dev] Work on a new RNG for OpenSSL
Sent by:"openssl-dev" <openssl-dev-boun...@openssl.org>



On 06/27/2017 04:51 PM, Kurt Roeckx wrote:
On Tue, Jun 27, 2017 at 11:56:04AM -0700, John Denker via openssl-dev 
wrote:


On 06/27/2017 11:50 AM, Benjamin Kaduk via openssl-dev wrote:


Do you mean having openssl just pass through to
getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
our own thing?

The former seems simpler and preferable to me (perhaps modulo linux's
broken idea about "running out of entropy")


That's a pretty big modulus.  As I wrote over on the crypto list:

The xenial 16.04 LTS manpage for getrandom(2) says quite explicitly:


Unnecessarily reading large quantities  of data will have a
negative impact on other users of the /dev/random and /dev/urandom
devices.


And that's an understatement.  Whether unnecessary or not, reading
not-particularly-large quantities of data is tantamount to a
denial of service attack against /dev/random and against its
upstream sources of randomness.

No later LTS is available.  Reference:
  http://manpages.ubuntu.com/manpages/xenial/man2/getrandom.2.html

Recently there has been some progress on this, as reflected in in
the zesty 17.04 manpage:
  http://manpages.ubuntu.com/manpages/zesty/man2/getrandom.2.html

However, in the meantime openssl needs to run on the platforms that
are out there, which includes a very wide range of platforms.


And I think it's actually because of changes in the Linux RNG that
the manpage has been changed, but they did not document the
different behavior of the kernel versions.

In case it wasn't clear, I think we should use the OS provided
source as a seed. By default that should be the only source of
randomness.



I think we can get away with using OS-provided randomness directly in many 
common cases.  /dev/urandom suffices once we know that the kernel RNG has 
been properly seeded.  On FreeBSD, /dev/urandom blocks until the kernel 
RNG is seeded; on other systems maybe we have to make one read from 
/dev/random to get the blocking behavior we want before switching to 
/dev/urandom for bulk reads.

-Ben-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev




-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Theodore Ts'o]

On Tue, Jun 27, 2017 at 09:02:54PM +, Salz, Rich wrote:
> > What is your concerns about syscall performance?  What are your
> > performance requirements?  I can tell you that Chrome has been using
> > /dev/urandom 
> 
> Well, Chrome ultimately works at human-scale.  On the server side, thousands 
> of connections per second and one or two syscalls per connection seems like 
> something we should avoid.
> 
> > My recommendation for Linux is to use getrandom(2) the flags field set to
> > zero. 
> 
> And for older Linux?

The Chacha20 based CRNG is available since Linux 4.9.  So Debian
Stable and Ubuntu LTS 16.04.3 (out in August) will be using a kernel
with the ChaCha20 based CRNG.

Using the attached benchmark program, which calls getrandom() a
million times to generate a 32 byte (256 bit) session key, the latency
of getrandom(2) is approximately 2.9 microseconds, when run on a
n1-standard-1 GCE VM.

For older Linux systems, where you are using a SHA1 based CRNG, the
time to run getrandom(2) is approximately 5.5 microseconds on the same
class of VM.  It really is not that much worse.  And I'll gently
suggest that the public key operations will probably dominate the cost
of calling getrandom(2) or reading from /dev/urandom.

The reason why I moved to the ChaCha20 based algorithm was only
partially about the speed.  It also had the significant bonus that I
didn't have keep on explaining that theoretical and demonstrated
attacks against SHA-1 weren't applicable to how it was being used in
Linux's RNG.  (See also previous comments about how a lot of
complaints about RNG's are more religious than real.)

It's also very unlikely that Enterprise distributions will bother
updating to a newer version of OpenSSL, since in general they don't
want to break shared library ABI's, and the version of OpenSSL that
they shipped with originally will almosrt certainly have an
incompatible ABI with a modern version of OpenSSL.  So I doubt this is
really a significant issue.

By the time a new version OpenSSL which uses getrandom(2) is picked up
by Linux distributions, it is extremely likely that they will be using
a 4.9 or newer kernel --- and unless you have a crypto accelerator to
speed up your symmetric key operations, I really doubt performance is
going to be an issue when you are using getrandom(2), regardless of
which kernel version you are using.

Cheers,

  - Ted

P.S.  The version of the getrandom(2) manpage in Ubuntu LTS's are very
much obsolete.  Even after Ubuntu LTS updates to a 4.9 or newer kernel
when 16.04.3 gets released in August 2017, it's unlikely they will
bother to update the manpages, and the 16.04 getrandom(2) man page was
outdated when the distribution was released.  So arguments that
getrandom(2) should not be used based on the manpage are not valid.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Benjamin Kaduk via openssl-dev]

On 06/27/2017 07:24 PM, Paul Dale wrote:
>
> The hierarchy of RNGs will overcome some of the performance concerns. 
> Only the root needs to call getrandom().
>
> I do agree that having a DRBG at the root level is a good idea though.
>
>  
>

Just to check my understanding, the claim is that adding more layers of
hashing and/or encryption will still be faster than a larger number of
syscalls?

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Dale]

The hierarchy of RNGs will overcome some of the performance concerns.  Only the 
root needs to call getrandom().

I do agree that having a DRBG at the root level is a good idea though.

 

Pauli

-- 

Oracle

Dr Paul Dale | Cryptographer | Network Security & Encryption 

Phone +61 7 3031 7217

Oracle Australia

 

From: Salz, Rich via openssl-dev [mailto:openssl-dev@openssl.org] 
Sent: Wednesday, 28 June 2017 4:56 AM
To: Kaduk, Ben <bka...@akamai.com>; openssl-dev@openssl.org; Matt Caswell 
<m...@openssl.org>
Subject: Re: [openssl-dev] Work on a new RNG for OpenSSL

 

For windows RAND_bytes should just call CryptGenRandom (or its equiv).  For 
modern Linux, probably call getrandom(2).  For OpenBSD call arc4random().

 

Getrandom() is a syscall, and I have concerns about the syscall performance.  I 
would rather feed getrandom (or /dev/random if that’s not available) into a 
FIPS DRBG generator.

 
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> In case it wasn't clear, I think we should use the OS provided source as a
> seed. By default that should be the only source of randomness.

I generally agree.  But some applications might save their state and restore it 
during boot time, for example.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Benjamin Kaduk via openssl-dev]

On 06/27/2017 04:51 PM, Kurt Roeckx wrote:
> On Tue, Jun 27, 2017 at 11:56:04AM -0700, John Denker via openssl-dev wrote:
>>
>> On 06/27/2017 11:50 AM, Benjamin Kaduk via openssl-dev wrote:
>>
>>> Do you mean having openssl just pass through to
>>> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
>>> our own thing?
>>>
>>> The former seems simpler and preferable to me (perhaps modulo linux's
>>> broken idea about "running out of entropy")
>> That's a pretty big modulus.  As I wrote over on the crypto list:
>>
>> The xenial 16.04 LTS manpage for getrandom(2) says quite explicitly:
>>
 Unnecessarily reading large quantities  of data will have a
 negative impact on other users of the /dev/random and /dev/urandom
 devices.
>> And that's an understatement.  Whether unnecessary or not, reading
>> not-particularly-large quantities of data is tantamount to a
>> denial of service attack against /dev/random and against its
>> upstream sources of randomness.
>>
>> No later LTS is available.  Reference:
>>   http://manpages.ubuntu.com/manpages/xenial/man2/getrandom.2.html
>>
>> Recently there has been some progress on this, as reflected in in
>> the zesty 17.04 manpage:
>>   http://manpages.ubuntu.com/manpages/zesty/man2/getrandom.2.html
>>
>> However, in the meantime openssl needs to run on the platforms that
>> are out there, which includes a very wide range of platforms.
> And I think it's actually because of changes in the Linux RNG that
> the manpage has been changed, but they did not document the
> different behavior of the kernel versions.
>
> In case it wasn't clear, I think we should use the OS provided
> source as a seed. By default that should be the only source of
> randomness.
>

I think we can get away with using OS-provided randomness directly in
many common cases.  /dev/urandom suffices once we know that the kernel
RNG has been properly seeded.  On FreeBSD, /dev/urandom blocks until the
kernel RNG is seeded; on other systems maybe we have to make one read
from /dev/random to get the blocking behavior we want before switching
to /dev/urandom for bulk reads.

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Tue, Jun 27, 2017 at 11:56:04AM -0700, John Denker via openssl-dev wrote:
> On 06/27/2017 02:28 AM, Matt Caswell wrote:
> >>
> >> I also agree that, by default, using the OS provided source makes a lot
> >> of sense.
> 
> Reality is more complicated than that;  see below.
> 
> On 06/27/2017 11:50 AM, Benjamin Kaduk via openssl-dev wrote:
> 
> > Do you mean having openssl just pass through to
> > getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
> > our own thing?
> > 
> > The former seems simpler and preferable to me (perhaps modulo linux's
> > broken idea about "running out of entropy")
> 
> That's a pretty big modulus.  As I wrote over on the crypto list:
> 
> The xenial 16.04 LTS manpage for getrandom(2) says quite explicitly:
> 
> >> Unnecessarily reading large quantities  of data will have a
> >> negative impact on other users of the /dev/random and /dev/urandom
> >> devices.
> 
> And that's an understatement.  Whether unnecessary or not, reading
> not-particularly-large quantities of data is tantamount to a
> denial of service attack against /dev/random and against its
> upstream sources of randomness.
> 
> No later LTS is available.  Reference:
>   http://manpages.ubuntu.com/manpages/xenial/man2/getrandom.2.html
> 
> Recently there has been some progress on this, as reflected in in
> the zesty 17.04 manpage:
>   http://manpages.ubuntu.com/manpages/zesty/man2/getrandom.2.html
> 
> However, in the meantime openssl needs to run on the platforms that
> are out there, which includes a very wide range of platforms.

And I think it's actually because of changes in the Linux RNG that
the manpage has been changed, but they did not document the
different behavior of the kernel versions.

In case it wasn't clear, I think we should use the OS provided
source as a seed. By default that should be the only source of
randomness.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> RAND_set_rand_method(RAND_drbg_method());
> 
> FIPS_drbg_set_reseed_interval()   and
> FIPS_drbg_set_callbacks()

Take a look at https://github.com/openssl/openssl/pull/3789  Thanks!
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Benjamin Kaduk via openssl-dev]

Hi Ted,

On 06/27/2017 03:40 PM, Theodore Ts'o wrote:
>
> My recommendation for Linux is to use getrandom(2) the flags field set
> to zero.  This will cause it to use a CRNG that will be reseeded every
> five minutes from environmental noise gathered primarily from
> interrupt timing data.  For modern kernels, the CRNG is based on
> ChaCha20.  For older kernels, it is based on SHA-1.
>
> There are a lot of people who have complained about whether or not
> Linux's urandom generator has met with there religious beliefs about
> how RNG's should be designed and implemented.  One of the things you
> will find is that many of these people are very vocal, and in some
> cases, their advice will be mutually exclusive.  So if you are going
> to be trying to design your own RNG for OpenSSL --- welcome to my
> world.
>
> (In other words, I do listen to many of the people who have opined on
> this thread.  I just don't happen to agree with all of them.  And I
> suspect you will find that in the end, it's impossible to make them
> all happy, and they will end up questioning your intelligence,
> judgement, and in some cases, your paternity.  :-)
>

Thanks for the input, and for reading what is being said.

While you're here, would you mind confirming/denying the claim I read
that the reason the linux /dev/random tracks an entropy estimate and
blocks when it gets too low is to preserve backward security in the face
of attacks against SHA1?

I'm happy to respect that there are different opinions, but it would be
nice to know the reasoning behind the behavior, even if I do not
necessarily agree with it.

Thanks,

Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Theodore Ts'o]

On Tue, Jun 27, 2017 at 06:55:47PM +, Salz, Rich via openssl-dev wrote:
> Getrandom() is a syscall, and I have concerns about the syscall
> performance.  I would rather feed getrandom (or /dev/random if
> that’s not available) into a FIPS DRBG generator.

What is your concerns about syscall performance?  What are your
performance requirements?  I can tell you that Chrome has been using
/dev/urandom (which has the same performance characteristics as the
getrandom system call) directly for all of its random number
generation needs (e.g., it's calling each time in dees to generate a
session key for TLS, etc.) and no one has complained.

My recommendation for Linux is to use getrandom(2) the flags field set
to zero.  This will cause it to use a CRNG that will be reseeded every
five minutes from environmental noise gathered primarily from
interrupt timing data.  For modern kernels, the CRNG is based on
ChaCha20.  For older kernels, it is based on SHA-1.

There are a lot of people who have complained about whether or not
Linux's urandom generator has met with there religious beliefs about
how RNG's should be designed and implemented.  One of the things you
will find is that many of these people are very vocal, and in some
cases, their advice will be mutually exclusive.  So if you are going
to be trying to design your own RNG for OpenSSL --- welcome to my
world.

(In other words, I do listen to many of the people who have opined on
this thread.  I just don't happen to agree with all of them.  And I
suspect you will find that in the end, it's impossible to make them
all happy, and they will end up questioning your intelligence,
judgement, and in some cases, your paternity.  :-)

- Ted

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> > Getrandom() is a syscall, and I have concerns about the syscall
> > performance.  I would rather feed getrandom (or /dev/random if that’s
> > not available) into a FIPS DRBG generator.
> 
> What is your concerns about syscall performance?  What are your
> performance requirements?  I can tell you that Chrome has been using
> /dev/urandom 

Well, Chrome ultimately works at human-scale.  On the server side, thousands of 
connections per second and one or two syscalls per connection seems like 
something we should avoid.

> My recommendation for Linux is to use getrandom(2) the flags field set to
> zero. 

And for older Linux?

> So if you are going to be trying to design your own RNG
> for OpenSSL --- welcome to my world.

We seem to have moved away from that somewhat.  That's a better place to be.

> find that in the end, it's impossible to make them all happy, and they will 
> end
> up questioning your intelligence, judgement, and in some cases, your
> paternity.  :-)

I miss Usenet. :)


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[John Denker via openssl-dev]

On 06/27/2017 02:28 AM, Matt Caswell wrote:
>>
>> I also agree that, by default, using the OS provided source makes a lot
>> of sense.

Reality is more complicated than that;  see below.

On 06/27/2017 11:50 AM, Benjamin Kaduk via openssl-dev wrote:

> Do you mean having openssl just pass through to
> getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
> our own thing?
> 
> The former seems simpler and preferable to me (perhaps modulo linux's
> broken idea about "running out of entropy")

That's a pretty big modulus.  As I wrote over on the crypto list:

The xenial 16.04 LTS manpage for getrandom(2) says quite explicitly:

>> Unnecessarily reading large quantities  of data will have a
>> negative impact on other users of the /dev/random and /dev/urandom
>> devices.

And that's an understatement.  Whether unnecessary or not, reading
not-particularly-large quantities of data is tantamount to a
denial of service attack against /dev/random and against its
upstream sources of randomness.

No later LTS is available.  Reference:
  http://manpages.ubuntu.com/manpages/xenial/man2/getrandom.2.html

Recently there has been some progress on this, as reflected in in
the zesty 17.04 manpage:
  http://manpages.ubuntu.com/manpages/zesty/man2/getrandom.2.html

However, in the meantime openssl needs to run on the platforms that
are out there, which includes a very wide range of platforms.

It could be argued that the best *long-term* strategy is to file
a flurry of bug reports against the various kernel RNGs, and then
at some *later* date rely on whatever the kernel provides ... but
still, in the meantime openssl needs to run on the platforms that
are out there.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

For windows RAND_bytes should just call CryptGenRandom (or its equiv).  For 
modern Linux, probably call getrandom(2).  For OpenBSD call arc4random().

Getrandom() is a syscall, and I have concerns about the syscall performance.  I 
would rather feed getrandom (or /dev/random if that’s not available) into a 
FIPS DRBG generator.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Benjamin Kaduk via openssl-dev]

On 06/27/2017 02:28 AM, Matt Caswell wrote:
>
> On 26/06/17 21:18, Kurt Roeckx wrote:
>
>> I think it should by default be provided by the OS, and I don't
>> think any OS is documenting how much randomness it can provide.
>>
> I also agree that, by default, using the OS provided source makes a lot
> of sense.
>

Do you mean having openssl just pass through to
getrandom()/read()-from-'/dev/random'/etc. or just using those to seed
our own thing?

The former seems simpler and preferable to me (perhaps modulo linux's
broken idea about "running out of entropy"), but the argument presented
about us being used in all sorts of environments that we can't even
enumerate has basically convinced me that we will need to provide some
alternative as well.  (It remains unclear how such environments will be
able to provide usable seed randomness, but there is only so much we can
do about that.)

-Ben
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Tue, Jun 27, 2017 at 02:42:52PM +0200, Matthias St. Pierre wrote:
> 
> So I have two questions:
> 
> - Do you intend to continue supporting RAND_set_rand_method() or will there 
> only be one 'perfect' random generator and no choice anymore?

I think we should have a default one, but an option to have a
different one.

> - Do you consider the SP800-90A DRBG outdated or will there be a chance that 
> it will be added to the OpenSSL master as
>   officially supported RAND method?

I think we should have at least 1 that follows SP800-90A, it's
clearly something some people will need.

> - Will the new OpenSSL RNG support a way to configure reseed intervals and 
> external entropy sources in a similar fashion
>   as the FIPS DRBG did?

It should at least reseed by default. Having an option to change
the default interval might make sense.

There clearly should be a way to use a source other than the one
provided by the kernel, which I think it needed for SP800-90A.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Mon, Jun 26, 2017 at 09:39:47PM -0700, John Denker via openssl-dev wrote:
> 
> I'm not mentioning any names, but some people are *unduly*
> worried about recovery following compromise of the PRNG
> internal state, so they constantly re-seed the PRNG, to
> the point where it becomes a denial-of-service attack
> against the upstream source of randomness.
> 
> This is also mostly pointless, because any attack that
> compromises the PRNG state will likely compromise so many
> other things that recovery will be very difficult.  All
> future outputs will be suspect.
> 
> So please let's not go overboard in that direction.
> 
> On the other hand, it seems reasonable to insist on /forward/
> secrecy.  That is, we should insist that /previous/ outputs
> should not be compromised.  This is achievable at small but
> not-quite-zero cost.

I think that's named backward secrecy?


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matthias St. Pierre]

On 27.06.2017 14:55, Salz, Rich via openssl-dev wrote:
> That's three questions :) But yes, we should address that. I'm not sure if 
> new RAND API's are the way to go or perhaps a RAND_control API that gives us 
> a bit more flexibility.

Ups, it used to be only two. That's always the problem with last minute changes 
;-).  Thanks for your quick reply and for considering my suggestions.

Regards,
Matthias


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> > Well maybe I can ignore section 10.3?
> >
> 
> That's a nice joke Rich, but the Dual_EC_DRBG chapter has been dropped in
> SP800-90Ar1, which supersedes SP800-90A:

I know.  I was trying to gently point out that even John makes mistakes :)

> - Do you intend to continue supporting RAND_set_rand_method() or will
> there only be one 'perfect' random generator and no choice anymore?

This will continue to work.
 
> - Do you consider the SP800-90A DRBG outdated or will there be a chance
> that it will be added to the OpenSSL master as
>   officially supported RAND method?

That's a great idea, I can work on that now.

> - Will the new OpenSSL RNG support a way to configure reseed intervals and
> external entropy sources in a similar fashion
>   as the FIPS DRBG did?

That's three questions :)  But yes, we should address that.  I'm not sure if 
new RAND API's are the way to go or perhaps a RAND_control API that gives us a 
bit more flexibility.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matthias St. Pierre]

On 26.06.2017 20:51, Salz, Rich via openssl-dev wrote:
>
>> Constructive suggestion:  If you want to see what a RNG looks like when
>> designed by cryptographers, take a look at:
>>   Elaine Barker and John Kelsey,
>>   “Recommendation for Random Number Generation Using Deterministic
>> Random Bit Generators”
>>   http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
>>
>> That design may look complicated, but if you think you can leave out some of
>> the blocks in their diagram, proceed with caution.  Every one of those blocks
>> is there for a reason.
> Well maybe I can ignore section 10.3?
>  

That's a nice joke Rich, but the Dual_EC_DRBG chapter has been dropped in 
SP800-90Ar1, which supersedes SP800-90A:

http://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-90Ar1.pdf

But seriously: OpenSSL already has an implementation of the SP800-90A DRBG, but 
unfortunately it is only part of the
FIPS object module (see reference [1] below). I always wondered why the code 
was never migrated to OpenSSL master,
(say, replacing the FIPS_drbg_* names by e.g. RAND_drbg_*). Then the SP800-90A 
DRBG would be usable by everyone
and could be activated by

RAND_set_rand_method(RAND_drbg_method());

To me, the design and implementation of the DRBG appeals sophisticated and I 
like its concept for reseeding which is highly
configurable using

FIPS_drbg_set_reseed_interval()   and
FIPS_drbg_set_callbacks()

In fact, we are currently using the AES-CTR DRBG in our product (see [2]) 
because we had the requirement that the random generator
should be seeded periodically from an external entropy source, for example a 
smart card or a  cryptographic acceleration unit.
This was easily achieved using the aforementioned DRBG callback mechanism.

So I have two questions:

- Do you intend to continue supporting RAND_set_rand_method() or will there 
only be one 'perfect' random generator and no choice anymore?

- Do you consider the SP800-90A DRBG outdated or will there be a chance that it 
will be added to the OpenSSL master as
  officially supported RAND method?

- Will the new OpenSSL RNG support a way to configure reseed intervals and 
external entropy sources in a similar fashion
  as the FIPS DRBG did?


Best regards,

Matthias  St. Pierre




[1] Section 6.1 of the OpenSSL FIPS User Guide 2.0 
https://www.openssl.org/docs/fips/UserGuide-2.0.pdf


[2] We link against a FIPS capable OpenSSL 1.0.2 crypto library and use the 
FIPS DRBG even in the case where FIPS mode
  is not enabled globally: In that case, during initialization we check 
whether FIPS mode initialization is successfull, then
  and then turn FIPS mode off again and only keep the random generator by 
calling RAND_set_rand_method(FIPS_drbg_method()).
  For Windows, we had to add some FIPS_drbg_* symbols to  libeay.num to 
make this work.








-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

John,

Thanks for pushing.  It can be a thankless task, and I wanted to say thank you 
:)


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Gilles Van Assche]

> We welcome your input.

On this topic, I would like to point out the construction we presented
at SAC 2011 [1]. It implements a reseedable pseudo-random number
generator in a rather simple way. One can supply seeds, extract
pseudo-random numbers and apply a ratchet mechanism at any chosen time.

We implemented it in the Keccak code package. The documentation can be
found there [2].

Kind regards,
Gilles, for the Keccak team

[1] http://eprint.iacr.org/2011/499.pdf
[2]
https://github.com/gvanas/KeccakCodePackage/blob/master/Modes/KeccakPRG.h

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matt Caswell]

On 26/06/17 21:18, Kurt Roeckx wrote:
>>   “Recommendation for Random Number Generation Using Deterministic Random 
>> Bit Generators”
>>   http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf
>>
>> That design may look complicated, but if you think you can
>> leave out some of the blocks in their diagram, proceed with
>> caution.  Every one of those blocks is there for a reason.
> 
> SP800-90A (or revision 1) can clearly be used as reference on how
> to implement it, even if we don't use an approved algorithm from
> it. And I really think we should look at that document when
> implementing it.
> 
> There should probably also be an option to use an RNG that
> conforms to it.

I am strongly in favour of this approach. We should be led by standards.

> 
>>> Randomness should be whitened.
>>
>> Whitening at the input is neither difficult nor necessary nor sufficient.
>> The hard part is obtaining a reliable lower bound on the amount of
>> useful randomness in the bit-blob when it appears at the input.  Where
>> did the bits come from?  Where did the bound come from?  Do you trust
>> the generic openssl user, who knows nothing about cryptology, to provide
>> either one?
> 
> I think it should by default be provided by the OS, and I don't
> think any OS is documenting how much randomness it can provide.
> 

I also agree that, by default, using the OS provided source makes a lot
of sense.

Matt
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Matt Caswell]

On 26/06/17 19:57, Salz, Rich via openssl-dev wrote:
> I was asked off-list why we're doing this.  A reasonable question. :)
> 
> There are many complains about the OpenSSL RNG.  For started:
>   https://github.com/openssl/openssl/issues/2168
>   https://github.com/openssl/openssl/issues/898
>   https://github.com/openssl/openssl/issues/2457
>   https://github.com/openssl/openssl/issues/3125
> 
> Also, there's things like this:
>   It uses MD5

Correction: it uses SHA1.

Matt
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[John Denker via openssl-dev]

In the context of

>> What's your threat model?

>> Are you designing to resist an output-text-only attack?  Or do you also want
>> "some" ability to recover from a compromise of the PRNG internal state?

On 06/26/2017 11:51 AM, Salz, Rich wrote:

> Our TCB border is the process.

That doesn't answer the question.

> then something like chacha.

That doesn't answer the question either.



I'm not mentioning any names, but some people are *unduly*
worried about recovery following compromise of the PRNG
internal state, so they constantly re-seed the PRNG, to
the point where it becomes a denial-of-service attack
against the upstream source of randomness.

This is also mostly pointless, because any attack that
compromises the PRNG state will likely compromise so many
other things that recovery will be very difficult.  All
future outputs will be suspect.

So please let's not go overboard in that direction.

On the other hand, it seems reasonable to insist on /forward/
secrecy.  That is, we should insist that /previous/ outputs
should not be compromised.  This is achievable at small but
not-quite-zero cost.

Specifically, ChaCha (or any other cipher) in counter mode
does *not* provide forward secrecy when the PRNG state is
compromised.
-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Paul Dale]

It looks like there are two different questions being addressed:

* How should OpenSSL architect the random number generators?

* How should these be seeded?

I agree with John that the larger threat comes from poor seeding, but both have 
to be considered.  A weak architecture will be just as bad from a security 
standpoint.


The pull request referenced was about the former.  It seems like the consensus 
is leaning toward cascaded RNGs with reseeding.  There are open questions about 
which algorithm(s) to use, where to place the RNG nodes in the hierarchy and 
integration with threading.

Given the variety of RNGs available, would an EVP RNG interface make sense?  
With a safe default in place (and no weak generators provided), the decision 
can be left to the user.
A side benefit is that the unit tests could implement a simple fully 
deterministic generator and the code for producing known sequences removed from 
the code base.


There seems to be more discussion about the seeding.

The quality of the seed sources is critical.  The problem is how to quantify 
this.  Determining a true lower bound for an entropy source is a difficult 
problem.  Turbid does this, I'm not aware of any other daemons that do and 
precious few HRNGs that attempt it (let alone prove it).

Estimates of the minimum entropy are available in NIST's SP800-90B draft.  
These aren't ideal and can easily be tricked but they do represent a reputable 
starting point.  As I mentioned, they appear to be well designed with sound 
statistics backing them.

Defence in depth seems prudent: independent sources with agglomeration and 
whitening.


We shouldn't trust the user to provide entropy.  I've seen what is typically 
provided.  Uninitialised buffers aren't random.  User inputs (mouse and 
keyboard) likewise aren't very good.  That both are still being suggested is 
frustrating.  I've seen worse suggestions, some to the effect that "time(NULL) 
^ getpid()" is too good and just time() is enough.


As for specific questions and comments:

John Denker wrote:
> If you trust the ambient OS to provide a seed, why not
> trust it for everything, and not bother to implement an
> openssl-specfic RNG at all?

I can think of a few possibilities:

* Diversifying the sources provides resistance to compromise of individual 
sources.  Although a full kernel compromise is unrecoverable, a kernel bug that 
leaked the internal pools in a read only manner isn't unforeseeable.

* Not all operating systems have good RNGs.

* Draining the kernel's entropy pools is unfriendly behaviour, other processes 
will typically want some randomness too.

* At boot time the kernel pools are empty (low or no quality).  This compounds 
when several things require seeding.

* Performance is also a consideration, although with a gradual collection 
strategy this should be less of a concern.  Except at start up.


John Denker wrote:
> Are you designing to resist an output-text-only attack?  Or do you
> also want "some" ability to recover from a compromise of
> the PRNG internal state?

Yes to both ideally.  Feeding additional seed material into a PRNG could help 
in the latter case.  It depends on how easy it is to compromise the PRNG state. 
 If it is trivial in terms of resources, it isn't going to be recoverable.  A 
short reseed interval can be effective against a slow attack (but not always).


John Denker wrote:
> Is there state in a persistent file, or only in memory?

Rich Salz replied:
> Only memory.

I'd propose secure memory.  Leaking RNG state is very bad.


John Denker wrote:
>> Do you think we need to use multiple sources of randomness?
> Quality is more important than quantity.

Given the difficulty of quantifying the quality of sources, I think that having 
multiple sources is prudent.  Finding out that a source is low quality when it 
was believed to be good is less embarrassing when you've other sources in 
operation.  Likewise, if a source fails.

I agree that one good source trumps a pile of low quality sources.  The aim is 
to have several good sources.

A proven source is better than one that is believed to be good.  These can fail 
(kill -STOP anyone) and a fall-back is not necessarily bad.


John Denker wrote:
> Cascading is absolutely necessary, and must be done "just so", to
> prevent track-and-hold attacks.  One of the weaknesses in the
> Enigma, exploited to the hilt by Bletchley Park, was that each
> change in the internal state was too small.  A large state space
> is not sufficient if the state /changes/ are small.

My suggestion was to XOR the entire state space with new random bits.  Some 
RNGs will have better reseeding procedures and they should be used.  I agree 
completely that small additions aren't useful.


Kurt Roeckx wrote:
> We currently have a ~/.rnd file. We should probably get rid of it.

Perhaps make it optional?  Embedded systems have trouble with random state at 
boot and a ~/.rnd file or equivalent is beneficial here.  I've 

Re: [openssl-dev] Work on a new RNG for OpenSSL

[John Denker via openssl-dev]

On 06/26/2017 12:41 PM, Salz, Rich wrote:

> We run in many environments, and I don't think it's reasonable to say
> that the RNG on someone's personal web server, perhaps on the
> Internet, is at the same level of criticality, say, as the same RNG
> running on something like a global CDN.  I am not trying to back out
> of our responsibilities here, but rather saying that I think a
> justifiable case can be made for accepting vague words like mediocre
> at times.

That argument cuts the other way, much more acutely.

When writing a low- to mid-level library such as openssl,
the problem is you *don't know* how it will be used.  If
you design a RNG that is good enough for a game of Go Fish,
it is entirely possible that some user will turn around and
use the same RNG to sign a multi-million dollar contract,
or encrypt some life-and-death critical messages.

The days when we could get away with mediocre security are
gone, and have been for quite a while now.

The idea of "provably correct" code has been around for decades
now.  I don't always succeed, but I try to write provably
correct code, even for things that are vastly less critical
than a cryptographic RNG.

In particular, the idea of combining several lousy upstream
sources and hoping for the best is 100% virgin serpentoleum.
It violates every engineering principle known to man, except
for Murphy's law.

The fact that RNGs are hard to test makes it easy to fool your
friends.  Your enemies will not be so easily fooled.  This
just makes it extra-super-important to insist on sound
engineering practices, top to bottom.

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Kurt Roeckx]

On Mon, Jun 26, 2017 at 01:18:58PM -0700, John Denker via openssl-dev wrote:
> On 06/26/2017 12:41 PM, Salz, Rich wrote:
> 
> > Suppose the chip supports RDRAND but the runtime doesn't have
> > getrandom or /dev/random?
> 
> That's an easy one!
> 
> Check the feature-test bit and then call RDRAND yourself.
> Code to do this exists, e.g.
>   
> https://en.wikipedia.org/wiki/RdRand#Sample_x86_asm_code_to_check_upon_RDRAND_instruction
> 
> A version of that for 64-bit architecture exists somewhere, too.

We already have code to detect it.


Kurt

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

Re: [openssl-dev] Work on a new RNG for OpenSSL

[Salz, Rich via openssl-dev]

> > Suppose the chip supports RDRAND but the runtime doesn't have
> > getrandom or /dev/random?
> 
> That's an easy one!
> 
> Check the feature-test bit and then call RDRAND yourself.
> Code to do this exists, e.g.

Yeah, that's kinda what I meant we'd do.


-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev