Re: [openssl-project] Entropy seeding the DRBG

[Dr Paul Dale]

Apologies for the name I’ve been sending under.  I don’t represent Oracle of 
course.
A temporary new MUA that isn’t quite doing what I expected.


Pauli

> On 8 May 2018, at 7:33 pm, Oracle  wrote:
> 
> Kurt wrote:
> 
>> The comment about not hashing it is if you want to use the tool to
>> do entropy estimation. Hashing it will not increase the entropy,
>> but the estimation will be totally wrong.
> 
> 
>> Passing the hashed data to the drbg as entropy input is fine if
>> you already know how much entropy that it contains.
> 
> 
> This is spot on.  Hash the data and it will appear to have eight bits per 
> byte of entropy regardless of the input.  The estimate output from NIST’s 
> suite will be around 7.8 bits per byte but that’s close enough.  The 
> standards refer to this as “whitening”.  It is fine to whiten the entropy 
> data before passing it to the DRBG but the entropy estimate must be based on 
> the pre-whitened data.
> 
> 
> Pauli
> 
> 
> 
> 
> ___
> openssl-project mailing list
> openssl-project@openssl.org
> https://mta.openssl.org/mailman/listinfo/openssl-project

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Oracle]

Kurt wrote:

> The comment about not hashing it is if you want to use the tool to
> do entropy estimation. Hashing it will not increase the entropy,
> but the estimation will be totally wrong.


> Passing the hashed data to the drbg as entropy input is fine if
> you already know how much entropy that it contains.


This is spot on.  Hash the data and it will appear to have eight bits per byte 
of entropy regardless of the input.  The estimate output from NIST’s suite will 
be around 7.8 bits per byte but that’s close enough.  The standards refer to 
this as “whitening”.  It is fine to whiten the entropy data before passing it 
to the DRBG but the entropy estimate must be based on the pre-whitened data.


Pauli




___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message  on Tue, 8 May 2018 
12:26:59 -0400, Oracle  said:

paul.dale> I can conform that it is measured in bits per sample size
paul.dale> (in this case bytes).  The estimate is very low and this is
paul.dale> not a great source.

Note that this is on a fairly inactive machine, and it's not *one*
source, but rather the concatenation of diverse counters all at once
(700+ bytes worth of data each time).  Also, I've had other
suggestions from the folks on comp.os.vms that I'm gonna try as well
as time allows.

paul.dale> We can explore other options and I should be able to spare
paul.dale> some time over ICMC to assist.  I’m not well versed in VMS
paul.dale> though.

Unfortunately, I'm not present there...  But I would see no problem
having a conversation directly with you, by email or by video link.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Oracle]

I can conform that it is measured in bits per sample size (in this case bytes). 
 The estimate is very low and this is not a great source.

We can explore other options and I should be able to spare some time over ICMC 
to assist.  I’m not well versed in VMS though.


Pauli

> On 30 Apr 2018, at 12:00 pm, Richard Levitte  wrote:
> 
> In message <20180430.164908.1424770216194967097.levi...@openssl.org> on Mon, 
> 30 Apr 2018 16:49:08 +0200 (CEST), Richard Levitte  said:
> 
> levitte> In message <20180430.152609.587396153749337701.levi...@openssl.org> 
> on Mon, 30 Apr 2018 15:26:09 +0200 (CEST), Richard Levitte 
>  said:
> levitte> 
> levitte> levitte> In message <20180430131000.ga25...@roeckx.be> on Mon, 30 
> Apr 2018 15:10:01 +0200, Kurt Roeckx  said:
> levitte> levitte> 
> levitte> levitte> kurt> The comment about not hashing it is if you want to 
> use the tool to
> levitte> levitte> kurt> do entropy estimation. Hashing it will not increase 
> the entropy,
> levitte> levitte> kurt> but the estimation will be totally wrong.
> levitte> levitte> kurt> 
> levitte> levitte> kurt> Passing the hashed data to the drbg as entropy input 
> is fine if
> levitte> levitte> kurt> you already know how much entropy that it contains.
> levitte> levitte> 
> levitte> levitte> Thanks, that's what I suspected.  Ok, on to the next step
> levitte> 
> levitte> Not done running, but does show some promise...
> levitte> 
> levitte> : ; ./a.out 
> ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin  8 -v
> levitte> Opening file: 
> ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin
> levitte> 
> levitte> Running non-IID tests...
> levitte> 
> levitte> Entropic statistic estimates:
> levitte> Most Common Value Estimate = 0.975224
> levitte> Collision Test Estimate = 0.902997
> levitte> Markov Test Estimate = 0.410808
> levitte> Compression Test Estimate = 0.811274
> levitte> 
> levitte> I assume that estimate is per "word" (i.e. per 8 bits of data in this
> levitte> case).
> 
> Ok, done running...  suffice to say, the first tests left me ever so
> hopeful...
> 
>: ; ./a.out 
> ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin  8 -v
>Opening file: 
> ../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin
> 
>Running non-IID tests...
> 
>Entropic statistic estimates:
>Most Common Value Estimate = 0.975224
>Collision Test Estimate = 0.902997
>Markov Test Estimate = 0.410808
>Compression Test Estimate = 0.811274
>t-Tuple Test Estimate = 0.0818796
>Longest Reapeated Substring Test Estimate = 0.0818772
> 
>Predictor estimates:
>Multi Most Common in Window (MultiMCW) Test: 100% complete
>   Correct: 507351
>   P_avg (global): 0.508671
>   P_run (local): 0.587891
>Multi Most Common in Window (Multi MCW) Test = 0.76638
>Lag Test: 100% complete
>   Correct: 269907
>   P_avg (global): 0.271051
>   P_run (local): 0.347168
>Lag Prediction Test = 1.52629
>MultiMMC Test: 100% complete
>   Correct: 11700
>   P_avg (global): 0.011977
>   P_run (local): 0.444824
>Multi Markov Model with Counting (MultiMMC) Prediction Test = 1.16869
>LZ78Y Test: 99% complete
>   Correct: 572107
>   P_avg (global): 0.573391
>   P_run (local): 0.615723
>LZ78Y Prediction Test = 0.699647
>Min Entropy: 0.0818772
> 
> So I'd like to have it confirmed that I'm reading this right, that's
> about 0.08 entropy bits per 8 data bits?  Or is it per data bit?
> Depending on the interpretation, we either have 1 bit of entropy per
> 12 data bits...  or per 100 data bits...  The latter has my heart
> sinking...
> 
> -- 
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/
> ___
> openssl-project mailing list
> openssl-project@openssl.org
> https://mta.openssl.org/mailman/listinfo/openssl-project

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Salz, Rich]

We have not committed to being FIPS/NIST capable with our RNG this release.  We 
have committed to other things, and we seem to be falling behind on those.

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180501084317.ga32...@roeckx.be> on Tue, 1 May 2018 10:43:17 
+0200, Kurt Roeckx  said:

kurt> If you actually follow SP800-90B, you should make a theoretical
kurt> model of how much entropy you expect, and then use the tool
kurt> to verify that your model is correct.

E...  look, I'm kind of a rookie in this particular area, so errr,
I'm not sure I have the knowledge to think of a theoretical model.
Given a crash course, I can probably come up with *something*, but at
this moment, I don't know where to start.

A side note to this discussion, the way the rand pool routines are
currently implemented, specifically rand_pool_bytes_needed(), we
cannot handle a source with less than 1 entropy bit per 8 bits of
data.  Or well, it can, if that particular routine isn't used, but
considering it's a fairly crucial routine for entropy acquisition, I'd
say it needs a small change.  PR coming up.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Tue, May 01, 2018 at 06:09:13AM +0200, Richard Levitte wrote:
> In message <20180430162209.ga4...@roeckx.be> on Mon, 30 Apr 2018 18:22:09 
> +0200, Kurt Roeckx  said:
> 
> kurt> On Mon, Apr 30, 2018 at 06:00:20PM +0200, Richard Levitte wrote:
> kurt> > 
> kurt> > So I'd like to have it confirmed that I'm reading this right, that's
> kurt> > about 0.08 entropy bits per 8 data bits?  Or is it per data bit?
> kurt> 
> kurt> Per symbol, being 8 bits for what you provided.
> kurt> 
> kurt> > Depending on the interpretation, we either have 1 bit of entropy per
> kurt> > 12 data bits...  or per 100 data bits...  The latter has my heart
> kurt> > sinking...
> kurt> 
> kurt> It's per 100 bits, and that's really still an overestimate. One
> kurt> of the models they used was able to predict it that well.
> 
> That well?  I'm not sure I understand, the final min-entropy value is
> the *lowest* of all different estimates.  Also, I'm not sure what
> makes you say it's an overestimate...  are you simply speculating?

Those are all just tests to see how easy it is to predict the
next value, but that really don't know anything about the data. It
might be possible to generate a better predictor, one that has an
even lower min-entropy value. That is why you should not rely on
the tool to give you a good min-entropy value, it just shows that
the maximum of the real value is the minimum reported by the tool.

If you actually follow SP800-90B, you should make a theoretical
model of how much entropy you expect, and then use the tool
to verify that your model is correct.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180501.060913.811991315461935857.levi...@openssl.org> on Tue, 01 
May 2018 06:09:13 +0200 (CEST), Richard Levitte  said:

levitte> Either way, this is quite discouraging, because this means that with
levitte> that estimate, I need to gather about 25 KiB of data to meet the
levitte> requirements of our DRBG.  Right?

Gah!  Too early in the morning to keep bits and bytes apart!  So, err,
about 3 KiB plus change...  Still not the most encouraging thought...

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180430162209.ga4...@roeckx.be> on Mon, 30 Apr 2018 18:22:09 
+0200, Kurt Roeckx  said:

kurt> On Mon, Apr 30, 2018 at 06:00:20PM +0200, Richard Levitte wrote:
kurt> > 
kurt> > So I'd like to have it confirmed that I'm reading this right, that's
kurt> > about 0.08 entropy bits per 8 data bits?  Or is it per data bit?
kurt> 
kurt> Per symbol, being 8 bits for what you provided.
kurt> 
kurt> > Depending on the interpretation, we either have 1 bit of entropy per
kurt> > 12 data bits...  or per 100 data bits...  The latter has my heart
kurt> > sinking...
kurt> 
kurt> It's per 100 bits, and that's really still an overestimate. One
kurt> of the models they used was able to predict it that well.

That well?  I'm not sure I understand, the final min-entropy value is
the *lowest* of all different estimates.  Also, I'm not sure what
makes you say it's an overestimate...  are you simply speculating?

Either way, this is quite discouraging, because this means that with
that estimate, I need to gather about 25 KiB of data to meet the
requirements of our DRBG.  Right?

kurt> It might be possible to create a better model.

I'm not sure I understand what you mean.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Mon, Apr 30, 2018 at 06:00:20PM +0200, Richard Levitte wrote:
> 
> So I'd like to have it confirmed that I'm reading this right, that's
> about 0.08 entropy bits per 8 data bits?  Or is it per data bit?

Per symbol, being 8 bits for what you provided.

> Depending on the interpretation, we either have 1 bit of entropy per
> 12 data bits...  or per 100 data bits...  The latter has my heart
> sinking...

It's per 100 bits, and that's really still an overestimate. One of
the models they used was able to predict it that well. It might be
possible to create a better model.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180430.164908.1424770216194967097.levi...@openssl.org> on Mon, 30 
Apr 2018 16:49:08 +0200 (CEST), Richard Levitte  said:

levitte> In message <20180430.152609.587396153749337701.levi...@openssl.org> on 
Mon, 30 Apr 2018 15:26:09 +0200 (CEST), Richard Levitte  
said:
levitte> 
levitte> levitte> In message <20180430131000.ga25...@roeckx.be> on Mon, 30 Apr 
2018 15:10:01 +0200, Kurt Roeckx  said:
levitte> levitte> 
levitte> levitte> kurt> The comment about not hashing it is if you want to use 
the tool to
levitte> levitte> kurt> do entropy estimation. Hashing it will not increase the 
entropy,
levitte> levitte> kurt> but the estimation will be totally wrong.
levitte> levitte> kurt> 
levitte> levitte> kurt> Passing the hashed data to the drbg as entropy input is 
fine if
levitte> levitte> kurt> you already know how much entropy that it contains.
levitte> levitte> 
levitte> levitte> Thanks, that's what I suspected.  Ok, on to the next step
levitte> 
levitte> Not done running, but does show some promise...
levitte> 
levitte> : ; ./a.out 
../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin  8 -v
levitte> Opening file: 
../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin
levitte> 
levitte> Running non-IID tests...
levitte> 
levitte> Entropic statistic estimates:
levitte> Most Common Value Estimate = 0.975224
levitte> Collision Test Estimate = 0.902997
levitte> Markov Test Estimate = 0.410808
levitte> Compression Test Estimate = 0.811274
levitte> 
levitte> I assume that estimate is per "word" (i.e. per 8 bits of data in this
levitte> case).

Ok, done running...  suffice to say, the first tests left me ever so
hopeful...

: ; ./a.out 
../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin  8 -v
Opening file: 
../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin

Running non-IID tests...

Entropic statistic estimates:
Most Common Value Estimate = 0.975224
Collision Test Estimate = 0.902997
Markov Test Estimate = 0.410808
Compression Test Estimate = 0.811274
t-Tuple Test Estimate = 0.0818796
Longest Reapeated Substring Test Estimate = 0.0818772

Predictor estimates:
Multi Most Common in Window (MultiMCW) Test: 100% complete
Correct: 507351
P_avg (global): 0.508671
P_run (local): 0.587891
Multi Most Common in Window (Multi MCW) Test = 0.76638
Lag Test: 100% complete
Correct: 269907
P_avg (global): 0.271051
P_run (local): 0.347168
Lag Prediction Test = 1.52629
MultiMMC Test: 100% complete
Correct: 11700
P_avg (global): 0.011977
P_run (local): 0.444824
Multi Markov Model with Counting (MultiMMC) Prediction Test = 1.16869
LZ78Y Test: 99% complete
Correct: 572107
P_avg (global): 0.573391
P_run (local): 0.615723
LZ78Y Prediction Test = 0.699647
Min Entropy: 0.0818772

So I'd like to have it confirmed that I'm reading this right, that's
about 0.08 entropy bits per 8 data bits?  Or is it per data bit?
Depending on the interpretation, we either have 1 bit of entropy per
12 data bits...  or per 100 data bits...  The latter has my heart
sinking...

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180430.152609.587396153749337701.levi...@openssl.org> on Mon, 30 
Apr 2018 15:26:09 +0200 (CEST), Richard Levitte  said:

levitte> In message <20180430131000.ga25...@roeckx.be> on Mon, 30 Apr 2018 
15:10:01 +0200, Kurt Roeckx  said:
levitte> 
levitte> kurt> The comment about not hashing it is if you want to use the tool 
to
levitte> kurt> do entropy estimation. Hashing it will not increase the entropy,
levitte> kurt> but the estimation will be totally wrong.
levitte> kurt> 
levitte> kurt> Passing the hashed data to the drbg as entropy input is fine if
levitte> kurt> you already know how much entropy that it contains.
levitte> 
levitte> Thanks, that's what I suspected.  Ok, on to the next step

Not done running, but does show some promise...

: ; ./a.out 
../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin  8 -v
Opening file: 
../../../levitte/vms-experiments/entropy-gathering/entropy-stats.bin

Running non-IID tests...

Entropic statistic estimates:
Most Common Value Estimate = 0.975224
Collision Test Estimate = 0.902997
Markov Test Estimate = 0.410808
Compression Test Estimate = 0.811274

I assume that estimate is per "word" (i.e. per 8 bits of data in this
case).

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180430131000.ga25...@roeckx.be> on Mon, 30 Apr 2018 15:10:01 
+0200, Kurt Roeckx  said:

kurt> The comment about not hashing it is if you want to use the tool to
kurt> do entropy estimation. Hashing it will not increase the entropy,
kurt> but the estimation will be totally wrong.
kurt> 
kurt> Passing the hashed data to the drbg as entropy input is fine if
kurt> you already know how much entropy that it contains.

Thanks, that's what I suspected.  Ok, on to the next step

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Mon, Apr 30, 2018 at 02:42:53PM +0200, Richard Levitte wrote:
> In message <20180424172439.ga8...@roeckx.be> on Tue, 24 Apr 2018 19:24:40 
> +0200, Kurt Roeckx  said:
> 
> kurt> On Tue, Apr 24, 2018 at 07:20:42AM +0200, Richard Levitte wrote:
> kurt> > Like I think I mentioned a few days ago, I'm currently on a 
> conference. I'll take this up in more depth later this week.
> kurt> > 
> kurt> > I have a question, though... Kurt said at some point that all that 
> was needed on the VMS side was to collect data, the rest can be done 
> elsewhere (thankfully). However, I don't really understand what the collected 
> data is supposed to be. Just the same stream of bytes that I would feed the 
> entropy acquisition, or something else? Is the time delta between samples a 
> factor in this?
> kurt> 
> kurt> The API support getting data that has 1 bit of entropy per 128 bit
> kurt> received (DRBG_MINMAX_FACTOR). If it's worse than that, you might
> kurt> have to write your own extract method.
> 
> I might have to either way, don't I.  A method I'm pondering is to
> pass all the data gathered (700-something bytes) through sha512 and
> add the result to the pool.  I have no idea what that says about the
> entropy of the original data, which is at somewhere between 0.1 and
> 0.2 entropy bits per data bit according the 3rd order entropy
> calculation that I replicated from the Linux /dev/urandom driver.
> 
> kurt> A stream of bytes it just fine.
> kurt> 
> kurt> I think the tme delta will really depend on your source. If it
> kurt> really changes all the time, it really doesn't matter much how
> kurt> fast you do it. But I think some (most?) of the variables don't
> kurt> change that often.
> 
> It doesn't change *all* the time, but with a 1-10 second sleep between
> data gatherings, there's always *something* that has changed enough
> to give a 3rd order diff from previous sampling that's > 0.
> 
> So what I've done for now is to make two files, one that's the raw
> data, repeatedly gathered every 1-10 seconds until I got about 1 Mib
> of data, the other being a concatenation of sha512 calculations of
> those same (*) data until I filled that file up to 1 Mib.  I suspect
> that the latter isn't quite valid, considering Paul said something
> about no transformation whatsoever, but I thought it would be worth a
> try.

The comment about not hashing it is if you want to use the tool to
do entropy estimation. Hashing it will not increase the entropy,
but the estimation will be totally wrong.

Passing the hashed data to the drbg as entropy input is fine if
you already know how much entropy that it contains.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Salz, Rich]

>pass all the data gathered (700-something bytes) through sha512 and
add the result to the pool.

I don't see a reason to hash it, just send it all 

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180424172439.ga8...@roeckx.be> on Tue, 24 Apr 2018 19:24:40 
+0200, Kurt Roeckx  said:

kurt> On Tue, Apr 24, 2018 at 07:20:42AM +0200, Richard Levitte wrote:
kurt> > Like I think I mentioned a few days ago, I'm currently on a conference. 
I'll take this up in more depth later this week.
kurt> > 
kurt> > I have a question, though... Kurt said at some point that all that was 
needed on the VMS side was to collect data, the rest can be done elsewhere 
(thankfully). However, I don't really understand what the collected data is 
supposed to be. Just the same stream of bytes that I would feed the entropy 
acquisition, or something else? Is the time delta between samples a factor in 
this?
kurt> 
kurt> The API support getting data that has 1 bit of entropy per 128 bit
kurt> received (DRBG_MINMAX_FACTOR). If it's worse than that, you might
kurt> have to write your own extract method.

I might have to either way, don't I.  A method I'm pondering is to
pass all the data gathered (700-something bytes) through sha512 and
add the result to the pool.  I have no idea what that says about the
entropy of the original data, which is at somewhere between 0.1 and
0.2 entropy bits per data bit according the 3rd order entropy
calculation that I replicated from the Linux /dev/urandom driver.

kurt> A stream of bytes it just fine.
kurt> 
kurt> I think the tme delta will really depend on your source. If it
kurt> really changes all the time, it really doesn't matter much how
kurt> fast you do it. But I think some (most?) of the variables don't
kurt> change that often.

It doesn't change *all* the time, but with a 1-10 second sleep between
data gatherings, there's always *something* that has changed enough
to give a 3rd order diff from previous sampling that's > 0.

So what I've done for now is to make two files, one that's the raw
data, repeatedly gathered every 1-10 seconds until I got about 1 Mib
of data, the other being a concatenation of sha512 calculations of
those same (*) data until I filled that file up to 1 Mib.  I suspect
that the latter isn't quite valid, considering Paul said something
about no transformation whatsoever, but I thought it would be worth a
try.

I'll have to try and feed this to the entropy test programs you
indicated earlier...

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Paul Dale]

A stream of bytes.  They don't need to contain eight bit data.

NIST's SP 800-90B wants 1,000,000 bytes to do the estimation (there are restart 
tests as well worry about them after good sources are determined).  The source 
shouldn't be manipulated with anything other than xor -- a wide source can be 
thinned by xoring the bytes together but not by shifting and xoring e.g.

I suspect you'll be sampling the sources periodically -- this is OS and 
workload dependent.  Even for a continuously changing good source, you'll want 
to sample slower to avoid impacting the CPU too much.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia

-Original Message-
From: Kurt Roeckx [mailto:k...@roeckx.be] 
Sent: Wednesday, 25 April 2018 3:25 AM
To: openssl-project@openssl.org
Subject: Re: [openssl-project] Entropy seeding the DRBG

On Tue, Apr 24, 2018 at 07:20:42AM +0200, Richard Levitte wrote:
> Like I think I mentioned a few days ago, I'm currently on a conference. I'll 
> take this up in more depth later this week.
> 
> I have a question, though... Kurt said at some point that all that was needed 
> on the VMS side was to collect data, the rest can be done elsewhere 
> (thankfully). However, I don't really understand what the collected data is 
> supposed to be. Just the same stream of bytes that I would feed the entropy 
> acquisition, or something else? Is the time delta between samples a factor in 
> this?

The API support getting data that has 1 bit of entropy per 128 bit received 
(DRBG_MINMAX_FACTOR). If it's worse than that, you might have to write your own 
extract method.

A stream of bytes it just fine.

I think the tme delta will really depend on your source. If it really changes 
all the time, it really doesn't matter much how fast you do it. But I think 
some (most?) of the variables don't change that often.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Tue, Apr 24, 2018 at 07:20:42AM +0200, Richard Levitte wrote:
> Like I think I mentioned a few days ago, I'm currently on a conference. I'll 
> take this up in more depth later this week.
> 
> I have a question, though... Kurt said at some point that all that was needed 
> on the VMS side was to collect data, the rest can be done elsewhere 
> (thankfully). However, I don't really understand what the collected data is 
> supposed to be. Just the same stream of bytes that I would feed the entropy 
> acquisition, or something else? Is the time delta between samples a factor in 
> this?

The API support getting data that has 1 bit of entropy per 128 bit
received (DRBG_MINMAX_FACTOR). If it's worse than that, you might
have to write your own extract method.

A stream of bytes it just fine.

I think the tme delta will really depend on your source. If it really
changes all the time, it really doesn't matter much how fast you
do it. But I think some (most?) of the variables don't change that
often.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

Like I think I mentioned a few days ago, I'm currently on a conference. I'll 
take this up in more depth later this week.

I have a question, though... Kurt said at some point that all that was needed 
on the VMS side was to collect data, the rest can be done elsewhere 
(thankfully). However, I don't really understand what the collected data is 
supposed to be. Just the same stream of bytes that I would feed the entropy 
acquisition, or something else? Is the time delta between samples a factor in 
this?

Cheers
Richard 

Paul Dale  skrev: (24 april 2018 00:31:39 CEST)
>I can possibly provide some input having done similar for a number of
>platforms and written faster but equivalent entropy assessment code to
>NIST's (for the second draft of SP 800-90B rather than the final
>version).
>
>I'm not knowledgeable about VMS though.
>
>We could discuss further at ICMC if you're in the area.
>
>
>Pauli

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Paul Dale]

I can possibly provide some input having done similar for a number of platforms 
and written faster but equivalent entropy assessment code to NIST's (for the 
second draft of SP 800-90B rather than the final version).

I'm not knowledgeable about VMS though.

We could discuss further at ICMC if you're in the area.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia

-Original Message-
From: Kurt Roeckx [mailto:k...@roeckx.be] 
Sent: Tuesday, 24 April 2018 5:46 AM
To: openssl-project@openssl.org
Subject: Re: [openssl-project] Entropy seeding the DRBG

On Sat, Apr 07, 2018 at 04:58:06PM +0200, Richard Levitte wrote:
> In the mean time, I've spent a few days going through the docs on all 
> kinds of data that you can get out from the VMS kernel, most notably 
> through a system service called sys$getrmi()...  there's a gazillion 
> data points, a treasure trove for anyone that's into statistics.  And 
> I intend to spend some time trying to estimate what kind of entropy I 
> can get out of them...
> 
> ... and that's where Kurt came in:
> 
> > Can I suggest you try something like 
> > https://github.com/usnistgov/SP800-90B_EntropyAssessment to at least 
> > get an idea? You would need to sample 1 variable and feed that into 
> > it.
> 
> And yeah, sure, especially if all it takes is to produce a stream of 
> bits from a source and feed that to the assessment program.  As long 
> as I don't have to port a C++11 program to VMS, 'cause unfortunately, 
> the existing C++ compiler hasn't had a real update for quite a while 
> :-/ (I'm sure that VSI is quite busy updating all they can, but they 
> haven't let anything out yet)
> 
> But either way, creating a better entropy gatherer is the longer term 
> goal.  I hope I get that part done before the release.

Any update on this?


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Sat, Apr 07, 2018 at 04:58:06PM +0200, Richard Levitte wrote:
> In the mean time, I've spent a few days going through the docs on all
> kinds of data that you can get out from the VMS kernel, most notably
> through a system service called sys$getrmi()...  there's a gazillion
> data points, a treasure trove for anyone that's into statistics.  And
> I intend to spend some time trying to estimate what kind of entropy I
> can get out of them...
> 
> ... and that's where Kurt came in:
> 
> > Can I suggest you try something like
> > https://github.com/usnistgov/SP800-90B_EntropyAssessment to at least
> > get an idea? You would need to sample 1 variable and feed that into
> > it.
> 
> And yeah, sure, especially if all it takes is to produce a stream of
> bits from a source and feed that to the assessment program.  As long
> as I don't have to port a C++11 program to VMS, 'cause unfortunately,
> the existing C++ compiler hasn't had a real update for quite a while
> :-/ (I'm sure that VSI is quite busy updating all they can, but they
> haven't let anything out yet)
> 
> But either way, creating a better entropy gatherer is the longer term
> goal.  I hope I get that part done before the release.

Any update on this?


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Sat, Apr 07, 2018 at 07:00:21PM +0200, Richard Levitte wrote:
> kurt> I wonder if it's useful to have a thread of VMS that collects
> kurt> such bits all the time, like the kernel is doing.
> 
> I was pondering something like that, and it does make sense.  That, or
> creating a generic device driver (RND0:) that works a bit like the
> random driver on Linux, or perhaps the one from OpenBSD...

So one problem with OpenSSL doing this is that it's probably going
to take a while (in the order of seconds to minutes) before it's
ready, and I think you want to avoid that each time an application
using openssl starts. So a system service would be much better.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180407174527.gc20...@roeckx.be> on Sat, 7 Apr 2018 19:45:28 
+0200, Kurt Roeckx  said:

kurt> On Sat, Apr 07, 2018 at 07:00:21PM +0200, Richard Levitte wrote:
kurt> > In message <20180407160031.gb12...@roeckx.be> on Sat, 7 Apr 2018 
18:00:32 +0200, Kurt Roeckx  said:
kurt> > 
kurt> > kurt> If you have such a program that collects the bits, I would like
kurt> > kurt> to review it. I would also like to test something like that over
kurt> > kurt> a range of machines it's expected to run on.
kurt> > 
kurt> > Errr  I have no idea what you're talking about.  I know of no
kurt> > other operating system that provides the system services VMS does, so
kurt> > I wonder how you expect to run the program gathering those data on
kurt> > anything other than VMS.
kurt> 
kurt> I mean multiple VMS machines on different processors / hardware.

VMS exists for Alpha and Itanium for the moment.  VSI is working on a
x86_64 port, with the first early adopters kit available late this
year (see http://www.vmssoftware.com/pdfs/VSI_Roadmap_20171215.pdf)

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Sat, Apr 07, 2018 at 07:00:21PM +0200, Richard Levitte wrote:
> In message <20180407160031.gb12...@roeckx.be> on Sat, 7 Apr 2018 18:00:32 
> +0200, Kurt Roeckx  said:
> 
> kurt> On Sat, Apr 07, 2018 at 04:58:06PM +0200, Richard Levitte wrote:
> kurt> > > Can I suggest you try something like
> kurt> > > https://github.com/usnistgov/SP800-90B_EntropyAssessment to at least
> kurt> > > get an idea? You would need to sample 1 variable and feed that into
> kurt> > > it.
> kurt> > 
> kurt> > And yeah, sure, especially if all it takes is to produce a stream of
> kurt> > bits from a source and feed that to the assessment program.  As long
> kurt> > as I don't have to port a C++11 program to VMS, 'cause unfortunately,
> kurt> > the existing C++ compiler hasn't had a real update for quite a while
> kurt> > :-/ (I'm sure that VSI is quite busy updating all they can, but they
> kurt> > haven't let anything out yet)
> kurt> 
> kurt> You only need to generate the bits on VMS, you can run the tool on
> kurt> some other machine.
> 
> Cool.
> 
> kurt> If you have such a program that collects the bits, I would like
> kurt> to review it. I would also like to test something like that over
> kurt> a range of machines it's expected to run on.
> 
> Errr  I have no idea what you're talking about.  I know of no
> other operating system that provides the system services VMS does, so
> I wonder how you expect to run the program gathering those data on
> anything other than VMS.

I mean multiple VMS machines on different processors / hardware.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <20180407160031.gb12...@roeckx.be> on Sat, 7 Apr 2018 18:00:32 
+0200, Kurt Roeckx  said:

kurt> On Sat, Apr 07, 2018 at 04:58:06PM +0200, Richard Levitte wrote:
kurt> > > Can I suggest you try something like
kurt> > > https://github.com/usnistgov/SP800-90B_EntropyAssessment to at least
kurt> > > get an idea? You would need to sample 1 variable and feed that into
kurt> > > it.
kurt> > 
kurt> > And yeah, sure, especially if all it takes is to produce a stream of
kurt> > bits from a source and feed that to the assessment program.  As long
kurt> > as I don't have to port a C++11 program to VMS, 'cause unfortunately,
kurt> > the existing C++ compiler hasn't had a real update for quite a while
kurt> > :-/ (I'm sure that VSI is quite busy updating all they can, but they
kurt> > haven't let anything out yet)
kurt> 
kurt> You only need to generate the bits on VMS, you can run the tool on
kurt> some other machine.

Cool.

kurt> If you have such a program that collects the bits, I would like
kurt> to review it. I would also like to test something like that over
kurt> a range of machines it's expected to run on.

Errr  I have no idea what you're talking about.  I know of no
other operating system that provides the system services VMS does, so
I wonder how you expect to run the program gathering those data on
anything other than VMS.

If you wanna know what I'm talking about, just have a look at this
page:

http://h41379.www4.hpe.com/doc/84final/4527/4527pro_contents.html

Every "command" that starts with a '$' is a system service.  The C API
has them prefixed with 'sys$'.  The one of hightes interest seems to
be $GETRMI (or sys$getrmi), which has all sorts of counters and other
data.

kurt> I wonder if it's useful to have a thread of VMS that collects
kurt> such bits all the time, like the kernel is doing.

I was pondering something like that, and it does make sense.  That, or
creating a generic device driver (RND0:) that works a bit like the
random driver on Linux, or perhaps the one from OpenBSD...

kurt> I'm going to guess that the 4 bit you count now is an overestimate.

Don't look at me, it was I who made that estimate...  but I agree with
your guess.

kurt> And I would like to be very conservative in estimating something
kurt> like that, and even divide what the tool returns by 10.

That's a reason I want to go for a computed estimate instead...  the
3rd order delta that Linux' random driver does with jiffies seemed
like a simple enough strategy.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Sat, Apr 07, 2018 at 04:58:06PM +0200, Richard Levitte wrote:
> > Can I suggest you try something like
> > https://github.com/usnistgov/SP800-90B_EntropyAssessment to at least
> > get an idea? You would need to sample 1 variable and feed that into
> > it.
> 
> And yeah, sure, especially if all it takes is to produce a stream of
> bits from a source and feed that to the assessment program.  As long
> as I don't have to port a C++11 program to VMS, 'cause unfortunately,
> the existing C++ compiler hasn't had a real update for quite a while
> :-/ (I'm sure that VSI is quite busy updating all they can, but they
> haven't let anything out yet)

You only need to generate the bits on VMS, you can run the tool on
some other machine.

If you have such a program that collects the bits, I would like to
review it. I would also like to test something like that over a
range of machines it's expected to run on.

I wonder if it's useful to have a thread of VMS that collects such
bits all the time, like the kernel is doing.

I'm going to guess that the 4 bit you count now is an overestimate.
And I would like to be very conservative in estimating something
like that, and even divide what the tool returns by 10.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <8c39cdf4-a91e-4dfb-be67-6799e07d3...@akamai.com> on Tue, 3 Apr 2018 
16:58:17 +, "Salz, Rich"  said:

rsalz> >Please note that that 50% extra is only used for
rsalz> >instantiating the DRBG. On reseed we it only uses 256
rsalz> >bits.

Instantiating is exactly the problem.  The VMS
rand_pool_acquire_entropy() currently generates 256 bits of entropy on
each call.  No more, no less.  And that's at an estimated 4 bits of
entropy per byte, and estimation that's from long ago.  Either way,
because instantiation demands more than 256 bits, the whole RNG breaks
down, and everything related to it in some way along with it.  In
other words, OpenSSL on VMS dies.

rsalz> True.  And now we're finding that VMS won't work.  And I bet
rsalz> there are other systems that will also find this amount
rsalz> excessive.

I'm thinking that for any platform that can support that, I don't see
a problem, at all.

So the current short term solution for this is to simply default to
AES-128-CTR instead of AES-256-CTR, specifically on VMS, which is
currently sitting in PR#5904.  It seems like the option to make
everyone happy, and everyone ends up with a better randomness
implementation either way (compared to OpenSSL 1.1.0 and older).

In the mean time, I've spent a few days going through the docs on all
kinds of data that you can get out from the VMS kernel, most notably
through a system service called sys$getrmi()...  there's a gazillion
data points, a treasure trove for anyone that's into statistics.  And
I intend to spend some time trying to estimate what kind of entropy I
can get out of them...

... and that's where Kurt came in:

> Can I suggest you try something like
> https://github.com/usnistgov/SP800-90B_EntropyAssessment to at least
> get an idea? You would need to sample 1 variable and feed that into
> it.

And yeah, sure, especially if all it takes is to produce a stream of
bits from a source and feed that to the assessment program.  As long
as I don't have to port a C++11 program to VMS, 'cause unfortunately,
the existing C++ compiler hasn't had a real update for quite a while
:-/ (I'm sure that VSI is quite busy updating all they can, but they
haven't let anything out yet)

But either way, creating a better entropy gatherer is the longer term
goal.  I hope I get that part done before the release.

Cheers,
Richard

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Dr. Matthias St. Pierre]

The internal state of the CTR-DRBG consists of a key K and a counter V (see 
figure 11 on page 48, which is the page before table 3). This is the reason why 
it requires bits_of(K) + bits_of(V) = keysize + blocksize = 256 + 128 = 384 
bits to initialize the internal state of the DRBG. 

But the seedlen (in bits) *does not* in general coincide with the 
security_strength! This is only in the case when _no_ derivation function is 
used! See the aforementioned table 3 on page 49:

If a derivation function is not used:
Minimum entropy input length:   seedlen
(min _length = blocklen + keylen)   


But we are in the case when a derivation function is used. Here only 
'security_strength' (= 256) bits of entropy are required.

If a derivation function is used:  
Minimum entropy input length:   security_strength
(min _length)

These bits are 'stretched out' to seed_length (=384) bits by the "derivation 
function", which is essentially an aes-ctr bit stream. Or, to be more precise: 
if we provide a large buffer of low entropy input, then this low entropy input 
will be condensed, stirred, and spread out equally over the 384 bits of the 
initial state by the derivation function.

Matthias

> -Ursprüngliche Nachricht-
> Von: openssl-project  Im Auftrag von 
> Richard Levitte
> Gesendet: Mittwoch, 4. April 2018 15:09
> An: openssl-project@openssl.org
> Betreff: Re: [openssl-project] Entropy seeding the DRBG
> 
> In message <122b3c36-21ad-4904-a692-351ade567...@akamai.com> on Wed, 4 Apr 
> 2018 11:58:54 +, "Salz, Rich"
>  said:
> 
> rsalz> Is it expected that the number of bits of seed must equal the
> rsalz> number of bits in the key strength?
> 
> It is expected that the number of bits of entropy in the seed (the VMS
> function declares 4 bits of entropy per byte, considering the sources
> it uses) equals a requirement, and it seems that the requirement is to
> have the DRBG strength (which is measure in number of entropy bits)
> match the number of bits of the block cipher used to generate the
> randomness bits.  If I understand things correctly...  and that does
> seem to match what's specified in SP800-90A r1.  I suggest a quick
> study of table 3 in section 10 (Definitions for the CTR_DRBG), seen on
> page 58 in 
> https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf
> Very specifically, there's the row with the title "Seed length
> (seedlen = outlen + keylen)" that very clearly says 384 bits for
> AES-256.
> 
> "Seed length" itself is defined in section 8:
> 
> 8.6.4 Seed Length
> 
> The minimum length of the seed depends on the DRBG mechanism and the
> security strength required by the consuming application, but shall be
> at least the number of bits of entropy required. See the tables in
> Section 10.
> 
> rsalz> But at any rate, raising the seed size to 256 seems mildly
> rsalz> tolerable, although I would prefer to keep it at 128.  Raising
> rsalz> it to 384 is wrong.
> 
> Note that with a nonce, that'll be 192 bits, unless I'm thinking
> wrong...  But I agree with you, at least from a very practical point
> of view.
> 
> --
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/
> ___
> openssl-project mailing list
> openssl-project@openssl.org
> https://mta.openssl.org/mailman/listinfo/openssl-project


smime.p7s
Description: S/MIME cryptographic signature
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Salz, Rich]

>Note that with a nonce, that'll be 192 bits, unless I'm thinking
wrong...  But I agree with you, at least from a very practical point
of view.
  
I think using a nonce is needless.  Use a personalization string (I used the 
address of the new DRBG).


___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message <122b3c36-21ad-4904-a692-351ade567...@akamai.com> on Wed, 4 Apr 2018 
11:58:54 +, "Salz, Rich"  said:

rsalz> Is it expected that the number of bits of seed must equal the
rsalz> number of bits in the key strength?

It is expected that the number of bits of entropy in the seed (the VMS
function declares 4 bits of entropy per byte, considering the sources
it uses) equals a requirement, and it seems that the requirement is to
have the DRBG strength (which is measure in number of entropy bits)
match the number of bits of the block cipher used to generate the
randomness bits.  If I understand things correctly...  and that does
seem to match what's specified in SP800-90A r1.  I suggest a quick
study of table 3 in section 10 (Definitions for the CTR_DRBG), seen on
page 58 in 
https://nvlpubs.nist.gov/nistpubs/specialpublications/nist.sp.800-90ar1.pdf
Very specifically, there's the row with the title "Seed length
(seedlen = outlen + keylen)" that very clearly says 384 bits for
AES-256.

"Seed length" itself is defined in section 8:

8.6.4 Seed Length 

The minimum length of the seed depends on the DRBG mechanism and the
security strength required by the consuming application, but shall be
at least the number of bits of entropy required. See the tables in
Section 10.

rsalz> But at any rate, raising the seed size to 256 seems mildly
rsalz> tolerable, although I would prefer to keep it at 128.  Raising
rsalz> it to 384 is wrong.

Note that with a nonce, that'll be 192 bits, unless I'm thinking
wrong...  But I agree with you, at least from a very practical point
of view.

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Salz, Rich]

Is it expected that the number of bits of seed must equal the number of bits in 
the key strength?

But at any rate, raising the seed size to 256 seems mildly tolerable, although 
I would prefer to keep it at 128.  Raising it to 384 is wrong.



___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Dr. Matthias St. Pierre]

> -Ursprüngliche Nachricht-
> Von: openssl-project  Im Auftrag von 
> Salz, Rich
> Gesendet: Mittwoch, 4. April 2018 02:59
> An: openssl-project@openssl.org
> Betreff: Re: [openssl-project] Entropy seeding the DRBG
> 
> If you say that AES256 needs CSPRNG seeding with 256 bits, then why doesn't 
> RSA 2048 keygen need seed to be seeded with 2048
> bits?  I am not a cryptographer, but I do not agree with this argument
> algorithms with a security level of 256 bit in TLS (like AES-256-CTR),
> so it is necessary that the random generator provides this level of
> security.

Because it's not about the number of bits needed to store the key, but about 
the estimated amount of time to break it. According to 
https://csrc.nist.gov/csrc/media/projects/key-management/documents/transitions/transitioning_cryptoalgos_070209.pdf,
 the estimated security level of RSA 2048 is 112 bits:

> The security strength is measured in bits and is, basically, a measure of the 
> difficulty of
> discovering the key. The understood security strength for each algorithm is 
> listed in SP
> 800-57. For example, RSA using a key length of 1024 bits (i.e., 1024-bit RSA) 
> has a
> security strength of 80 bits, as does 2-key Triple DES, while 2048-bit RSA 
> and 3-key
> Triple DES have a security strength of 112 bits. See Table 2 in Part 1 of SP 
> 800-57 for
> further security strength information.


> But if it is true, an AES128-CTR DRBG is still sufficient for generating 
> keys.  For the same reason that it is sufficient for generating
> Ed4418 or RSA2048 keys.
> 
> >The use of the nonce is mandated by section 10.2.1.3.2 of Nist SP 
> > 800-90Ar1:
> 
> We are not going for FIPS validation here.  This might be a nice to have, but 
> it is *NOT* a requirement for this release.  Especially if it
> puts the seeding requirement beyond the reach of some of our supported 
> platforms.
> 

I don't object to reverting it. After all, it takes only 88 bit to measure the 
age of the universe in nanoseconds, so 256 bits security should be fairly 
enough for the moment, even without a nonce.

Matthias





smime.p7s
Description: S/MIME cryptographic signature
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Salz, Rich]

If you say that AES256 needs CSPRNG seeding with 256 bits, then why doesn't RSA 
2048 keygen need seed to be seeded with 2048 bits?  I am not a cryptographer, 
but I do not agree with this argument
algorithms with a security level of 256 bit in TLS (like AES-256-CTR),
so it is necessary that the random generator provides this level of
security.

But if it is true, an AES128-CTR DRBG is still sufficient for generating keys.  
For the same reason that it is sufficient for generating Ed4418 or RSA2048 keys.

>The use of the nonce is mandated by section 10.2.1.3.2 of Nist SP 
> 800-90Ar1:
  
We are not going for FIPS validation here.  This might be a nice to have, but 
it is *NOT* a requirement for this release.  Especially if it puts the seeding 
requirement beyond the reach of some of our supported platforms.



___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Dr. Matthias St. Pierre]

Since both pull requests mentioned by Richard were reviewed and approved
by me, I would to add a few remarks on those two pull requests:

Ad #5401:  Switch the DRBGs from AES-128-CTR to AES-256-CTR

> The requirement change from 128 to 256 happened with this commit:
>
> commit 32bda2b2e4900308cb025020d8c8692e1d3c2ba9
> Author: Kurt Roeckx 
> Date:   Sun Feb 18 19:16:13 2018 +0100
> 
> Switch the DRBGs from AES-128-CTR to AES-256-CTR
> 
> Reviewed-by: Dr. Matthias St. Pierre

> GH: #5401

At first I was reluctant whether this increase was really necessary,
because AFAIK a security level of 128 bit is still considered sufficient
for most use cases. But Kurt argumented that OpenSSL supports encryption
algorithms with a security level of 256 bit in TLS (like AES-256-CTR),
so it is necessary that the random generator provides this level of
security. This sounded reasonable and after checking that the Linux
Random Number Generator was seeded with 256 bits of entropy
, I followed his arguments.

My original intention was to have the security strength configurable
with an option like --with-rand-strength=[128,192,256]. One could even
have chosen different defaults for different platforms. But this
approach was in conflict with Kurts argument above, stating that TLS
can't support 256 bit encryption algorithms if the underlying random
generator does not support this security level. So my TODO was dropped
in the above commit, see
.

IMHO there is no alternative to commit
32bda2b2e4900308cb025020d8c8692e1d3c2ba9, unless we say we only support
128 bit encryption algorithms.

Ad #5503:  Make sure we use a nonce when a nonce is required

> And then there's this one, which did the added 50%:
>
> commit 2a70d65b99e1f2376be705d18bca88703b7e774a
> Author: Kurt Roeckx 
> AuthorDate: Sat Mar 3 23:19:03 2018 +0100
> Commit: Kurt Roeckx 
> CommitDate: Sun Apr 1 21:11:26 2018 +0200
> 
> Make sure we use a nonce when a nonce is required
> 
> If a nonce is required and the get_nonce callback is NULL,
request 50%
> more entropy following NIST SP800-90Ar1 section 9.1.
> 
> Reviewed-by: Dr. Matthias St. Pierre

> GH: #5503

This pull request started initially when Kurt noticed that the NIST
standard requires the use of a nonce during instantiation, when a
derivation function is used, and that we were not providing
get_nonce()/cleanup_nonce() callbacks ourselves 
.

The use of the nonce is mandated by section 10.2.1.3.2 of Nist SP 800-90Ar1:

>10.2.1.3.2 Instantiation When a Derivation Function is Used
>When instantiation is performed using this method, the entropy
input may or may not have full entropy; in either case, a nonce is required

In Section 8.6.7 verious methods to obtain a nonce are discussed, see
also :

>8.6.7 Nonce
>
>A nonce may be required in the construction of a seed during
instantiation in order to provide a security cushion to block certain
attacks. The nonce shall be either:
>  a. A value with at least (security_strength/2) bits of entropy, or
>  b. A value that is expected to repeat no more often than a
(security_strength/2)-bit random string would be expected to repeat.
>
>A nonce may be composed of one (or more) of the following
components (other components may also be appropriate):
>
>1. A random value that is generated anew for each nonce, using an
approved random bit generator.
>2. A timestamp of sufficient resolution (detail) so that it is
different each time it is used.
>3. A monotonically increasing sequence number, or
>4. A combination of a timestamp and a monotonically increasing
sequence number, such that the sequence number is reset when and only
when the timestamp changes. For example, a timestamp may show the date
but not the time of day, so a sequence number is appended that will not
repeat during a particular day.
>
> For case 1 above, the random value could be acquired from the same
source and at the same time as the entropy input. In this case, the seed
could be considered to be constructed from an “extra strong” entropy
input and the optional personalization string, where the entropy for the
entropy input is equal to or greater than (3/2 security_strength) bits.

I argued for using variant 4
,
while Kurt favoured variant 1
.
And he is still convinced that variant 1 is the best solution:

Am 03.04.2018 um 18:54 schrieb Kurt Roeckx:
> There is an alternative to that 50% extra, but it's 

Re: [openssl-project] Entropy seeding the DRBG

[Paul Dale]

Richard wrote:

> (I'm tempted to try this with /dev/random only on Unix...  do I remember it 
> right, that it blocks for a while after every 8 byte chunk on some Unixen?)

Many but not all /dev/random sources will block to fulfill the requested 
entropy if sufficient isn't in the gathering pool already.  (it's not quite 
that simple but that's the gist of it).  Some but not most  /dev/urandom 
sources will do the same.  Later kernels are better than older ones in general. 
 I'm not aware of a kernel that limits reads to 8 bytes at a time, although 
there usually is a single read size maximum.


Pauli
-- 
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption 
Phone +61 7 3031 7217
Oracle Australia

-Original Message-
From: Richard Levitte [mailto:levi...@openssl.org] 
Sent: Tuesday, 3 April 2018 11:29 PM
To: openssl-project@openssl.org
Subject: Re: [openssl-project] Entropy seeding the DRBG

In message  on Tue, 3 Apr 2018 
12:52:50 +, "Salz, Rich"  said:

rsalz> I had not realized that we just increased the "entropy"
rsalz> requirements by 50%, from 256 to 384. The original DRBG 
rsalz> submission that I did only required 128 bits.  I think that is 
rsalz> wrong, and I think the PR that did it (#5503) should be reverted.
rsalz> 
rsalz> I am concerned that we are trying to meet requirements that we 
rsalz> really don't have.  The original code was a huge improvement.
rsalz> 
rsalz> Requiring 384 bits of random seed is silly.  I think it is 
rsalz> ridiculous.  One way or another we HAVE to fix that before the 
rsalz> release.
rsalz> 
rsalz> Thoughts?

FYI, here's the magic number that lies behind this:

: ; git grep RAND_DRBG_STRENGTH
...
include/openssl/rand_drbg.h:# define RAND_DRBG_STRENGTH 256

The requirement change from 128 to 256 happened with this commit:

commit 32bda2b2e4900308cb025020d8c8692e1d3c2ba9
Author: Kurt Roeckx 
Date:   Sun Feb 18 19:16:13 2018 +0100

Switch the DRBGs from AES-128-CTR to AES-256-CTR

Reviewed-by: Dr. Matthias St. Pierre 
GH: #5401

And then there's this one, which did the added 50%:

commit 2a70d65b99e1f2376be705d18bca88703b7e774a
Author: Kurt Roeckx 
AuthorDate: Sat Mar 3 23:19:03 2018 +0100
Commit: Kurt Roeckx 
CommitDate: Sun Apr 1 21:11:26 2018 +0200

Make sure we use a nonce when a nonce is required

If a nonce is required and the get_nonce callback is NULL, request 50%
more entropy following NIST SP800-90Ar1 section 9.1.

Reviewed-by: Dr. Matthias St. Pierre 
GH: #5503

Each of them seen by itself make sense.  The combined result, though, leaves me 
wondering...

(I'm tempted to try this with /dev/random only on Unix...  do I remember it 
right, that it blocks for a while after every 8 byte chunk on some Unixen?)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Salz, Rich]

>Please note that that 50% extra is only used for instantiating the
DRBG. On reseed we it only uses 256 bits.
  
True.  And now we're finding that VMS won't work.  And I bet there are other 
systems that will also find this amount excessive.
  
>There is an alternative to that 50% extra, but it's not making
sense to me.
  
Shrug.
  
>The 1.1.0 version also used 256 bit.
  
The 1.1.0 code was pre-DRBG and was a piece of crap.  Using AES/DRBG is 
stronger, better, and for the normal case 128 bits is enough.



___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Kurt Roeckx]

On Tue, Apr 03, 2018 at 12:52:50PM +, Salz, Rich wrote:
> I had not realized that we just increased the “entropy” requirements by 50%, 
> from 256 to 384. The original DRBG submission that I did only required 128 
> bits.  I think that is wrong, and I think the PR that did it (#5503) should 
> be reverted.
> 
> I am concerned that we are trying to meet requirements that we really don’t 
> have.  The original code was a huge improvement.
> 
> Requiring 384 bits of random seed is silly.  I think it is ridiculous.  One 
> way or another we HAVE to fix that before the release.

Please note that that 50% extra is only used for instantiating the
DRBG. On reseed we it only uses 256 bits.

There is an alternative to that 50% extra, but it's not making
sense to me.

The 1.1.0 version also used 256 bit.


Kurt

___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project

Re: [openssl-project] Entropy seeding the DRBG

[Richard Levitte]

In message  on Tue, 3 Apr 2018 
12:52:50 +, "Salz, Rich"  said:

rsalz> I had not realized that we just increased the “entropy”
rsalz> requirements by 50%, from 256 to 384. The original DRBG
rsalz> submission that I did only required 128 bits.  I think that is
rsalz> wrong, and I think the PR that did it (#5503) should be
rsalz> reverted.
rsalz> 
rsalz> I am concerned that we are trying to meet requirements that we
rsalz> really don’t have.  The original code was a huge improvement.
rsalz> 
rsalz> Requiring 384 bits of random seed is silly.  I think it is
rsalz> ridiculous.  One way or another we HAVE to fix that before the
rsalz> release.
rsalz> 
rsalz> Thoughts?

FYI, here's the magic number that lies behind this:

: ; git grep RAND_DRBG_STRENGTH
...
include/openssl/rand_drbg.h:# define RAND_DRBG_STRENGTH 256

The requirement change from 128 to 256 happened with this commit:

commit 32bda2b2e4900308cb025020d8c8692e1d3c2ba9
Author: Kurt Roeckx 
Date:   Sun Feb 18 19:16:13 2018 +0100

Switch the DRBGs from AES-128-CTR to AES-256-CTR

Reviewed-by: Dr. Matthias St. Pierre 
GH: #5401

And then there's this one, which did the added 50%:

commit 2a70d65b99e1f2376be705d18bca88703b7e774a
Author: Kurt Roeckx 
AuthorDate: Sat Mar 3 23:19:03 2018 +0100
Commit: Kurt Roeckx 
CommitDate: Sun Apr 1 21:11:26 2018 +0200

Make sure we use a nonce when a nonce is required

If a nonce is required and the get_nonce callback is NULL, request 50%
more entropy following NIST SP800-90Ar1 section 9.1.

Reviewed-by: Dr. Matthias St. Pierre 
GH: #5503

Each of them seen by itself make sense.  The combined result, though,
leaves me wondering...

(I'm tempted to try this with /dev/random only on Unix...  do I
remember it right, that it blocks for a while after every 8 byte chunk
on some Unixen?)

-- 
Richard Levitte levi...@openssl.org
OpenSSL Project http://www.openssl.org/~levitte/
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project