Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

Steffen Nurpmeso wrote in <20190107183136.-ew61%stef...@sdaoden.eu>:
  ...
 |  ...
 ||RAND_bytes() has always documented that it can fail. Most function
 ...
 |So, to me.., i do not see any possible error condition, since the
 |initial seeding has been testified with RAND_status().
 |
 |This is different now, and i will change the implementation as
 |soon as possible.  (This week.)

So i did, we disable the OpenSSL reseeding by directly calling
RAND_DRBG_set_reseed_defaults() after calling OPENSSL_init_ssl(),
which i hope will always be possible.
Be warned that i gave credit to both of you.

I have seen DRBG offers a lot of possibilities to control what
OpenSSL does, also regarding the fork handlers and all that.
Thanks for these possibilities, it is a terribly huge interface,
but it allows users to have control on what happens, instead of
sitting on an intransparent black box!  Getting something going on
such a thing causes grief, as it is hacky and otherwise
troublesome!

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

Jakob Bohm via openssl-users wrote in <07f4dea3-1a62-0c8c-76a4-cbe56abc8\
7...@wisemo.com>:
 |On 07/01/2019 22:31, Steffen Nurpmeso wrote:
 |> Good evening.
 |> 
 |> Jakob Bohm via openssl-users wrote in <95bceb59-b299-015a-f9c2-e2487a699\
 |> 8...@wisemo.com>:
 |>|Small corrections below:
 |>| ...
 |
 |Note that I do not represent the project at all, I am just another user 
 |trying to help you.

Surprise!  Indeed i do see your name only once in the commits!
So now, what to do?  What is due?  Congratulations, or even an
excuse?  I had a very false impression indeed.  Excuse this miss
of mine, please.

 |As such, I cannot really respond to your criticism of other project 
 |aspects, some of which I myself don't like either, while others are 
 |obvious to me as someone who has more or less followed along since this 
 |was a two-man Australian project.

Then you lead me by more than a decade in respect to "following"
the project.  (I only saw, otherwise offline, the headers and
libraries fly by.)  (And in my granted lengthy text i thought
about changing my ARC4 thing to ChaCha, and not feed the bytes of
the pool as such, but run a Blake2 digest on top of it, and feed
the output of that.  For now, however, this ARC4 PRNG, and then
also well seeded, is really good enough for my purpose(s).)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Jakob Bohm via openssl-users]

On 07/01/2019 22:31, Steffen Nurpmeso wrote:
> Good evening.
> 
> Jakob Bohm via openssl-users wrote in <95bceb59-b299-015a-f9c2-e2487a699\
> 8...@wisemo.com>:
>   |Small corrections below:
>   | ...

Note that I do not represent the project at all, I am just another user 
trying to help you.

As such, I cannot really respond to your criticism of other project 
aspects, some of which I myself don't like either, while others are 
obvious to me as someone who has more or less followed along since this 
was a two-man Australian project.

Enjoy

Jakob
-- 
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded 
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

Good evening.

Jakob Bohm via openssl-users wrote in <95bceb59-b299-015a-f9c2-e2487a699\
8...@wisemo.com>:
 |Small corrections below:
 |
 |On 07/01/2019 19:31, Steffen Nurpmeso wrote:
 |
 ||...
 |||RAND_load_file() and all this stuff. Just make sure the os entropy \
 |||sources
 |||are available and then simply use RAND_bytes(). But don't expect it to
 |||succeed always.
 |
 ||And may i ask, what to do if it fails?  Try it in a loop a finite
 ||number of times, assuming it over-and-over detects it is in an
 ||error state and tries to reseed unless that succeeds, and bail if
 ||still without success thereafter?
 |
 ||Talking about the only program in the public that i represent:
 ||i will not abort that one just because a PRNG fails to produce
 ||some bytes that this program uses for the purpose of, well, having
 ||some (pseudo but) random bytes; because: this is _completely_
 ||disproportionate to the use cases of those random numbers.
 ||I guess the same could be said about UUIDS generated via PRNGs
 ||instead of how the standard defines them, etc. etc.
 ||Please do not say "use random(3) or rand(3)" now, please.
 ||So what to do?
 |
 |"Just failing" is precisely to allow you (and others) to do something 
 |sensible without the random bytes, such as using something less secure, 
 |or doing a polite error reporting to the user.

I have to say i struggle with the OpenSSL project release
announcements.  Really.  I see a short mail which simply points
users to some (granted super duper) web site (with custom fonts
and such -- it must look fantastic on such a tablet thing that
i do not own!), and on these pages, which i, unfortunately and
until now, over and over again look at, ... to see nothing.

I mean, OpenSSL is such a vivid part of the infrastructure, top
that with security, of the world, it is used a billion of times
each and every day, and how many people really now about the
details inside this library with the terrible interface names (a
few minutes ago i discovered the also new ADMISSIONS.pod
documentation, and this is a statement for a language with
namespaces and method-on-object not instance-on-function; i always
think by myself that some university should perform neurological
science on what looking at

  const STACK_OF(ADMISSIONS) *ADMISSION_SYNTAX_get0_contentsOfAdmissions(const 
ADMISSION_SYNTAX *as);

from 9 to 5 (or 14 to 04 or whatever)/365-52 actually causes, how
do you say that in english?, what damages to the brain it causes,
anyway.)

The thing is, i have just seen the release candidate announcement
for bash 5.0 a while ago, and know what my own release message
will (have to) look like.

I did know nothing of all that.

 |||> I think i will move away from RAND_ then, nonetheless, and at
 |||> least for the things i have control of.
 |||
 |||I don't think this is a good idea. In the case when there is no suitable \
 |||\
 |||entropy
 |||source, the OpenSSL RNG will fail to generate random bytes, indeed.
 |||But how do you expect your application to seed the RNG properly in this
 |||case? What is your application's entropy source? If you have one, which
 |
 ||Yes, that is the problem.  We do have several possibilities for
 ||VAL_RANDOM.  Some of them only seed our builtin ARC4
 ||implementation, which until now exposes the internal buffer as
 ...
 ||randomizing those weakly through the well-known algorithm from
 ||"Random number generators: good ones are hard to find", Park and
 ||Miller, Communications of the ACM, vol. 31, no. 10, October 1988,
 ||p. 1195 (a_aux_rand_weak()).  This is the code:
 ...
 |Note that since that ancient article, ARC4 was not only invented, 
 |but also found too insecure for modern use.

I seem to know that for things like save transport.
I have for example suppressed adding the 

   for(u.i = 5 * sizeof(a_aux_rand->b8); u.i != 0; --u.i)
  a_aux_rand_get8();

which follows the unscientific seeding.  (And which likely is also
unscientific given that we have been seeded with random data
rather than with key/nonce ... whatever.)

 |Enjoy

Not really!  For example i read

  The reseeding default values are applied only during creation of
  a DRBG instance.  To ensure that they are applied to the global
  and thread-local DRBG instances (, resp.  and
  ), it is necessary to call
  RAND_DRBG_set_reseed_defaults() before creating any thread and
  before calling any cryptographic routines that obtain random
  data directly or indirectly.

Does this give me at least the promise that OPENSSL_init_ssl() can
be called without rendering RAND_DRBG_set_reseed_defaults(0,0,0,0)
effectively a useless call?
I see that said function may return error, but it seems that the
MAX_RESEED_INTERVAL it fails to accept is not a public variable!
I tend to do the much more expensive dance with
RAND_DRBG_get0_public() and RAND_DRBG_get0_master() and
individually applying RAND_DRBG_set_reseed_time_interval() as well
as RAND_DRBG_set_reseed_interval() on each of them.
Is this even allowed 

Re: [openssl-users] RNG behavior by default

[Jakob Bohm via openssl-users]

  
  
Small corrections below:
  
  On 07/01/2019 19:31, Steffen Nurpmeso wrote:


  ...


 |> That is really bad.  Of course you had to do it like this, and you
 |> surely have looked around to see what servers and other software
 |> which use OpenSSL do with the PRNG after forking (i.e., whether
 |> they reiterate the [RAND_file_name(),] RAND_load_file(),
 |> [:[RAND_add(),] RAND_status()], [RAND_write_file()] dance as
 |> documented, or not).
 |
 |I really don't understand your frustration: the new PRNG was designed
 |to relieve the application from the burden of seeding. It is easier to use,
 |not more complicated: No need to call RAND_add(), RAND_seed(),
 |RAND_load_file() and all this stuff. Just make sure the os entropy sources
 |are available and then simply use RAND_bytes(). But don't expect it to
 |succeed always.

And may i ask, what to do if it fails?  Try it in a loop a finite
number of times, assuming it over-and-over detects it is in an
error state and tries to reseed unless that succeeds, and bail if
still without success thereafter?

Talking about the only program in the public that i represent:
i will not abort that one just because a PRNG fails to produce
some bytes that this program uses for the purpose of, well, having
some (pseudo but) random bytes; because: this is _completely_
disproportionate to the use cases of those random numbers.
I guess the same could be said about UUIDS generated via PRNGs
instead of how the standard defines them, etc. etc.
Please do not say "use random(3) or rand(3)" now, please.
So what to do?


"Just failing" is precisely to allow you (and others) to do
  something 
  sensible without the random bytes, such as using something less
  secure, 
  or doing a polite error reporting to the user.


  

 |> I think i will move away from RAND_ then, nonetheless, and at
 |> least for the things i have control of.
 |
 |I don't think this is a good idea. In the case when there is no suitable \
 |entropy
 |source, the OpenSSL RNG will fail to generate random bytes, indeed.
 |But how do you expect your application to seed the RNG properly in this
 |case? What is your application's entropy source? If you have one, which

Yes, that is the problem.  We do have several possibilities for
VAL_RANDOM.  Some of them only seed our builtin ARC4
implementation, which until now exposes the internal buffer as
such (sufficient for our purpose yet).  Examples of this are
/dev/urandom, Linux getrandom as system call or library call,
aehh...  If that seeding fails, we create a completely
unscientific homebrew seed via CLOCK_REALTIME or gettimeofday(2),
whatever is available, wildly XORing through the buffer, further
randomizing those weakly through the well-known algorithm from
"Random number generators: good ones are hard to find", Park and
Miller, Communications of the ACM, vol. 31, no. 10, October 1988,
p. 1195 (a_aux_rand_weak()).  This is the code:


Note that since that ancient article, ARC4 was not only
  invented, 
  but also found too insecure for modern use.


Enjoy

Jakob
-- 
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded 
  

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

A wonderful Monday in the beautiful Winter time i wish.

I am sorry for the late reply again, i got a bug report for the
mailer i maintain, and from a long time user.
I hope it is ok that i compress the answers in one message, i am
talking much too much...

Kurt Roeckx wrote in <20190105221506.ga18...@roeckx.be>:
 |On Sat, Jan 05, 2019 at 08:33:18PM +0100, Steffen Nurpmeso wrote:
 |> 
 |> (I am also really interested and will look into OpenSSL to see if
 |> the abort() that seems to happen if the initial seed fails is in
 |> a linker-resolved constructor, and if not, why later failures do
 |> not also abort.
 |
 |We do not call abort(). A function like RAND_bytes() or
 |RAND_status() will just return an error.

Aah, i see.  I obviously had a completely false impression of the
new design.  I do not know where this came from, if i now search
in doc/ of the [master] there no mention of such wording, neither
abort nor exit nor whatever.  Hmm.?

  ...
 |RAND_bytes() has always documented that it can fail. Most function
 |calls can return errors for various reasons. Not checking for such
 |errors is in my opinion a bad style.

Well i, and until just now, interpreted the PRNG as something like
a "cipher stream", which simply applies (magic, to me) math to
a buffer of undefined content and an algorithm-chosen size.  Just
like MD5, which' RFC (1321) does not have error conditions.  (A
digest, but i had the RFC number at hand.)

So, to me.., i do not see any possible error condition, since the
initial seeding has been testified with RAND_status().

This is different now, and i will change the implementation as
soon as possible.  (This week.)

 |> i do not like that the stuff is instantiated in all threads
 |
 |It's only created in each thread that tries to the RNG. I'm not
 |sure what exactly your problem with it is. Either we need a global
 |lock, or we need an RNG per thread. We went with the per thread
 |RNG.

All my fault.  As a side note, i remember a commit message of
DragonFlyBSD and M. Dillon fly by (some years ago) where he
utilized the effect of SMP races as a random source.  If i recall
correctly.  Of course you had to protect the reseed count or
whatever (but there atomic cas or a spinlock would suffice
i guess).

 |We have a master DRBG that you can get with
 |RAND_DRBG_get0_master(). I recommend that you do not use it. It
 |requires that you take an external lock to use it. Internally we
 |lock it, but there is no way for you to use the same lock.
 |
 |> which
 |> in addition requires forks handlers to be installed all over the
 |> place.
 |
 |OpenSSL adds it's own fork handler now. You should not need to do
 |anything on fork related to OpenSSL.

It is just, to me, lack of standard.  But i do not dlopen() you,
and i do not dlclose() you.  In this scenario libraries may do use
anything, install atexit, onfork, whatever.  Other SSL libraries
even removed the possibility to hook the memory allocation
methods!

 |> the Linux kernel that drives
 |> the world from smallest to hugest has one internal entropy pool
 |> that feeds two public pools, whereas i the lucent little hobby
 |> server from user space get an armada of these.  Wow.
 |
 |Linux has a master pool, and a pool per core for the very same
 |reason as we also have a master DRBG and per thread DRBG.

I have read [1], but not that thorougly.  I was referring to the
user space interface, anyway.  (The thing is, i would have
preferred reading a one or two pages abstract instead of flying
over 138 pages.  I seem to know he knows.  I have read some
dissertations in my life, and i like the dedication if people
translate a complete system manual to German there, or show in
detail that they know know know their topic.  Having said that,
a very nice such work i have read from the Uni Kaiserslautern, it
was about 30 pages and it had been clear that "he can", shaking
off his hand.  Why need more?  Very good.  Imho.)

  [1] 
https://www.bsi.bund.de/SharedDocs/Downloads/EN/BSI/Publications/Studies/LinuxRNG/LinuxRNG_EN.pdf?__blob=publicationFile=7

 --End of <20190105221506.ga18...@roeckx.be>

Dr. Matthias St. Pierre wrote in :
 |>|Both manpages got an update during the DRBG rewrite (by me) and I don't
 |>|see any contradiction. You bring it to the point yourself:
 |> 
 |> I had a superficial look yesterday, but i think i have to reread
 |> them in total, anyway.
 |
 |Yes, please start with RAND(7) and RAND_DRBG(7).

Thanks.  I did just now.

 |> That is really bad.  Of course you had to do it like this, and you
 |> surely have looked around to see what servers and other software
 |> which use OpenSSL do with the PRNG after forking (i.e., whether
 |> they reiterate the [RAND_file_name(),] RAND_load_file(),
 |> [:[RAND_add(),] RAND_status()], [RAND_write_file()] dance as
 |> documented, or not).
 |
 |I really don't understand your frustration: the new PRNG was designed
 |to relieve the application from the burden of seeding. It is easier to use,
 |not more 

Re: [openssl-users] RNG behavior by default

[Dr. Matthias St. Pierre]

>  |Both manpages got an update during the DRBG rewrite (by me) and I don't
>  |see any contradiction. You bring it to the point yourself:
> 
> I had a superficial look yesterday, but i think i have to reread
> them in total, anyway.

Yes, please start with RAND(7) and RAND_DRBG(7).

> That is really bad.  Of course you had to do it like this, and you
> surely have looked around to see what servers and other software
> which use OpenSSL do with the PRNG after forking (i.e., whether
> they reiterate the [RAND_file_name(),] RAND_load_file(),
> [:[RAND_add(),] RAND_status()], [RAND_write_file()] dance as
> documented, or not).

I really don't understand your frustration: the new PRNG was designed
to relieve the application from the burden of seeding. It is easier to use,
not more complicated: No need to call RAND_add(), RAND_seed(),
RAND_load_file() and all this stuff. Just make sure the os entropy sources
are available and then simply use RAND_bytes(). But don't expect it to
succeed always.

> I think i will move away from RAND_ then, nonetheless, and at
> least for the things i have control of.

I don't think this is a good idea. In the case when there is no suitable entropy
source, the OpenSSL RNG will fail to generate random bytes, indeed.
But how do you expect your application to seed the RNG properly in this
case? What is your application's entropy source? If you have one, which
OpenSSL does not handle yet, you could theoretically register your
own get_entropy callback for the master DRBG at application startup time.

But if you don't have a better entropy source than OpenSSL, you are bound to
fail, too. And isn't it better for your application to fail gracefully in this 
case
than to provide snake oil security?

HTH,
Matthias


P.S: As for automatic reseeding: There are potential issues when upgrading
applications which do a fork and chroot from 1.1.0 to 1.1.1. Because in
version 1.1.1, the application will reseed after fork, and the reseeding can 
fail
if the application developer (or administrator) did not pay attention to provide
a random device in the chroot jail. This was no problem with 1.1.0 and below,
because the RNG never reseeded automatically.

OpenSSL does everything to avoid such a chroot reseeding failure (#6432 and 
#7437).
Also, on modern linux systems OpenSSL will prefer the getrandom system call
which does not have this limitation of the random devices. 


https://github.com/openssl/openssl/pull/6432
https://github.com/openssl/openssl/pull/7437



-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Kurt Roeckx]

On Sat, Jan 05, 2019 at 08:33:18PM +0100, Steffen Nurpmeso wrote:
> 
> (I am also really interested and will look into OpenSSL to see if
> the abort() that seems to happen if the initial seed fails is in
> a linker-resolved constructor, and if not, why later failures do
> not also abort.

We do not call abort(). A function like RAND_bytes() or
RAND_status() will just return an error.

> Yes, while i am going, the full truth is that
> i do not like it, i do not like that possibly a constructor call
> is made for something that is possibly not needed, if it is done
> like that, that someone simply aborts because of a some internal
> PRNG, especially so if it is not in a constructor call and if
> errors can and are expected to be handled by PRNG users anyway,

RAND_bytes() has always documented that it can fail. Most function
calls can return errors for various reasons. Not checking for such
errors is in my opinion a bad style.

> i do not like that the stuff is instantiated in all threads

It's only created in each thread that tries to the RNG. I'm not
sure what exactly your problem with it is. Either we need a global
lock, or we need an RNG per thread. We went with the per thread
RNG.

We have a master DRBG that you can get with
RAND_DRBG_get0_master(). I recommend that you do not use it. It
requires that you take an external lock to use it. Internally we
lock it, but there is no way for you to use the same lock.

> which
> in addition requires forks handlers to be installed all over the
> place.

OpenSSL adds it's own fork handler now. You should not need to do
anything on fork related to OpenSSL.

> the Linux kernel that drives
> the world from smallest to hugest has one internal entropy pool
> that feeds two public pools, whereas i the lucent little hobby
> server from user space get an armada of these.  Wow.

Linux has a master pool, and a pool per core for the very same
reason as we also have a master DRBG and per thread DRBG.


Kurt

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

Dr. Matthias St. Pierre wrote in :
 |I agree with Kurt, except for one point:
 |
 |> The RAND_bytes and RAND_status manpages can clearly be improved.
 |
 |Both manpages got an update during the DRBG rewrite (by me) and I don't
 |see any contradiction. You bring it to the point yourself:

I had a superficial look yesterday, but i think i have to reread
them in total, anyway.

 |> So _IF_ it is seeded it is seeded...
 |
 |It is true that the DRBG will automatically seed, but it is equally true
 |that it can still end up in an unseeded (error) state, if no suitable \
 |entropy
 |source is available. And since this can also happen during reseeding (which
 |in particular is enforced after a fork), it is always necessary to \
 |check the return
 |value of the RAND_bytes() function. Because in the error state, the \
 |buffer is not
 |filled at all.

That is really bad.  Of course you had to do it like this, and you
surely have looked around to see what servers and other software
which use OpenSSL do with the PRNG after forking (i.e., whether
they reiterate the [RAND_file_name(),] RAND_load_file(),
[:[RAND_add(),] RAND_status()], [RAND_write_file()] dance as
documented, or not).

I think i will move away from RAND_ then, nonetheless, and at
least for the things i have control of.
But i will definitely reread the manual now.

Thanks for your answer.
Ciao and a nice weekend from Germany,

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

Good evening.

Please excuse the late reply.

Kurt Roeckx wrote in <20190104180735.ga25...@roeckx.be>:
 |On Fri, Jan 04, 2019 at 02:48:48PM +0100, Steffen Nurpmeso wrote:
 |> Dr. Matthias St. Pierre wrote in <450169f8ca7c43d1841c4c8052e78c72@Ex13.\
 |> ncp.local>:
 ...
 |>|2. If something goes wrong with PRNG initialization, that it will fail \
 |>|hard rather than fall back to something less secure. And if so how \
 |>|I detect such a failure.
 |>|
 |>|If the (re-)seeding fails, the DRBG enters an error state. When you \
 |>|try to generate random bytes it will detect the error state and try
 |>|automatically to heal the error state by reinstantiating. But if \
 |>|reseeding \
 |>|fails, it will return and error code and not generate any pseudo random \
 |>|bytes.
 |>|
 |>|Citing from the manual pages:
 |>  ...
 |>| As a normal application developer, you do not have to worry about \
 |>| any details, just use RAND_bytes(3)
 |>| to obtain random data. Having said that, there is one important rule \
 |>| to obey: Always check the error
 |>| return value of RAND_bytes(3) and do not take randomness for granted.
 |>|
 |>| https://www.openssl.org/docs/man1.1.1/man7/RAND.html
 |> 
 |> That is new however, _imho_.  The wording of RAND_bytes(3) (still)
 |> says that "an error occurs [.if.] not [been] seeded with enough
 |> [data]", and RAND_status(3) returns 1 if the PRNG "has been seeded
 |> with enough data".  So if it is seeded it is seeded, in my
 |> understanding anything further on up the road only mixes in noise
 |> (which likely will undergo further maths and be stirred into the
 |> pool, i have not looked, actually).  I do not test RAND_bytes(3)
 |> return (yet), because i have ensured the PRNG is sufficiently
 |> seeded, and RAND_status(3) returns success, before RAND_bytes(3)
 |> is used the first time.
 |
 |For 1.1.0 and older that works, because they do not reseed. Since
 |1.1.1 it does reseed, and if the reseed fails, it will go to an
 |error state. So yes, this is new behavior.

Thanks.  Well this is good to know, i have to think about how
i will deal with this.

(I am also really interested and will look into OpenSSL to see if
the abort() that seems to happen if the initial seed fails is in
a linker-resolved constructor, and if not, why later failures do
not also abort.  Yes, while i am going, the full truth is that
i do not like it, i do not like that possibly a constructor call
is made for something that is possibly not needed, if it is done
like that, that someone simply aborts because of a some internal
PRNG, especially so if it is not in a constructor call and if
errors can and are expected to be handled by PRNG users anyway,
i do not like that the stuff is instantiated in all threads, which
in addition requires forks handlers to be installed all over the
place.  Now, on NetBSD for example, and because of libraries i am
linking in, i do have two complete sets of fork handlers and PRNG
buffers all over the place, one for NetBSD LibC, one for OpenSSL.
In my unfortunately not so humble opinion this looks like a modern
white first world child.  Including the postural defect.

I am sorry for this rant.  Yeah, i think OpenSSL even offers PRNG
objects which i can use and control directly, and through this
i regain a bit control of resource usage etc.  Without rewriting
software in order to pregenerate randoms in the main( manager)
thread to pass them through to jobs in thread pools.  Just look
for those and do not use the TSD/fork-aware RAND_bytes().
But before i become a total troll.. the Linux kernel that drives
the world from smallest to hugest has one internal entropy pool
that feeds two public pools, whereas i the lucent little hobby
server from user space get an armada of these.  Wow.

It is not your fault of course.  I am sorry for this rant.
Thanks for your answer.)

How do i deal with this?  Maybe i simply always use my builtin
ARC4, it is so small, or upgrade to ChaCha20 or so, and add
something like a Blake2 filter to protect that pool, and only
serve the output of it.  Or something.

 |The RAND_bytes and RAND_status manpages can clearly be improved.
 |Since you always have to check RAND_bytes's return value now,
 |RAND_status is mostly useless.

Yes, in my opinion good documentation is hard, almost impossible.
I am not using it, but i always enjoy reading the concise
(concise!  Says Mr. McIlroy.) yet helpful manuals from the really
great guys, for example the Plan9 manuals regarding programming.
Maybe a bit too concise for end-users, except academics.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Kurt Roeckx]

On Sat, Jan 05, 2019 at 08:45:37AM +1000, Dr Paul Dale wrote:
> I’m not sure about the quality of Android’s sources, but would expect them to 
> be decent.

Android is just a Linux kernel. It always had /dev/urandom. Oreo
(8.0) requires at least Linux kernel 4.4. There were no
requirements for the kernel version before this. Some devices with
Marshmallow (6.0) will run a Linux kernel with 3.18, but not all
of them.


Kurt

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Dr. Matthias St. Pierre]

I agree with Kurt, except for one point:

> The RAND_bytes and RAND_status manpages can clearly be improved.

Both manpages got an update during the DRBG rewrite (by me) and I don't
see any contradiction. You bring it to the point yourself:

> So _IF_ it is seeded it is seeded...

It is true that the DRBG will automatically seed, but it is equally true
that it can still end up in an unseeded (error) state, if no suitable entropy
source is available. And since this can also happen during reseeding (which
in particular is enforced after a fork), it is always necessary to check the 
return
value of the RAND_bytes() function. Because in the error state, the buffer is 
not
filled at all.

Matthias


-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Dr Paul Dale]

I know that iOS (which was listed) has a good randomness source 
(SecRandomCopyBytes 
)
 but I don’t think OpenSSL uses it yet.
I’m not sure about the quality of Android’s sources, but would expect them to 
be decent.


Pauli


> On 4 Jan 2019, at 10:46 pm, Dr. Matthias St. Pierre 
>  wrote:
> 
>> So my concerns are:
>> 1. Whether I really can count on getting a high-entropy PRNG across these 
>> various platforms, without any explicit initialization.
> 
> Yes, for the mentioned platforms, the default configuration is 
> `--with-rand-seed=os`, which means the DRBG automatically seeds
> and reseeds using os entropy sources.
> 
> 2. If something goes wrong with PRNG initialization, that it will fail hard 
> rather than fall back to something less secure. And if so how I detect such a 
> failure.
> 
> If the (re-)seeding fails, the DRBG enters an error state. When you try to 
> generate random bytes it will detect the error state and try
> automatically to heal the error state by reinstantiating. But if reseeding 
> fails, it will return and error code and not generate any pseudo random bytes.
> 
> Citing from the manual pages:
> 
>   OpenSSL comes with a default implementation of the RAND API which is 
> based on the
>   deterministic random bit generator (DRBG) model as described in [NIST 
> SP 800-90A Rev. 1].
>   The default random generator will initialize automatically on first use 
> and will be fully functional
>   without having to be initialized ('seeded') explicitly. It seeds and 
> reseeds itself automatically using
>   trusted random sources provided by the operating system.
> 
>   As a normal application developer, you do not have to worry about any 
> details, just use RAND_bytes(3)
>   to obtain random data. Having said that, there is one important rule to 
> obey: Always check the error
>   return value of RAND_bytes(3) and do not take randomness for granted.
> 
>   https://www.openssl.org/docs/man1.1.1/man7/RAND.html
> 
> (See also https://www.openssl.org/docs/man1.1.1/man7/RAND_DRBG.html)
> 
> Matthias
> 
> -- 
> openssl-users mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Kurt Roeckx]

On Fri, Jan 04, 2019 at 02:48:48PM +0100, Steffen Nurpmeso wrote:
> Dr. Matthias St. Pierre wrote in <450169f8ca7c43d1841c4c8052e78c72@Ex13.\
> ncp.local>:
>  |> So my concerns are:
>  |> 1. Whether I really can count on getting a high-entropy PRNG across \
>  |> these various platforms, without any explicit initialization.
>  |
>  |Yes, for the mentioned platforms, the default configuration is `--with-r\
>  |and-seed=os`, which means the DRBG automatically seeds
>  |and reseeds using os entropy sources.
>  |
>  |2. If something goes wrong with PRNG initialization, that it will fail \
>  |hard rather than fall back to something less secure. And if so how \
>  |I detect such a failure.
>  |
>  |If the (re-)seeding fails, the DRBG enters an error state. When you \
>  |try to generate random bytes it will detect the error state and try
>  |automatically to heal the error state by reinstantiating. But if reseeding \
>  |fails, it will return and error code and not generate any pseudo random \
>  |bytes.
>  |
>  |Citing from the manual pages:
>  ...
>  | As a normal application developer, you do not have to worry about \
>  | any details, just use RAND_bytes(3)
>  | to obtain random data. Having said that, there is one important rule \
>  | to obey: Always check the error
>  | return value of RAND_bytes(3) and do not take randomness for granted.
>  |
>  | https://www.openssl.org/docs/man1.1.1/man7/RAND.html
> 
> That is new however, _imho_.  The wording of RAND_bytes(3) (still)
> says that "an error occurs [.if.] not [been] seeded with enough
> [data]", and RAND_status(3) returns 1 if the PRNG "has been seeded
> with enough data".  So if it is seeded it is seeded, in my
> understanding anything further on up the road only mixes in noise
> (which likely will undergo further maths and be stirred into the
> pool, i have not looked, actually).  I do not test RAND_bytes(3)
> return (yet), because i have ensured the PRNG is sufficiently
> seeded, and RAND_status(3) returns success, before RAND_bytes(3)
> is used the first time.

For 1.1.0 and older that works, because they do not reseed. Since
1.1.1 it does reseed, and if the reseed fails, it will go to an
error state. So yes, this is new behavior.

The RAND_bytes and RAND_status manpages can clearly be improved.
Since you always have to check RAND_bytes's return value now,
RAND_status is mostly useless.


Kurt

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Steffen Nurpmeso]

Dr. Matthias St. Pierre wrote in <450169f8ca7c43d1841c4c8052e78c72@Ex13.\
ncp.local>:
 |> So my concerns are:
 |> 1. Whether I really can count on getting a high-entropy PRNG across \
 |> these various platforms, without any explicit initialization.
 |
 |Yes, for the mentioned platforms, the default configuration is `--with-r\
 |and-seed=os`, which means the DRBG automatically seeds
 |and reseeds using os entropy sources.
 |
 |2. If something goes wrong with PRNG initialization, that it will fail \
 |hard rather than fall back to something less secure. And if so how \
 |I detect such a failure.
 |
 |If the (re-)seeding fails, the DRBG enters an error state. When you \
 |try to generate random bytes it will detect the error state and try
 |automatically to heal the error state by reinstantiating. But if reseeding \
 |fails, it will return and error code and not generate any pseudo random \
 |bytes.
 |
 |Citing from the manual pages:
 ...
 | As a normal application developer, you do not have to worry about \
 | any details, just use RAND_bytes(3)
 | to obtain random data. Having said that, there is one important rule \
 | to obey: Always check the error
 | return value of RAND_bytes(3) and do not take randomness for granted.
 |
 | https://www.openssl.org/docs/man1.1.1/man7/RAND.html

That is new however, _imho_.  The wording of RAND_bytes(3) (still)
says that "an error occurs [.if.] not [been] seeded with enough
[data]", and RAND_status(3) returns 1 if the PRNG "has been seeded
with enough data".  So if it is seeded it is seeded, in my
understanding anything further on up the road only mixes in noise
(which likely will undergo further maths and be stirred into the
pool, i have not looked, actually).  I do not test RAND_bytes(3)
return (yet), because i have ensured the PRNG is sufficiently
seeded, and RAND_status(3) returns success, before RAND_bytes(3)
is used the first time.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Dr. Matthias St. Pierre]

> So my concerns are:
> 1. Whether I really can count on getting a high-entropy PRNG across these 
> various platforms, without any explicit initialization.

Yes, for the mentioned platforms, the default configuration is 
`--with-rand-seed=os`, which means the DRBG automatically seeds
and reseeds using os entropy sources.

2. If something goes wrong with PRNG initialization, that it will fail hard 
rather than fall back to something less secure. And if so how I detect such a 
failure.

If the (re-)seeding fails, the DRBG enters an error state. When you try to 
generate random bytes it will detect the error state and try
automatically to heal the error state by reinstantiating. But if reseeding 
fails, it will return and error code and not generate any pseudo random bytes.

Citing from the manual pages:

OpenSSL comes with a default implementation of the RAND API which is 
based on the
deterministic random bit generator (DRBG) model as described in [NIST 
SP 800-90A Rev. 1].
The default random generator will initialize automatically on first use 
and will be fully functional
without having to be initialized ('seeded') explicitly. It seeds and 
reseeds itself automatically using
trusted random sources provided by the operating system.

As a normal application developer, you do not have to worry about any 
details, just use RAND_bytes(3)
to obtain random data. Having said that, there is one important rule to 
obey: Always check the error
return value of RAND_bytes(3) and do not take randomness for granted.

https://www.openssl.org/docs/man1.1.1/man7/RAND.html

(See also https://www.openssl.org/docs/man1.1.1/man7/RAND_DRBG.html)

Matthias

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] RNG behavior by default

[Kurt Roeckx]

On Thu, Jan 03, 2019 at 11:03:01AM -0500, Mike Blaguszewski wrote:
> I am using the EVP API (version 1.1.1) for performing public key and 
> symmetric key operations across a variety of platforms (macOS, Windows, 
> Linux, iOS and Android). I am currently not doing anything to explicitly seed 
> OpenSSL’s random number generator. My understanding is that the default 
> behavior  should be 
> cryptographically secure.
> 
> So my concerns are:
> 1. Whether I really can count on getting a high-entropy PRNG across these 
> various platforms, without any explicit initialization.
> 2. If something goes wrong with PRNG initialization, that it will fail hard 
> rather than fall back to something less secure. And if so how I detect such a 
> failure.
> 
> Our current implementation uses libsodium, which relies on the usual system 
> calls to generate entropy, so if I can count on OpenSSL always doing this 
> then I’m happy. 

It will make use of system calls when available. Those are known
to provide system calls:
- Linux since 3.17
- Darwin since 16 (OSX 10.12, IOS 10.0).
- Solaris since 11.3
- OpenBSD since 5.6
- FreeBSD since 12.0 (1200061)

By default it will fall back to use something like /dev/urandom if
the system call is not available or returns an error.

On Windows we are also using the system provided entropy by using
function calls.

You do not need to do anything to initialize RNG. It will
automatically initiailze on first use.

It will hard fail when it's not able to get entropy.

Since it now reseeds from time to time, it can actually start to
fail after having run succesfully for some time. But it's very
unlikely that you would run into that, by default we should make
sure that we can always get entropy.


Kurt

-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users