Re: Want to help upstream software improve their random?
Sent: Friday, December 12, 2014 at 5:02 AM
From: Theo de Raadt dera...@cvs.openbsd.org
To: t...@cvs.openbsd.org
Subject: Want to help upstream software improve their random?
Not my business, but how do you handle cases of rand() srand()-like
calls from software like awk?
What is and what should be the result of:
for w in `seq 1 10`; do awk 'BEGIN{print int(rand() * 32767)}'; done
or
for w in `seq 1 10`; do awk 'BEGIN{srand(); print int(rand() * 32767)}'; done
Re: Want to help upstream software improve their random?
Sent: Friday, December 12, 2014 at 5:02 AM From: Theo de Raadt dera...@cvs.openbsd.org To: t...@cvs.openbsd.org Subject: Want to help upstream software improve their random? Not my business, but how do you handle cases of rand() srand()-like calls from software like awk? awk in OpenBSD has been true-random since this commit: CVSROOT:/cvs Module name:src Changes by: mill...@cvs.openbsd.org 2011/04/20 16:28:39 Modified files: usr.bin/awk: run.c Log message: Stash the old seed when srand() is called and use it as the return value as per POSIX and the fine manual. Fix sent upstream to bwk. OK deraadt@ If you use srand() directly in awk, it reverts to the standards-defined behaviour. The same semantic is present in our ksh.
Re: Want to help upstream software improve their random?
Theo de Raadt dera...@cvs.openbsd.org wrote:
In all of these code blocks are a well-known piece of information
(same time on your machine as everywhere else) is being used to seed a
deterministic number generator.
At some later point, deterministic numbers are taken out using rand(),
random(), drand48(), lrand48(), mrand48(), or srand48(), or some
derivative function inside the program itself, and used for WHO KNOWS
WHAT PURPOSE.
I did not audit what the numbers are being used for.
Quite likely some numbers are just used to help hashing. Some could
be used to print pretty pictures. But in xulrunner? In the zip password
creator? In postgresql, or say in openldap (a network related thing)?
It is doubtful they are all fine.
For the benefit of other projects who haven't taken the same steps as
OpenBSD, it would be nice if some people helped out these pieces of
software.
I took one as an example.
apr-util-1.5.3srand((unsigned int)(((time_now 32) ^ time_now)
0x));
apr-util-1.5.3srand((unsigned int)apr_time_now());
Here is the only usage of rand in its entirety.
/* true_random -- generate a crypto-quality random number. */
static int true_random(void)
{
apr_uint64_t time_now;
#if APR_HAS_RANDOM
unsigned char buf[2];
if (apr_generate_random_bytes(buf, 2) == APR_SUCCESS) {
return (buf[0] 8) | buf[1];
}
#endif
/* crap. this isn't crypto quality, but it will be Good Enough */
time_now = apr_time_now();
srand((unsigned int)(((time_now 32) ^ time_now) 0x));
return rand() 0x0;
}
Interestingly this is inside a UUID generator, which already uses time
as part of the unique value. Deterministic statistically random numbers
based on time don't help at all.
I looked into apr_generate_random_bytes and found that it is a giant
ifdef, which, if nothing is set, will reduce to a one line function
returning success. I hope their configure script doesn't continue if
nothing is set.
From APR 1.5.1, which Theo's script didn't seem to catch:
#define arc4random() rand()
But if mkstemp is present, they assume random is also (I wonder why?)
and use
#define arc4random() random()
as if that were more secure.
I find no evidence of real arc4random in their tree. It's as if they're
relying on the magic of the name to protect them.
APR also has it's own random number generator in tree, which neither
the UUID generator nor the mkstemp replacement uses. I don't know enough
math to pass judgement on it other than saying random generation belongs
in some library or the kernel.
Apache HTTPD contains a large snarl of code intended to do OpenSSL's
job for it and seed it, but I don't know enough about OpenSSL to pass
judgement.
There are several more rand calls in HTTPD, including what appears to
be another random number generator, complete with comments indicating
that APR should have its own random number generator.
What I'm ultimately saying is that upstream's views on random number
generation can be mighty strange. Many are of the opinion that an
insecure fallback is better than refusing to compile. Not that it is an
excuse for poor software engineering, but many are old enough to be
rather jumbled in their library use.
I tried to CC their list but it won't let me post without subscribing
and furthermore I couldn't seem to subscribe.
-- Martin
Re: Want to help upstream software improve their random?
12 дек. 2014 г. 8:04 пользователь Theo de Raadt dera...@cvs.openbsd.org
написал:
In all of these code blocks are a well-known piece of information
(same time on your machine as everywhere else) is being used to seed a
deterministic number generator.
At some later point, deterministic numbers are taken out using rand(),
random(), drand48(), lrand48(), mrand48(), or srand48(), or some
derivative function inside the program itself, and used for WHO KNOWS
WHAT PURPOSE.
I did not audit what the numbers are being used for.
Quite likely some numbers are just used to help hashing. Some could
be used to print pretty pictures. But in xulrunner? In the zip password
creator? In postgresql, or say in openldap (a network related thing)?
It is doubtful they are all fine.
For the benefit of other projects who haven't taken the same steps as
OpenBSD, it would be nice if some people helped out these pieces of
software.
EMBOSS-6.0.1srand((unsigned) time(tm));
ORBit2-2.14.19 srand (t.tv_sec ^ t.tv_usec ^ getpid () ^ getuid ());
apr-util-1.5.3srand((unsigned int)(((time_now 32) ^ time_now)
0x));
apr-util-1.5.3srand((unsigned int)apr_time_now());
aqualung-0.9beta11 srand(time(0));
aqualung-0.9beta11 srand(time(NULL));
audacious-3.5.2srand (time (NULL));
audacious-plugins-3.5.2srand(time(NULL));
audacity-1.3.9 srand(time(0));
audacity-1.3.9 srand(time(NULL));
audacity-1.3.9srand( (unsigned int) time(NULL) );
birda-1.1srandom(t.tv_sec^t.tv_usec);
boost-1.53.0std::srand( runtime_config::random_seed() );
boost-1.53.0 srand(time(0));
boost-1.53.0generator() { srand(time(0)); }
boost-1.53.0generator() { srand(time(0)); }
boost-1.53.0std::srand(time(0) + world.rank());
boost-1.53.0std::srand(time(0) + world.rank());
boost-1.53.0 srand(time(0) + world.rank());
boost-1.53.0 srand(time(0) + world.rank());
boost-1.53.0 std::srand(time(0) + world.rank());
boost-1.53.0 std::srand(time(0) + world.rank());
boost-1.53.0srand( time(NULL) );
boost-1.53.0srand( time( NULL ) );
boost-1.53.0srand ( time(NULL) );
boost-1.53.0std::srand(static_castunsigned(std::time(0)));
boost-1.53.0std::srand(static_castunsigned(std::time(0)));
boost-1.53.0 srand(time(0));
boost-1.53.0 srand(time(0));
boost-1.53.0std::srand((unsigned int)std::time(NULL));
boost-1.53.0srand(time(0));
bullet-2.81// srand(time(NULL) / 30);
bullet-2.81 srand((unsigned)time(NULL)); // Seed it...
bullet-2.81 srand ( time ( 0x0 ) );
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
caps-plugins-0.4.4 srandom (tv.tv_sec ^ tv.tv_usec);
celestia-1.6.1 std::srand(std::time(NULL));
celestia-1.6.1 std::srand(time(NULL));
celestia-1.6.1srandom(time(NULL));
celt-0.11.1 srand(time(NULL));
celt07-0.7.1 srand(time(NULL));
cgdb-0.6.8srand(time(NULL));
clementine-1.2.3 srandom((int)[[NSDate date] timeIntervalSince1970]);
clementine-1.2.3srandom(time(NULL));
clementine-1.2.3srand ( time ( NULL ) );
clementine-1.2.3 qsrand((time.tv_sec * 1000) + (time.tv_usec / 1000));
cmake-3.0.2srand((unsigned)time(0));
cmake-3.0.2 srand((unsigned int)time(NULL)+randomizer++); /* seed */
codeblocks-13.12srand( time(NULL) );
codeblocks-13.12inline void ini_random() { srand(time(0)); };
codeblocks-13.12srand((unsigned)time(0));
codeblocks-13.12srand(time(nullptr));
codeworker-4.5.4if (iSeed = 0) srand((unsigned) iSeed);
codeworker-4.5.4else srand((unsigned) time(NULL));
db-3.1.17 srand((u_int)time(NULL));
db-3.1.17 srand(getpid() | time(NULL));
db-3.1.17 srand((unsigned int)time(NULL));
db-4.6.21 srand((u_int)time(NULL));
db-4.6.21 srand(getpid() | time(NULL));
db-4.6.21 srand((unsigned int)time(NULL));
db-4.6.21 srand((u_int)time(NULL) % (u_int)getpid());
db-4.6.21 srand((u_int)(time(NULL) | getpid()));
db-4.6.21 srand((u_int)(time(NULL) | getpid()));
deadbeef-0.6.2srand (time (NULL));
deadbeef-0.6.2//srand ((uint) ::time(NULL));
deadbeef-0.6.2 srand(time(NULL));
deadbeef-0.6.2 fixed random playback bug caused by libsidplay2 calling
srand(time(NULL))
festival-1.95beta#define seed_random() srand((unsigned)time(NULL))
festival-1.95beta#define seed_random() srandom(time(NULL));
festival-1.95betasrand(time(NULL));
flac-1.3.0 srand((unsigned)time(0));
flac-1.3.0 srand((unsigned)time(0));
flac-1.3.0
Want to help upstream software improve their random?
In all of these code blocks are a well-known piece of information
(same time on your machine as everywhere else) is being used to seed a
deterministic number generator.
At some later point, deterministic numbers are taken out using rand(),
random(), drand48(), lrand48(), mrand48(), or srand48(), or some
derivative function inside the program itself, and used for WHO KNOWS
WHAT PURPOSE.
I did not audit what the numbers are being used for.
Quite likely some numbers are just used to help hashing. Some could
be used to print pretty pictures. But in xulrunner? In the zip password
creator? In postgresql, or say in openldap (a network related thing)?
It is doubtful they are all fine.
For the benefit of other projects who haven't taken the same steps as
OpenBSD, it would be nice if some people helped out these pieces of
software.
EMBOSS-6.0.1srand((unsigned) time(tm));
ORBit2-2.14.19 srand (t.tv_sec ^ t.tv_usec ^ getpid () ^ getuid ());
apr-util-1.5.3srand((unsigned int)(((time_now 32) ^ time_now)
0x));
apr-util-1.5.3srand((unsigned int)apr_time_now());
aqualung-0.9beta11 srand(time(0));
aqualung-0.9beta11 srand(time(NULL));
audacious-3.5.2srand (time (NULL));
audacious-plugins-3.5.2srand(time(NULL));
audacity-1.3.9 srand(time(0));
audacity-1.3.9 srand(time(NULL));
audacity-1.3.9srand( (unsigned int) time(NULL) );
birda-1.1srandom(t.tv_sec^t.tv_usec);
boost-1.53.0std::srand( runtime_config::random_seed() );
boost-1.53.0 srand(time(0));
boost-1.53.0generator() { srand(time(0)); }
boost-1.53.0generator() { srand(time(0)); }
boost-1.53.0std::srand(time(0) + world.rank());
boost-1.53.0std::srand(time(0) + world.rank());
boost-1.53.0 srand(time(0) + world.rank());
boost-1.53.0 srand(time(0) + world.rank());
boost-1.53.0 std::srand(time(0) + world.rank());
boost-1.53.0 std::srand(time(0) + world.rank());
boost-1.53.0srand( time(NULL) );
boost-1.53.0srand( time( NULL ) );
boost-1.53.0srand ( time(NULL) );
boost-1.53.0std::srand(static_castunsigned(std::time(0)));
boost-1.53.0std::srand(static_castunsigned(std::time(0)));
boost-1.53.0 srand(time(0));
boost-1.53.0 srand(time(0));
boost-1.53.0std::srand((unsigned int)std::time(NULL));
boost-1.53.0srand(time(0));
bullet-2.81// srand(time(NULL) / 30);
bullet-2.81 srand((unsigned)time(NULL)); // Seed it...
bullet-2.81 srand ( time ( 0x0 ) );
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
c3270-3.3.11.6 srand(time(NULL));
c3270-3.3.11.6 srandom(time(NULL));
caps-plugins-0.4.4 srandom (tv.tv_sec ^ tv.tv_usec);
celestia-1.6.1 std::srand(std::time(NULL));
celestia-1.6.1 std::srand(time(NULL));
celestia-1.6.1srandom(time(NULL));
celt-0.11.1 srand(time(NULL));
celt07-0.7.1 srand(time(NULL));
cgdb-0.6.8srand(time(NULL));
clementine-1.2.3 srandom((int)[[NSDate date] timeIntervalSince1970]);
clementine-1.2.3srandom(time(NULL));
clementine-1.2.3srand ( time ( NULL ) );
clementine-1.2.3 qsrand((time.tv_sec * 1000) + (time.tv_usec / 1000));
cmake-3.0.2srand((unsigned)time(0));
cmake-3.0.2 srand((unsigned int)time(NULL)+randomizer++); /* seed */
codeblocks-13.12srand( time(NULL) );
codeblocks-13.12inline void ini_random() { srand(time(0)); };
codeblocks-13.12srand((unsigned)time(0));
codeblocks-13.12srand(time(nullptr));
codeworker-4.5.4if (iSeed = 0) srand((unsigned) iSeed);
codeworker-4.5.4else srand((unsigned) time(NULL));
db-3.1.17 srand((u_int)time(NULL));
db-3.1.17 srand(getpid() | time(NULL));
db-3.1.17 srand((unsigned int)time(NULL));
db-4.6.21 srand((u_int)time(NULL));
db-4.6.21 srand(getpid() | time(NULL));
db-4.6.21 srand((unsigned int)time(NULL));
db-4.6.21 srand((u_int)time(NULL) % (u_int)getpid());
db-4.6.21 srand((u_int)(time(NULL) | getpid()));
db-4.6.21 srand((u_int)(time(NULL) | getpid()));
deadbeef-0.6.2srand (time (NULL));
deadbeef-0.6.2//srand ((uint) ::time(NULL));
deadbeef-0.6.2 srand(time(NULL));
deadbeef-0.6.2 fixed random playback bug caused by libsidplay2 calling
srand(time(NULL))
festival-1.95beta#define seed_random() srand((unsigned)time(NULL))
festival-1.95beta#define seed_random() srandom(time(NULL));
festival-1.95betasrand(time(NULL));
flac-1.3.0 srand((unsigned)time(0));
flac-1.3.0 srand((unsigned)time(0));
flac-1.3.0 srand((unsigned)time(0));
fldigi-3.21.83//srand(time(NULL));
fritzing-0.9.0 srand ( time(NULL) );
fritzing-0.9.0srand((unsigned)(time(NULL) ^ ZCR_SEED2));
giblib-1.2.4 srand(getpid()
Re: Want to help upstream software improve their random?
On 12 Dec 2014, at 5:02, Theo de Raadt wrote: In all of these code blocks are a well-known piece of information (same time on your machine as everywhere else) is being used to seed a deterministic number generator. At some later point, deterministic numbers are taken out using rand(), random(), drand48(), lrand48(), mrand48(), or srand48(), or some derivative function inside the program itself, and used for WHO KNOWS WHAT PURPOSE. I did not audit what the numbers are being used for. Quite likely some numbers are just used to help hashing. Some could be used to print pretty pictures. But in xulrunner? In the zip password creator? In postgresql, or say in openldap (a network related thing)? It is doubtful they are all fine. For the benefit of other projects who haven't taken the same steps as OpenBSD, it would be nice if some people helped out these pieces of software. EMBOSS-6.0.1srand((unsigned) time(tm)); [...] What you say makes sense. Is there a best practice alternative you suggest or did I miss that? Perhaps just some better initiation value, preferably not all from the same place? devin -- contact info: http://nacredata.com/devin gpg public key: http://www.nacredata.com/public_key.txt Use unique, strong passwords! https://www.nacredata.com/password.php
Re: Want to help upstream software improve their random?
On 12 Dec 2014, at 5:02, Theo de Raadt wrote: In all of these code blocks are a well-known piece of information (same time on your machine as everywhere else) is being used to seed a deterministic number generator. At some later point, deterministic numbers are taken out using rand(), random(), drand48(), lrand48(), mrand48(), or srand48(), or some derivative function inside the program itself, and used for WHO KNOWS WHAT PURPOSE. I did not audit what the numbers are being used for. Quite likely some numbers are just used to help hashing. Some could be used to print pretty pictures. But in xulrunner? In the zip password creator? In postgresql, or say in openldap (a network related thing)? It is doubtful they are all fine. For the benefit of other projects who haven't taken the same steps as OpenBSD, it would be nice if some people helped out these pieces of software. EMBOSS-6.0.1srand((unsigned) time(tm)); [...] What you say makes sense. Is there a best practice alternative you suggest or did I miss that? Perhaps just some better initiation value, preferably not all from the same place? These code sequences do not need deterministic values. They actually want non-deterministic values. Steps are like this: 1. Remove the srand(), srandom(), srand48(), seed48(), lcong48() calls. 2. Replace all rand(), random(), lrand48(), mrand48() calls with arc4random() 3. If the calls use %, consider using arcrandom_uniform() instead. 4. If it loops to fill a buffer, use arc4random_buf() 5. Replace drand48() with arc4random and some floating point, but be careful. drand48() is very rare. Now, talk to the upstream projects. They will reject your changes because Linux lacks arc4random(). Basically, only Linux and Solaris lack arc4random(). All the other platforms have it. There are libraries available which provide arc4random() on Linux, so maybe you find an upstream software provider who is willing to create a dependency on such a library on Linux. Lots of software is doing precisely that, so don't be afraid.
Re: Want to help upstream software improve their random?
On 12 Dec 2014, at 5:43, Theo de Raadt wrote: On 12 Dec 2014, at 5:02, Theo de Raadt wrote: In all of these code blocks are a well-known piece of information (same time on your machine as everywhere else) is being used to seed a deterministic number generator. At some later point, deterministic numbers are taken out using rand(), random(), drand48(), lrand48(), mrand48(), or srand48(), or some derivative function inside the program itself, and used for WHO KNOWS WHAT PURPOSE. I did not audit what the numbers are being used for. Quite likely some numbers are just used to help hashing. Some could be used to print pretty pictures. But in xulrunner? In the zip password creator? In postgresql, or say in openldap (a network related thing)? It is doubtful they are all fine. For the benefit of other projects who haven't taken the same steps as OpenBSD, it would be nice if some people helped out these pieces of software. EMBOSS-6.0.1srand((unsigned) time(tm)); [...] What you say makes sense. Is there a best practice alternative you suggest or did I miss that? Perhaps just some better initiation value, preferably not all from the same place? These code sequences do not need deterministic values. They actually want non-deterministic values. Steps are like this: 1. Remove the srand(), srandom(), srand48(), seed48(), lcong48() calls. 2. Replace all rand(), random(), lrand48(), mrand48() calls with arc4random() 3. If the calls use %, consider using arcrandom_uniform() instead. 4. If it loops to fill a buffer, use arc4random_buf() 5. Replace drand48() with arc4random and some floating point, but be careful. drand48() is very rare. Now, talk to the upstream projects. They will reject your changes because Linux lacks arc4random(). Basically, only Linux and Solaris lack arc4random(). All the other platforms have it. There are libraries available which provide arc4random() on Linux, so maybe you find an upstream software provider who is willing to create a dependency on such a library on Linux. Lots of software is doing precisely that, so don't be afraid. Got it. Thanks. devin -- contact info: http://nacredata.com/devin gpg public key: http://www.nacredata.com/public_key.txt Use unique, strong passwords! https://www.nacredata.com/password.php
Re: Want to help upstream software improve their random?
On 11 December 2014 at 21:43, Theo de Raadt dera...@cvs.openbsd.org wrote: On 12 Dec 2014, at 5:02, Theo de Raadt wrote: In all of these code blocks are a well-known piece of information (same time on your machine as everywhere else) is being used to seed a deterministic number generator. At some later point, deterministic numbers are taken out using rand(), random(), drand48(), lrand48(), mrand48(), or srand48(), or some derivative function inside the program itself, and used for WHO KNOWS WHAT PURPOSE. I did not audit what the numbers are being used for. Quite likely some numbers are just used to help hashing. Some could be used to print pretty pictures. But in xulrunner? In the zip password creator? In postgresql, or say in openldap (a network related thing)? It is doubtful they are all fine. For the benefit of other projects who haven't taken the same steps as OpenBSD, it would be nice if some people helped out these pieces of software. EMBOSS-6.0.1srand((unsigned) time(tm)); [...] What you say makes sense. Is there a best practice alternative you suggest or did I miss that? Perhaps just some better initiation value, preferably not all from the same place? These code sequences do not need deterministic values. They actually want non-deterministic values. Steps are like this: 1. Remove the srand(), srandom(), srand48(), seed48(), lcong48() calls. 2. Replace all rand(), random(), lrand48(), mrand48() calls with arc4random() 3. If the calls use %, consider using arcrandom_uniform() instead. 4. If it loops to fill a buffer, use arc4random_buf() 5. Replace drand48() with arc4random and some floating point, but be careful. drand48() is very rare. Now, talk to the upstream projects. They will reject your changes because Linux lacks arc4random(). Basically, only Linux and Solaris lack arc4random(). All the other platforms have it. There are libraries available which provide arc4random() on Linux, so maybe you find an upstream software provider who is willing to create a dependency on such a library on Linux. Lots of software is doing precisely that, so don't be afraid. Thank you. Are there any specific good libraries you know of? -- The best the little guy can do is what the little guy does right
Re: Want to help upstream software improve their random?
There are libraries available which provide arc4random() on Linux, so maybe you find an upstream software provider who is willing to create a dependency on such a library on Linux. Lots of software is doing precisely that, so don't be afraid. Thank you. Are there any specific good libraries you know of? libbsd will do. It is not the best body of code for this, it is really old. It will hopefully improve. In any case for these purposes it is more than good enough.
Re: Want to help upstream software improve their random?
On Thu, Dec 11, 2014 at 09:52:46PM -0800, Eugene Yunak wrote: Thank you. Are there any specific good libraries you know of? -- The best the little guy can do is what the little guy does right LibreSSL :-) -Bryan.
Re: Want to help upstream software improve their random?
On Thu, Dec 11, 2014 at 09:52:46PM -0800, Eugene Yunak wrote: Thank you. Are there any specific good libraries you know of? LibreSSL :-) Indeed, if a system has LibreSSL, you will find the arc4random family in -lcrypto.
