Hi Florian,
> I wonder if this change to src/share/lib/security/java.security-linux
>
> -securerandom.source=file:/dev/urandom
> +securerandom.source=file:/dev/random
>
> causes the return of the blocking behavior.
Welcome to my "can of worms." [1] I hope everything I've said below is
correct, and haven't made any typos!
> In the past, I saw
> /dev/random-related blocking during server start-up because too many
> SecureRandom instances needed seeding. If I follow the code
> correctly,
If you came to this code fresh and understood it all in 7 hours, I would
either give you a medal, or hand in my badge. :) It's really twisted
stuff.
> seeding of non-strong generators now uses /dev/random again, which is
> subject to blocking.
Given the long history of the JDK and concerns about backwards
compatibility, our SecureRandom implementations have become *REALLY*
hard to understand. Trying to make sense of all this, and now cleaning
it up, was actually more than 1/2 of the project. It took me a couple
weeks of intense code scrutiny/archeology, and then needed to write a
wiki page just to keep my head wrapped around it. Likely the most
confusing issue I've ever worked on.
In a nutshell, the current implementation of SHA1PRNG does indeed read
from /dev/random while seeding itself and supplying bytes via
generateSeed(), which is in opposition to the documented behavior of the
security properties: Remember the infamous "file:/dev/./urandom"
workaround for SHA1PRNG? [2] (sigh...)
I'll try to simplify as much as I can. I'll be including parts of this
for the JDK 8 Oracle's Provider Impl page.
In the current environment, here's our current list of providers, in
default preference order.
Solaris:
PKCS11/NativePRNG/SHA1PRNG
Linux/Mac:
NativePRNG/SHA1PRNG
Windows:
SHA1PRNG/Windows-PRNG
As you probably know, "new SecureRandom()" obtains the first
implementation in the list, unless you call
"SecureRandom.getInstance("alg") or SecureRandom.getInstance("alg",
"provider"), which gets a specific algorithm, either most preferred
version or by a specific provider, respectively.
The securerandom.source/java.security.egd currently only affects
SHA1PRNG, but there's a slight wrinkle in that certain property values
promote NativePRNG ahead of SHA1PRNG in the list of algorithms in the
SUN provider. More on this later.
Current behavior:
=================
NativePRNG:
-----------
does not use securerandom.source/java.security.egd at all
engineGenerateSeed: always reads from /dev/random
engineNextBytes: always reads from /dev/urandom
SHA1PRNG:
---------
Since this is a real PRNG [3], SHA1PRNG requires a very strong seed
(initial) value, as strong as possible. If not initially seeded by the
user via setSeed, SHA1PRNG will call SeedGenerator.getSystemEntropy +
SeedGenerator.generateSeed() to generate a strong seed for itself.
There are three seed generators which have SeedGenerator as their parent:
ThreadedSeedGenerator was the original Seeder which uses
system load to generate seed values, and is only used as a
last resort. (SLOW!)
URLSeedGenerator takes a URL and reads from it.
NativeSeedGenerator uses /dev/random.
When the SeedGenerator class initializes, if the properties
(securerandom.source/java.security.egd) resolve to either the exact
strings "file:/dev/random" or "file:/dev/urandom", it will use
NativeSeedGenerator (/dev/random). (Yeah, try explaining that to
customers!) If the properties resolve to any other URL, then we use
URLSeedGenerator with that URL. Note, this is why the infamous
workaround "file:///dev/urandom" or "file:/dev/./urandom" works. [2]
As someone once said, this has got to be the most confusing workaround
in the history of the JDK. I would agree! :) If neither SeedGenerator
is available, then we use ThreadedSeedGenerator as the last resort.
Once seeded, then later calls to this SHA1PRNG SecureRandom instance are:
engineGenerateSeed: always reads from SeedGenerator
engineNextBytes: Number generation using the current state as
input into SHA1.
So even though the EGD properties point to file:/dev/urandom, it's not
correct when it comes to the seeder.
Back in JDK 5, adding NativePRNG was the "workaround" for this problem.
When setting up the default order of implementations in the SUN
provider, if the properties URL is "file:/dev/urandom", then NativePRNG
becomes the most preferred provider (over SHA1PRNG). So for those Linux
implementations which just asked for the "most preferred" (i.e. new
SecureRandom()), they get an instance of NativePRNG which uses
/dev/urandom for nextBytes. (Note well:
NativeSeedGenerator.generateSeed() still reads from /dev/random).
So since you're not seeding a SHA1PRNG, using NativePRNG.nextBytes()
makes it appear as though there is no more stalling, as long as you only
call nextBytes() and not generateSeed(). But if you specifically asked
for SHA1PRNG (which a lot of customers had baked into their code), you
still go through the above, and will stall when SHA1PRNG tried to seed
itself.
(Again, try to explain this to customers. We've had many escalations
over the years!)
PKCS11/Windows-PRNG:
--------------------
Didn't touch. Their behavior doesn't allow have the concept of a EGD.
New Behavior:
=============
So, what do my change do?
. Corrects the java.security files to reflect the new implementation,
and corrects vagueness/long-standing errors. In particular, the
security property default becomes "file:/dev/random" which matches the
old AND new implementations, which is what triggered this long discussion!
. SUN provider: If the properties are *EITHER* "file:/dev/random" or
"file:/dev/urandom", then it promotes NativePRNG ahead of SHA1PRNG in
the list of preferred impls. If it's not one of those, then SHA1PRNG is
preferred over NativePRNG.
. NativePRNG: Now respects the seeder properties. If the property
isn't "file:/" or doesn't exist, we'll fall back to /dev/random.
. Adds blocking and nonblocking variants of NativePRNG:
NativePRNGBlocking/NativePRNGJNonBlocking. The
generateSeed()/nextBytes() methods both use /dev/random or /dev/urandom,
respectively.
. NativeSeedGenerator: essentially, this becomes the same as a
URLSeedGenerator. Instead of only using "/dev/random", it will the
specified value of "file:/dev/random" or "file:/dev/urandom"
. Adds SecureRandom.getStrongSecureRandom. Uses a security property to
specify the strongest impl(s) on this system. See the API for more details.
. Minor style cleanups
I hope this is clear.
Brad
[1] https://www.google.com/search?q=define%3A+can+of+worms
[2]
http://security.stackexchange.com/questions/14386/what-do-i-need-to-configure-to-make-sure-my-software-uses-dev-urandom
[3] http://en.wikipedia.org/wiki/Pseudorandom_number_generator
