Re: regarding the changes to kernel entropy gathering

[RVP]

On Sun, 4 Apr 2021, Taylor R Campbell wrote:


No, because the output of /dev/random and /dev/urandom is the output
of a pseudorandom number generator that meets modern standards of
security.

If anyone had _ever_ published statistical tests that the PRNG failed
in a detectable way, then (a) this would be an earthshattering
development in the cryptography literature, which would be hotly
discussed in much more significant forums than NetBSD mailing lists,
and (b) we would stop using this PRNG and switch to another one.



Right. Well, it's been quite a few years since I ran this sort of test
on /dev/*random, and things, obviously, have been fixed. (Back in the
early 2000s, the tests I mentioned turned up their noses at /dev/random
data from the Linux kernel.)

Then, the issue here is one of predictability. NetBSD doesn't want, for
extremely valid reason, to incorporate any perturbation sources which
have been pooh-poohed in the technical literature. But, its /dev/random
is good enough that no statistical tests of randomness will fail it.

Hmm. I have to say, that now I find myself not disagreeing with Greg's
point of view: Maybe NetBSD's default is too strict and a knob like
kern.entropy.use_pooh_poohed_sources=1 would not be a bad thing for
some users--with all appropriate sysinst warnings of course.

Or, perhaps statistical tests of the raw in-kernel sources will demonstrate
exactly why things like timing jitter have been pooh-poohed in the
literature?

-RVP

PS: As an aside, I'm perfectly OK with the choices currently made in 9.99:
``if there's any doubt, leave it out''

Re: regarding the changes to kernel entropy gathering

[Lloyd Parkes]

With some trepidation, I'm going to dip into this conversation even 
though I haven't read all of.  I don't have the mental fortitude for 
that. I have two suggestions, one short and one long.


Firstly, we could just have an rc.d script that checks to see if the 
system has /var/db/entropy-file or an rng device, and if not then it 
prints a warning and then generates some simplistic entropy with "ls -lR 
/ ; dd if=/dev/urandom of=/dev/random bs=32 count=1 ; sysctl -w 
kern.entropy.consolidate=1". The system owner has been warned and the 
system proceeds to run.


Secondly we could fix what I see as the biggest problem with the new 
implementation that I see right now and that is that it is unreasonably 
difficult for people to work out how to make their system go forwards 
once it has stopped. Note that making the system go forwards is easy, 
it's work out what to do that's hard. We can fix that.


The current implementation prints out a message whenever it blocks a 
process that wants randomness, which immediately makes this 
implementation superior to all others that I have ever seen. The number 
of times I've logged into systems that have stalled on boot and made 
them finish booting by running "ls -lR /" over the past 20 years are too 
many to count. I don't know if I just needed to wait longer for the boot 
to finish, or if generating entropy was the fix, and I will never know. 
This is nuts.


We can use the message to point the system administrator to a manual 
page that tells them what to do, and by "tells them what to do", I mean 
in plain simple language, right at the top of the page, without scaring 
them.


How about this..

"entropy: pid %d (%s) blocking due to lack of entropy, see entropy(4)"

and then in entropy(4) we can start with something like

"If you are reading this because you have read a kernel message telling 
you that a process is blocking due to a lack of entropy then it is 
almost certainly because your hardware doesn't have a reliable source of 
randomness. If you have no particular requirements for cryptographic 
security on your system, you can generate some entropy and then tell the 
kernel that this entropy is 'enough' with the commands

    ls -lR /
    dd if=/dev/urandom of=/dev/random bs=32 count=1
    sysctl -w kern.entropy.consolidate=1
If have strong requirements for cryptographic security on your system 
then you should run 'rndctl -S /root/seed' on a system with hardware 
random number generate (most modern CPUs), copy the seed file over to 
this system as /var/db/entropy-file and then run 'rndctl -L 
/var/db/entropy-file'.


This only needs to be done once since scripts in rc.d will take care of 
saving and restoring system entropy in /var/db/entropy-file across reboots."


We could even do both of these things.

Re: regarding the changes to kernel entropy gathering

[Robert Elz]

Date:Mon, 5 Apr 2021 01:14:01 +0200
From:Joerg Sonnenberger 
Message-ID:  

  | That is discussed in the security model Taylor presented a long time
  | ago. In short: nothing. In most use cases, you are screwed at this point
  | anyway

This is where the disconnect is happening I think.   Many of you are
simply not understanding the point.

I am not screwed, I just don't care.Is that so hard to understand?

Let me make it plainer.

I run systems on which I allow root logins with no password.   I have run
systems where root ssh access is permitted, put those together and you
get root access from over the net (and telnet would allow that as well).

Alternatively I can aim for greater security, and configure a root
password ... like say the system's host name.

NetBSD allows me to do all that - it might not be the standard configuration,
but it is possible.   You might think it is insane, and that's fine, but
there are reasons.

On recent NetBSD, as I understand it, I can

dd if=/dev/zero bs=N count=1 >/dev/random

and now I have "entropy".   But it refuses to provide a simpler knob
to do the same thing (or perhaps something a little saner, but equally
as simple to use).

The logic behind that makes no sense to me.

I understand that some people desire highly secure systems (I'm not
convinced that anyone running NetBSD can really justify that desire,
but that's beside the point) and that's fine - make the system be able
to be as secure as possible, just don't require me to enable it, and
don't make it impossible or even difficuly to disable it - and allow
some kind of middle ground, just just "perfectly secure" and "hopeless".

kre

Re: regarding the changes to kernel entropy gathering

[Mouse]

> All that changed is that we don't pretend it provides entropy.

Instead, you pretend it provides none.

Neither pretense is accurate (where the "pretend it provides entropy"
refers to providing any non-configurable fixed amount).  The real
problem here, as I see it, is that NetBSD qua NetBSD cannot tell where
between those two extremes any particular case falls.  Compounding this
is that most sysadmins are not competent to accurately describe where a
given install falls, even were NetBSD to provide a means to do so.
(Though I suspect that most who understand the issues could provide a
usefully high lower bound on the entry provided.  For example, I'd be
satisfied with one millibit per event - network packet or disk transfer
- and in my environment I am confident that is an understimate.)

> As I mentioned elsewhere, a lot of the classic entropy sources are
> surprisingly bad nowadays when someone can observe the kernel,
> especially in a virtualized environment.

Yes...but, as gwoods indirectly but correctly pointed out, that may not
matter: other VMs under the same hypervisor can be anywhere from "known
to be actively hostile" to "fully trusted", depending on the site.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: regarding the changes to kernel entropy gathering

[Brian Buhrow]

Hello.  As I understand it, Greg ran into this problem on a xen domu.  In 
checking my NetBSD-9
system running as a domu under xen-4.14.1, there is no rdrand or rdseed feature 
exposed to
domu's by xen.  This observation is confirmed by looking at the xen command 
line reference
page: https://xenbits.xen.org/docs/unstable/misc/xen-command-line.html

So, it seems the best answer is to update our documentation to say that 
the xen
hypervisor, by default, doesn't provide the rdrand and rdseed instructions to 
the xen guests
and NetBSD doesn't trust the random sources provided by the xennet(4) and 
xbd(4) drivers.
Therefore, the only solution to get randomness working for the first time on a 
newlyinstalled
domu is to write 32 bytes to /dev/random.
-thanks
-Brian

Re: regarding the changes to kernel entropy gathering

[Joerg Sonnenberger]

On Sun, Apr 04, 2021 at 11:47:10PM +0700, Robert Elz wrote:
> If not, what prevents someone from reading (copying) the file from the
> system while it is stopped (assessing the storage device via other methods)
> and then knowing exactly what the seed is going to be when the system boots?

That is discussed in the security model Taylor presented a long time
ago. In short: nothing. In most use cases, you are screwed at this point
anyway since various other cryptographic material like the host ssh key
is also lost. There is one special case here where this has to be taken
under consideration and that is cloning virtual machines. The short
answer is that you as system integrator are responsible for handling it
in an appropiate manner. Ensuring that the VM sees enough entropic
action before the entropy is accessed would ensure that. The seed file
doesn't replace the entropy pool, so any entropy that actually did get
added during the boot process still remains.

> I think I'd prefer possibly insecure, but difficult to obtain from outside
> like disk drive interrupt timing low order bits than that.   Regardless of
> how unproven that method might be.

See above, that's still the case. Noone said anything about not using
sources of potential entropy. All that changed is that we don't pretend
it provides entropy. As I mentioned elsewhere, a lot of the classic
entropy sources are surprisingly bad nowadays when someone can observe
the kernel, especially in a virtualized environment.

Joerg

Re: regarding the changes to kernel entropy gathering

[Joerg Sonnenberger]

On Sun, Apr 04, 2021 at 03:32:08PM -0700, Greg A. Woods wrote:
> At Mon, 05 Apr 2021 00:14:30 +0200 (CEST), Havard Eidnes  
> wrote:
> Subject: Re: regarding the changes to kernel entropy gathering
> >
> > > What about architectures that have nothing like RDRAND/RDSEED?  Are
> > > they, effectively, totally unsupported now?
> >
> > Nope, not entirely.  But they have to be seeded once.  If they
> > have storage which survives reboots, and entropy is saved and
> > restored on reboot, they will be ~fine.
> 
> BTW, to me reusing the same entropy on every reboot seems less secure.

Except that's not what the system is doing. It removes the seed file on
boot and creates a new one on shutdown.

> > Systems without persistent storage and also without RDRAND/RDSEED
> > will however be ... a more challenging problem.
> 
> Leaving things like that would be totally silly.
> 
> With my patch the old way of gathering entropy from devices works just
> fine as it always did, albeit with the second patch it does require a
> tiny bit of extra configuration.

You keep repeating yourself. It doesn't make your claims any less false.
At this point, can we please just stop this thread?

Joerg

Re: regarding the changes to kernel entropy gathering

[RVP]

On Sun, 4 Apr 2021, Greg A. Woods wrote:


At Sun, 4 Apr 2021 09:49:58 +, Taylor R Campbell  
wrote:


Your change _creates_ the lie that every bit of data entered this way
is drawn from a source with independent uniform distribution.


No, my change _allows_ the administrator to decide which devices can be
used as estimating/counting entropy sources.  For example I know that
many of the devices on almost all of my machines (virtual or otherwise)
are equally good sources of entropy for their uses.



I think running the /dev/random bit-stream through some statistical
tests, (both on RDRAND/RDSEED-based and estimator-based as in your
patch) would be useful here.

Binary packages already have the dieharder RNG tester. Then, there
is John Walker's ent for PRNGs: https://fourmilab.ch/random/

NIST has some too, I believe (I can't locate them right now).

-RVP

PS. Is there a way to get the bit-stream from the various in-kernel
sources so that we can run them through these sort of tests? That
way we can check--not intuit--how random the bit-streams they
produce really are.

Re: regarding the changes to kernel entropy gathering

[Joerg Sonnenberger]

On Mon, Apr 05, 2021 at 12:07:49AM +0200, Havard Eidnes wrote:
> I am still of the fairly firm beleif that the mistrust in the
> hardware vendors' ability to make a reasonable and robust
> implementation is without foundation.

It's not without foundation. Remember the first hardware RNG on x86? It
got killed by a down-shrink in the pre-release phase. The hardware still
said it is present. That's the reason why we don't use RDSEED directly
in place of the entropy pool and still merge other potential entropy
sources in.

Joerg

Re: regarding the changes to kernel entropy gathering

[Mouse]

>> Nope, not entirely.  But they have to be seeded once.  If they have
>> storage which survives reboots, and entropy is saved and restored on
>> reboot, they will be ~fine.
> BTW, to me reusing the same entropy on every reboot seems less
> secure.

Hence the "saved and" part of the above.  (I still think it's
misguided, but I don't expect NetBSD to agree.  It's been a long time
since NetBSD and I saw eye to eye on...well, on many points.)

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: regarding the changes to kernel entropy gathering

[Mouse]

> I am still of the fairly firm beleif that the mistrust in the
> hardware vendors' ability to make a reasonable and robust
> implementation is without foundation.

I don't doubt the ability.  I don't doubt that they _can_.

I question whether they _do_.  (And, indeed, there has been at least
one incident that demonstrates that on occasion they don't.)

If I am ever in a situation where I need randomness good enough that I
care about things like the accuracy of entropy estimates, I expect the
applicable threat model will consider CPU manufacturers untrusted.
Thus, I would want the system to consider RDRAND and its ilk same as
any potential other source of entropy: trusted to generate real
unpredictability only when specifically configured that way.  (The
question of what the default should be is a separate one.)

Of course, in a situation like that, I would also want to do all the
relevant processing on CPUs (and, where applicable, other hardware) old
enough to predate the reasons for caring about that level of
unpredictability, because it makes them significantly less likely to
have been trojaned.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: regarding the changes to kernel entropy gathering

[Mansour Moufid]

On Sun, Apr 4, 2021 at 7:14 PM Havard Eidnes  wrote:

> Well.  That depends on what you mean by "entropy".

Hit the nail on the head.

Although there is only one definition of entropy in information theory
/ cryptography, it has different colloquial meanings, especially on
the web. It makes communication impossible and these arguments come up
over and over again...

Anyway, the NetBSD definition and implementation are correct. :)

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Sun, 4 Apr 2021 23:09:18 +, Taylor R Campbell  
wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> If you know this (and this is something I certainly can't confidently
> assert!), you can write 32 bytes to /dev/random, save a seed, and be
> done with it.

I don't have random data easily available at install time.

I don't have random data easily available every time I boot a machine
with non-persistent storage (e.g. a test ISO image).

I _do_ trust well enough the sources of randomness in some device
drivers to provide me with a secure enough amount of entropy, for my
purposes.

And so with my fix(es) I don't need to feed supposedly random data to
every system on every install and/or every reboot.

What's worse?  My fixes, or something like this in /etc/rc.local:

   echo -n "" > /dev/random

> But users who don't go messing around with obscure rndctl settings in
> rc.conf will be proverbially shot in the foot by this change -- except
> they won't notice because there is practically guaranteed to be no
> feedback whatsoever for a security disaster until their systems turn
> up in a paper published at Usenix like .

You're really stretching your argument thinly if you are assuming
everyone _needs_ perfect entropy here.

Also, that's only if the default RND_FLAG_ESTIMATE_* bits are turned off.

AND only if the system doesn't have some true hardware RNG.

> What your change does is equivalent to going around to every device
> driver that previously said `this provides zero entropy, or I don't
> know how much entropy it provides' and replacing that claim by `this
> is a sample of an independent and perfectly uniform random string of
> bits', which is a much stronger (and falser) claim than even the old
> `entropy estimation' confabulation that NetBSD used to do.

No, only if the default RND_FLAG_ESTIMATE_* bits are ***NOT*** turned off.

AND only if the user is like me and stuck with some poor second-grade
ancient hardware that doesn't have some fancy new true hardware RNG.

In the mean time a more productive approach would be to figure out
what's best for those of us who don't need perfection every time and/or
to fix those device drivers that could feed sufficiently random data to
the entropy pool, and then to recommend a suitable value for
rndctl_flags in /etc/rc.conf.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpnOADtmWrjC.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Mon, 5 Apr 2021 01:05:58 +0200, Joerg Sonnenberger  wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> Part of the problem here is that most of the non-RNG data sources are
> easily observable either from the local system (e.g. any malicious user)
> or other VMs on the same machine (in case of a hypervisor) or local
> machines on the same network (in case of network interrupts).

It _Just_ _Doesn't_ _Matter_  (i.e. for many of us, most of the time).

Now ideally in the hypervisor scenario we would have a backend device
that read from /dev/random and offered it to the VM guest as a virtual
hardware RNG.  Or maybe it's as simple as passing a those few bytes
through a custom Xenstore string and having a script in the VM read them
and inject them into /dev/random.  But that's not been done yet.

BTW, personally, on at least on some machines, I don't have any worry
whatsoever at the moment about one VM guest spying on, or influencing
the PRNG, in another.  Zero worry.  They're all _me_.  I don't need some
theoretically perfect level of protection from myself.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpFPOplfhwSl.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Taylor R Campbell]

> Date: Sun, 4 Apr 2021 21:24:56 + (UTC)
> From: RVP 
> 
> I think running the /dev/random bit-stream through some statistical
> tests, (both on RDRAND/RDSEED-based and estimator-based as in your
> patch) would be useful here.

No, because the output of /dev/random and /dev/urandom is the output
of a pseudorandom number generator that meets modern standards of
security.

If anyone had _ever_ published statistical tests that the PRNG failed
in a detectable way, then (a) this would be an earthshattering
development in the cryptography literature, which would be hotly
discussed in much more significant forums than NetBSD mailing lists,
and (b) we would stop using this PRNG and switch to another one.

(Device-dependent health tests do make sense in the HWRNG device
driver, to detect broken devices before we treat them as having
entropy, which is why we do them wherever we can, e.g. to detect the
AMD RDRAND bugs.)

Re: regarding the changes to kernel entropy gathering

[Havard Eidnes]

>> > No amount of uptime and activity was increasing the entropy in my
>> > system before I patched it.
>>
>> As I understand it, entropy was being contributed.  What wasn't
>> happening was the random driver code recognizing and acknowledging that
>> entropy, because it had no way to tell how much of it there really was.
>
> Clearly there was no entropy being contributed in any way shape or form.

Well.  That depends on what you mean by "entropy".

Samples are still being collected and mixed into the pool from your
listed sources.  By your statement, that should have contributed some
"randomness" into the pool, some might call this "entropy".

However, since the quality of those samples are unknown, and there is
no reliable model to estimate the actual quality of those samples,
they are in NetBSD-current not being counted as contributing to the
"entropy estimate" counter.  That is by design: the entropy estimation
is now quite conservative, as you have noticed.

I also presented a workaround for this problem; if you are reasonably
certain that you actually have mixed in a sufficient number of bits of
sufficient quality into the randomness pool (see "rndctl -l -v"), you
can do

# dd if=/dev/urandom of=/dev/random count=1

since if this is done by root, it counts as the "/dev/random" source,
and the bits fed in there by root are counted 1:1 as contributing to
the entropy estimate.  After this, your system will not block anymore
reading on /dev/random, and this state of affairs will be preserved
across reboots as long as you save and restore the entropy pool on
reboot.  (Which, admittedly, requires r/w storage for the relevant
file/directory, ref. your other ongoing thread.)

> I told the system to "count" the entropy being gathered by the
> appropriate driver(s), but it was being ignored entirely.

Well, you are now counting bitstrings of unknown and perhaps dubious
quality as contributing 1:1 to the "entropy estimate".  It's by design
that we don't do that anymore.

> After my fix the system behaved as I told it to.

But now with perhaps questionable estimation of the actual entropy
sitting in your pool.

Regards,

- Håvard

Re: regarding the changes to kernel entropy gathering

[Taylor R Campbell]

> Date: Sun, 04 Apr 2021 12:58:09 -0700
> From: "Greg A. Woods" 
> References: 
>   <20210404094958.692f360...@jupiter.mumble.net>
> 
> At Sun, 4 Apr 2021 09:49:58 +, Taylor R Campbell  
> wrote:
> Subject: Re: regarding the changes to kernel entropy gathering
> >
> > Your change _creates_ the lie that every bit of data entered this way
> > is drawn from a source with independent uniform distribution.
> 
> No, my change _allows_ the administrator to decide which devices can be
> used as estimating/counting entropy sources.  For example I know that
> many of the devices on almost all of my machines (virtual or otherwise)
> are equally good sources of entropy for their uses.

If you know this (and this is something I certainly can't confidently
assert!), you can write 32 bytes to /dev/random, save a seed, and be
done with it.

But users who don't go messing around with obscure rndctl settings in
rc.conf will be proverbially shot in the foot by this change -- except
they won't notice because there is practically guaranteed to be no
feedback whatsoever for a security disaster until their systems turn
up in a paper published at Usenix like .

What your change does is equivalent to going around to every device
driver that previously said `this provides zero entropy, or I don't
know how much entropy it provides' and replacing that claim by `this
is a sample of an independent and perfectly uniform random string of
bits', which is a much stronger (and falser) claim than even the old
`entropy estimation' confabulation that NetBSD used to do.

Re: regarding the changes to kernel entropy gathering

[Joerg Sonnenberger]

On Sun, Apr 04, 2021 at 09:24:56PM +, RVP wrote:
> PS. Is there a way to get the bit-stream from the various in-kernel
> sources so that we can run them through these sort of tests? That
> way we can check--not intuit--how random the bit-streams they
> produce really are.

Part of the problem here is that most of the non-RNG data sources are
easily observable either from the local system (e.g. any malicious user)
or other VMs on the same machine (in case of a hypervisor) or local
machines on the same network (in case of network interrupts). That's the
real reason why their entropy is hard to estimate. It becomes even more
annoying with modern hardware features like interrupt moderation of
nics. They can make the timing of interrupts highly predicable.

Joerg

Re: regarding the changes to kernel entropy gathering

[Taylor R Campbell]

> Date: Sun, 4 Apr 2021 11:14:31 -0700
> From: John Nemeth 
> 
>  I understand the need for good random sources, and won't argue
> it.  My question is, how can we tell what random sources a system
> actually has, i.e. is there some flag that cpuctl identify shows
> when a system has RDRAND/RDSEED?

# cpuctl identify 0 | grep -e RDRAND -e RDSEED
cpu0: features1 0x7fbae3bf

>   Are there other sources that can
> be positively identified as providing randomness?

`rndctl -l' will tell you whether any sources you have on your system
have provided any entropy.  The system generally tries to read from
HWRNGs as soon as possible at boot, so unless something is wrong you
will see such sources listed next to nonzero bits of entropy in
`rndctl -l' as soon as you can run that.

You can grep the code for rnd_add_data and rnd_add_data_sync to find
the drivers that pass nonzero values as the last argument, which is
the number of bits of entropy in the process that generated the sample
being fed in.

Lots of SoCs have on-board RNGs these days; there are Intel and ARM
CPU instructions (no ARMv8.5 hardware yet that I know of, but we're
ready for its RNG!); some crypto decelerators like tpm(4), ubsec(4),
and hifn(4) have RNGs; and there are some dedicated RNG devices like
ualea(4).

Re: regarding the changes to kernel entropy gathering

[Taylor R Campbell]

> Date: Sun, 04 Apr 2021 23:47:10 +0700
> From: Robert Elz 
> 
> Date:Sun, 4 Apr 2021 15:28:13 +
> From:Taylor R Campbell 
> Message-ID:  <20210404152814.3c56360...@jupiter.mumble.net>
> 
>   | you can let NetBSD take care of it automatically
>   | on subsequent boots by running `/etc/rc.d/random_seed stop' to save a
>   | seed to disk.)
> 
> Is that file encrypted?   If it is, where does the decryption key come from?

No.  If the adversary can bypass file system privileges to read (and,
most likely, write) arbitrary data on your disk, that adversary is
outside the threat model and you need to do something else to deal
with such an adversary -- for example, by encrypting your disk with
cgd(4).

If the random seed is stored on a networked file system, then NetBSD
incorporates it but treats it as having zero entropy since a network
eavesdropper could read it.

(Side note: Disclosure of the _current_ seed file does not enable
retroactive recovery of _past_ secrets generated from /dev/urandom.)

> I think I'd prefer possibly insecure, but difficult to obtain from outside
> like disk drive interrupt timing low order bits than that.   Regardless of
> how unproven that method might be.

The seed is hashed together with disk drive interrupt timings, so you
get security from either or both, along with RDRAND/RDSEED output if
available.  The kernel also uses the hardclock timer to periodically
sample a cycle counter as a makeshift ring oscillator, if nothing else
is available.  Every device listed in `rndctl -l' has the opportunity
to influence the output of /dev/urandom; for security it is enough
that any one of them be adequately unpredictable.

> And what's the scheme for cheap low-end devices that have no
> writable storage?  (The proverbial internet toaster, for example).

If the device has a HWRNG, great.  If not, this may be a difficult
problem for the system engineer designing the device to solve.
There's no free lunch.

> Lastly, why would anyone presume that RDRAND generates less predictable
> bits (less predictable to someone who knows how it works) than any of
> the other methods that are used.

We generally take the vendor's word in hardware documentation about
what's actually behind the device registers, just like we do for any
other device we have a driver for (e.g., we generally expect devices
not to leak the contents of arbitrary RAM across the network).

In the case of Intel RDRAND, the design is documented in

Intel Digital Random Number Generator Software Implementation
Guide, Revision 2.1, October 17, 2019.

https://web.archive.org/web/20190404022757/https://software.intel.com/sites/default/files/managed/98/4a/DRNG_Software_Implementation_Guide_2.1.pdf

The design samples from ring oscillators -- two independently clocked
circuits, one of which rapidly alternates between states and the other
of which periodically samples the first, so the state is determined by
the random timing jitter between the two clocks, influenced by
unpredictable thermal noise in the silicon.  This class of designs is
reasonably well-studied in the literature -- much moreso than, e.g.,
disk seek times or network packet times.

We also incorporate new information about the devices if available.
For example, on certain AMD CPUs, RDRAND and/or RDSEED sometimes gets
into bad states, which the device driver checks for; in this event the
driver counts zero entropy.

We sometimes do a bit of science on the devices too.  See
https://nxr.netbsd.org/xref/src/sys/arch/mips/cavium/dev/octeon_rnm.c
for some notes on the Cavium Octeon HWRNG, and
https://nxr.netbsd.org/xref/src/sys/arch/arm/sunxi/sun8i_crypto.c for
an example of a driver for a thing that might be an OK HWRNG but is
not clearly endorsed by vendor documentation and has a very nonuniform
distribution so we obviously need more than 256 bits of sample
material to get 256 bits of entropy -- but how much more is not clear,
which is why it counts as zero entropy for now.

> If we want really good security, I'd submit we need to disable
> the random seed file, and RDRAND (and anything similar) until we
> have proof that they're perfect.

...right

FYI: If you really want, you can stop the kernel from counting any
particular entropy source by setting rndctl_flags in /etc/rc.conf:

rndctl_flags="-E -d rdrand; -E -d rdseed; -E -d rdrand/rdseed; -E -d seed"

Then any attempt to read from /dev/random will block until some other
entropy source has provided enough entropy.

Re: regarding the changes to kernel entropy gathering

[Joerg Sonnenberger]

On Sun, Apr 04, 2021 at 02:16:41PM -0700, Paul Goyette wrote:
> > Personally, I'm happy with anything that your average high school
> > student is unlikely to be able to crack in an hour.   I don't run
> > a bank, or a military installation, and I'm not the NSA.   If someone
> > is prepared to put in the effort required to break into my systems,
> > then let them, it isn't worth the cost to prevent that tiny chance.
> > That's the same way that my house has ordinary locks - I'm sure they
> > can be picked by someone who knows what they're doing, and better
> > security is available, at a price, but a nice happy medium is what
> > fits me best.
> 
> FWIW, I used to work for a company whose marketing motto was
> 
>   Good enough isn't!
> 
> But I definitely agree with you - what we used to have is "good
> enough" for the vast bulk of our users and potential users.
> 
> Perhaps sysinst(8) should ask
> 
>   Do you need a hyper-secure system?
> 
> If yes, then leave things as they are today.  But if you answer no,
> we should automatically copy enough pseudo-entropy bits to /dev/rnd
> to prevent future blocking.

For most architectures, sysinst does do exactly that. It assumes that
you don't just reset or reboot, but properly shutdown the system.

Joerg

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Mon, 05 Apr 2021 00:14:30 +0200 (CEST), Havard Eidnes  
wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> > What about architectures that have nothing like RDRAND/RDSEED?  Are
> > they, effectively, totally unsupported now?
>
> Nope, not entirely.  But they have to be seeded once.  If they
> have storage which survives reboots, and entropy is saved and
> restored on reboot, they will be ~fine.

BTW, to me reusing the same entropy on every reboot seems less secure.

> Systems without persistent storage and also without RDRAND/RDSEED
> will however be ... a more challenging problem.

Leaving things like that would be totally silly.

With my patch the old way of gathering entropy from devices works just
fine as it always did, albeit with the second patch it does require a
tiny bit of extra configuration.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpgeBbtqrqWg.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Mon, 05 Apr 2021 00:07:49 +0200 (CEST), Havard Eidnes  
wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> Indeed, that's also compatible with what I wrote.  The samples
> from whatever sources you have are still being mixed into the
> pool, but they are not being counted as contributing to the
> entropy estimate, because the quality of the samples is at best
> unknown.

Perhaps we're talking past each other?

Until I made the fix no amount of time or activity or of me telling the
system to make use of the driver inputs was unblocking getrandom(2) or
/dev/random, so it doesn't really matter if anything was being "mixed
into the pool" so to speak as the pool was empty.

> A possible workaround is, once you have some uptime and some bits
> mixed into the pool, you can do:

I don't need a work-around -- I found a fix.  I corrected some code that
was purposefully ignoring my orders for how it should behave.

> I am still of the fairly firm beleif that the mistrust in the
> hardware vendors' ability to make a reasonable and robust
> implementation is without foundation.

Well there are still millions of systems out there without the fancy
newer hardware RNGs available to make them more secure than Fort Knox.
At least a small handful of them run NetBSD for me, and want them to
work for my needs and I was, and am, quite happy with using entropy that
can be collected from various devices that my systems (virtual and real)
actually have.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpw8NF4N8YCU.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Sun, 4 Apr 2021 16:39:11 -0400 (EDT), Mouse  
wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> > No amount of uptime and activity was increasing the entropy in my
> > system before I patched it.
>
> As I understand it, entropy was being contributed.  What wasn't
> happening was the random driver code recognizing and acknowledging that
> entropy, because it had no way to tell how much of it there really was.

Clearly there was no entropy being contributed in any way shape or form.

It wasn't the driver code at fault.

It was the code I fixed with my patch that was at fault.

I told the system to "count" the entropy being gathered by the
appropriate driver(s), but it was being ignored entirely.

After my fix the system behaved as I told it to.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpKRv3dDs3Kt.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Havard Eidnes]

>> My question is, how can we tell what random sources a system actually
>> has, i.e. is there some flag that cpuctl identify shows when a system
>> has RDRAND/RDSEED?
>
> What about architectures that have nothing like RDRAND/RDSEED?  Are
> they, effectively, totally unsupported now?

Nope, not entirely.  But they have to be seeded once.  If they
have storage which survives reboots, and entropy is saved and
restored on reboot, they will be ~fine.

Systems without persistent storage and also without RDRAND/RDSEED
will however be ... a more challenging problem.

Regards,

- Håvard

Re: regarding the changes to kernel entropy gathering

[Havard Eidnes]

>> Do note, the existing randomness sources are still being sampled and
>> mixed into the pool, so even if the starting state from the saved
>> entropy may be known (by violating the security of the storage),
>> it's still not possible to predict the complete stream of randomness
>> data once the system has seen a bit of uptime (given that there are
>> actual other sources of (unverified) entropy which aren't all of too
>> low quality).
>
> No amount of uptime and activity was increasing the entropy in my system
> before I patched it.

Indeed, that's also compatible with what I wrote.  The samples
from whatever sources you have are still being mixed into the
pool, but they are not being counted as contributing to the
entropy estimate, because the quality of the samples is at best
unknown.

> The unpatched implementation completely and entirely prevents
> the system from ever using any of those sources, despite
> showing that they are enabled for use.

As far as I know, those samples are still being *used*, but as
said above, not counted as contributing to the entropy count.

A possible workaround is, once you have some uptime and some bits
mixed into the pool, you can do:

% su
# dd if=/dev/urandom of=/dev/random count=1

If you then ensure that the entropy is saved and restored on
reboot, as is typically done, after this initialization, neither
/dev/random nor /dev/urandom will block(!).  Ref. the attachment.

>> Besides, the implementation has been thoroughly vetted.  E.g. the
>> reference [7] from the wikipedia article states in the conclusion on
>> page 20
>>
>>Overall, the Ivy Bridge RNG is a robust design with a large
>>margin of safety that ensures good random data is generated even
>>if the Entropy Source is not operating as well as predicted.
>
> "design" != implementation

Well, if I'm not mistaken, the actual implementation was tested,
not just a theoretical study of the design.  And, as I said,
thermal noise is one of the well-known physical systems which
provide actual entropy.

I am still of the fairly firm beleif that the mistrust in the
hardware vendors' ability to make a reasonable and robust
implementation is without foundation.

Regards,

- Håvard
b# uname -a
NetBSD b.urc.uninett.no 9.99.81 NetBSD 9.99.81 (GENERIC) #0: Sat Apr  3 
23:24:06 UTC 2021  
mkre...@mkrepro.netbsd.org:/usr/src/sys/arch/i386/compile/GENERIC i386
b# rndctl -l
Source Bits Type  Flags
/dev/random   0 ???  estimate, collect, v
wd0   0 disk estimate, collect, v, t, dt
cd0   0 disk estimate, collect, v, t, dt
cpu0  0 vm   estimate, collect, v, t, dv
hardclock 0 skew estimate, collect, t
bge0  0 net  estimate, v, t, dt
system-power  0 power estimate, collect, v, t, dt
autoconf  0 ???  estimate, collect, t
seed256 ???  estimate, collect, v
b# rndctl -s
0 bits mixed into pool
  256 bits currently stored in pool (max 256)
0 bits of entropy discarded due to full pool
0 hard-random bits generated
0 pseudo-random bits generated
b# dd if=/dev/random of=/dev/null count=1024
0+1024 records in
64+0 records out
32768 bytes transferred in 0.021 secs (1560380 bytes/sec)
b# rndctl -s
0 bits mixed into pool
  256 bits currently stored in pool (max 256)
0 bits of entropy discarded due to full pool
0 hard-random bits generated
0 pseudo-random bits generated
b#
b# rndctl -lv
Source Bits Type  Flags
/dev/random   0 ???  estimate, collect, v
Dt samples = 0
Dt bits = 0
Dv samples = 0
Dv bits = 0
wd0   0 disk estimate, collect, v, t, dt
Dt samples = 5559
Dt bits = 0
Dv samples = 5869
Dv bits = 0
cd0   0 disk estimate, collect, v, t, dt
Dt samples = 0
Dt bits = 0
Dv samples = 0
Dv bits = 0
cpu0  0 vm   estimate, collect, v, t, dv
Dt samples = 183
Dt bits = 0
Dv samples = 183
Dv bits = 0
hardclock 0 skew estimate, collect, t
Dt samples = 853
Dt bits = 0
Dv samples = 0
Dv bits = 0
bge0  0 net  estimate, v, t, dt
Dt samples = 0
Dt bits = 0
Dv samples = 0
Dv bits = 0
system-power  0 power estimate, collect, v, t, dt
Dt samples = 0
Dt bits = 0
Dv samples = 0
Dv bits = 0
autoconf  0 ???  estimate, collect, t
Dt samples = 43
Dt bits = 0
Dv samples = 0
Dv bits = 0
seed256 ???  estimate, collect, v
Dt samples = 0
Dt bits = 0
Dv samples = 1
Dv bits = 256
b#

Re: regarding the changes to kernel entropy gathering

[Paul Goyette]

Personally, I'm happy with anything that your average high school
student is unlikely to be able to crack in an hour.   I don't run
a bank, or a military installation, and I'm not the NSA.   If someone
is prepared to put in the effort required to break into my systems,
then let them, it isn't worth the cost to prevent that tiny chance.
That's the same way that my house has ordinary locks - I'm sure they
can be picked by someone who knows what they're doing, and better
security is available, at a price, but a nice happy medium is what
fits me best.


FWIW, I used to work for a company whose marketing motto was

Good enough isn't!

But I definitely agree with you - what we used to have is "good
enough" for the vast bulk of our users and potential users.

Perhaps sysinst(8) should ask

Do you need a hyper-secure system?

If yes, then leave things as they are today.  But if you answer no,
we should automatically copy enough pseudo-entropy bits to /dev/rnd
to prevent future blocking.



++--+---+
| Paul Goyette   | PGP Key fingerprint: | E-mail addresses: |
| (Retired)  | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org   |
++--+---+

re: UVM behavior under memory pressure

[matthew green]

one additional thing about the behaviour of vm.*{min,max}
is that they do not (currently?) consider how much memory
is consumed by major kernel consumers like pools.  eg, there
is a known radeondrmkms leak we have not figured out why it
happens only sometimes, and this ends up making the kmem-160
or kmem-192 (size depends on DIAGNOSTIC, IIRC) pool grow,
possibly using serious percentage of the total memory, which
can end up meaning that adding vm.*min to 95% actually means
way more than 100% of available memory.

so, also check what "vmstat -m" says pools are using.

(we should fix this.  anyone want to define what the right
solution is? :-)


.mrg.

Re: regarding the changes to kernel entropy gathering

[Mouse]

> No amount of uptime and activity was increasing the entropy in my
> system before I patched it.

As I understand it, entropy was being contributed.  What wasn't
happening was the random driver code recognizing and acknowledging that
entropy, because it had no way to tell how much of it there really was.

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: regarding the changes to kernel entropy gathering

[Mouse]

> My question is, how can we tell what random sources a system actually
> has, i.e. is there some flag that cpuctl identify shows when a system
> has RDRAND/RDSEED?

What about architectures that have nothing like RDRAND/RDSEED?  Are
they, effectively, totally unsupported now?

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Sun, 04 Apr 2021 21:14:31 +0200 (CEST), Havard Eidnes  
wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> Do note, the existing randomness sources are still being sampled and
> mixed into the pool, so even if the starting state from the saved
> entropy may be known (by violating the security of the storage),
> it's still not possible to predict the complete stream of randomness
> data once the system has seen a bit of uptime (given that there are
> actual other sources of (unverified) entropy which aren't all of too
> low quality).

No amount of uptime and activity was increasing the entropy in my system
before I patched it.  /dev/random remained blocked after days of busy
system activity.  I would argue that most, if not all, of the sources of
entropy identified by rndctl(8) on my systems are high-quality and
secure sources in my circumstances and for my uses.

Perhaps the unpatched implementation isn't doing exactly what you think
it is?

The unpatched implementation completely and entirely prevents the system
from ever using any of those sources, despite showing that they are
enabled for use.

> However, in the new scheme of things, because most of the
> traditional sources have unknown quality, and we have no reliable
> method to estimate how much "actual entropy" those sources
> provide, they no longer count towards the *estimate* of what is
> now a lower bound on the "real" entropy available in the pool.

It really doesn't matter what can be determined in general and from a
distance.

What matters is what a given administrator can determine in particular
for a given application in a given circumstance.

Before my patch the system was not behaving as documented and could not
be made to behave as the documentation said it could be made to behave.

With my patch I can choose which to trust from amongst the available
sources.  Without that patch my choices are ignored and the system lies
to me about using my choices.  I would argue my patch fixes a critical
bug.

> Besides, the implementation has been thoroughly vetted.  E.g. the
> reference [7] from the wikipedia article states in the conclusion on
> page 20
>
>Overall, the Ivy Bridge RNG is a robust design with a large
>margin of safety that ensures good random data is generated even
>if the Entropy Source is not operating as well as predicted.

"design" != implementation

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpjs3QaPXmot.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Sun, 04 Apr 2021 23:47:10 +0700, Robert Elz  wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> If we want really good security, I'd submit we need to disable
> the random seed file, and RDRAND (and anything similar) until we
> have proof that they're perfect.

Indeed, I concur.

I trust the randomness and in-observability and isolation of the
behaviour of my system's fans far more than I would trust Intel's RDRAND
or RDSEED instructions.

I even trust the randomness of the timings of the virtual disks in my
Xen domU virtual machines more-so, even with multiple sibling guests,
even if some of those other guests can be influenced by untrusted third
parties at critical times.

> Personally, I'm happy with anything that your average high school
> student is unlikely to be able to crack in an hour.   I don't run
> a bank, or a military installation, and I'm not the NSA.   If someone
> is prepared to put in the effort required to break into my systems,
> then let them, it isn't worth the cost to prevent that tiny chance.
> That's the same way that my house has ordinary locks - I'm sure they
> can be picked by someone who knows what they're doing, and better security
> is available, at a price, but a nice happy medium is what fits me best.

Indeed again.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpvuqMttwSyI.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Greg A. Woods]

At Sun, 4 Apr 2021 09:49:58 +, Taylor R Campbell  
wrote:
Subject: Re: regarding the changes to kernel entropy gathering
>
> > Date: Sat, 03 Apr 2021 12:24:29 -0700
> > From: "Greg A. Woods" 
> >
> > Updating a system, even on -current, shouldn't create a long-lived
> > situation where the system documentation and the behaviour and actions
> > of system commands is completely out of sync with the behaviour of the
> > kernel, and in fact lies to the administrator about the abilities of the
> > system.
>
> It would help if you could identify specifically what you are calling
> a lie.
>
> > @@ -1754,21 +1766,21 @@
> >  rnd_add_uint32(struct krndsource *rs, uint32_t value)
> >  {
> >
> > -   rnd_add_data(rs, , sizeof value, 0);
> > +   rnd_add_data(rs, , sizeof value, sizeof value * ABBY);
> >  }
>
> The rnd_add_uint32 function is used by drivers to feed in data from
> sources _with no known model for their entropy_.

Indeed -- that's the idea.

> It's how drivers
> toss in data that might be helpful but might totally predictable, and
> the driver has no way to know.

Yeah, so?  They don't need to know this.  I'm not actually asking random
drivers to decide the amount of physical entropy they can collect.
That is controlled elsewhere.

> Your change _creates_ the lie that every bit of data entered this way
> is drawn from a source with independent uniform distribution.

No, my change _allows_ the administrator to decide which devices can be
used as estimating/counting entropy sources.  For example I know that
many of the devices on almost all of my machines (virtual or otherwise)
are equally good sources of entropy for their uses.

An addition change, one which I would also find totally acceptable,
would be to disable the current default of allowing "estimation" on
devices which are not true hardware RNGs.  I.e. maybe this simple change
would suffice (though I haven't checked beyond a quick grep to see that
this flag is the mostly commonly used one -- perhaps some real RNG
devices could also be changed to use explicit flags to enable estimation
by default):

--- sys/sys/rndio.h.~1.2.~  2016-07-23 14:36:45.0 -0700
+++ sys/sys/rndio.h 2021-04-04 12:39:15.609936311 -0700
@@ -91,8 +91,7 @@
 #define RND_FLAG_ESTIMATE_TIME 0x4000  /* estimate entropy on time */
 #define RND_FLAG_ESTIMATE_VALUE0x8000  /* estimate entropy on 
value */
 #defineRND_FLAG_HASENABLE  0x0001  /* has enable/disable 
fns */
-#define RND_FLAG_DEFAULT   (RND_FLAG_COLLECT_VALUE|RND_FLAG_COLLECT_TIME|\
-RND_FLAG_ESTIMATE_TIME)
+#define RND_FLAG_DEFAULT   (RND_FLAG_COLLECT_VALUE|RND_FLAG_COLLECT_TIME)

 #defineRND_TYPE_UNKNOWN0   /* unknown source */
 #defineRND_TYPE_DISK   1   /* source is physical disk */


There are a vast number of ways this re-tooling of entropy collection
could have been done better.

I'm asking for discussion on what amount to some VERY simple changes
which completely and totally solve many real-world uses of this code
while at the same time not just allowing, but defaulting to, the very
strict and secure operation for special situations.

--
Greg A. Woods 

Kelowna, BC +1 250 762-7675   RoboHack 
Planix, Inc.  Avoncote Farms 


pgpXj_p1tBVqr.pgp
Description: OpenPGP Digital Signature

Re: regarding the changes to kernel entropy gathering

[Havard Eidnes]

> Is that file encrypted?

As I understand it, no.

> I think I'd prefer possibly insecure, but difficult to obtain from outside
> like disk drive interrupt timing low order bits than that.   Regardless of
> how unproven that method might be.

Do note, the existing randomness sources are still being sampled and
mixed into the pool, so even if the starting state from the saved
entropy may be known (by violating the security of the storage),
it's still not possible to predict the complete stream of randomness
data once the system has seen a bit of uptime (given that there are
actual other sources of (unverified) entropy which aren't all of too
low quality).

However, in the new scheme of things, because most of the
traditional sources have unknown quality, and we have no reliable
method to estimate how much "actual entropy" those sources
provide, they no longer count towards the *estimate* of what is
now a lower bound on the "real" entropy available in the pool.

> Lastly, why would anyone presume that RDRAND generates less predictable
> bits (less predictable to someone who knows how it works) than any of
> the other methods that are used.

Looking at

  https://en.wikipedia.org/wiki/RDRAND

and the reference [3] at

  
http://software.intel.com/sites/default/files/m/d/4/1/d/8/441_Intel_R__DRNG_Software_Implementation_Guide_final_Aug7.pdf

reveals that the on-chip entropy source samples thermal noise on
the chip, ref. page 12 where it says:

   The Entropy Source runs asynchronously on a self-timed circuit
   and uses thermal noise within the silicon to output a random
   stream of bits at the rate of 3GHz.

That bitstream is then fed through a "AES-CBC-MAC" based
conditioner and an AES-CTR based deterministic random bit
generator, before the result is given to the user via RDRAND.

If I'm not very much mistaken, thermal noise is one of the well-
known physical sources of actual entropy.

Besides, the implementation has been thoroughly vetted.  E.g. the
reference [7] from the wikipedia article states in the conclusion on
page 20

   Overall, the Ivy Bridge RNG is a robust design with a large
   margin of safety that ensures good random data is generated even
   if the Entropy Source is not operating as well as predicted.

Personally, I as a non-expert can't find anything which seems
overly worrisome with RDRAND.

Best regards,

- Håvard

Re: regarding the changes to kernel entropy gathering

[Martin Husemann]

On Sun, Apr 04, 2021 at 11:14:31AM -0700, John Nemeth wrote:
>  I understand the need for good random sources, and won't argue
> it.  My question is, how can we tell what random sources a system
> actually has, i.e. is there some flag that cpuctl identify shows
> when a system has RDRAND/RDSEED?  Are there other sources that can
> be positively identified as providing randomness?

I am not sure I understand the question correctly. rndctl will show the
entropy available from the source. For cpu internal random sources,
on x86 cpuctl identify shows them:

# cpuctl identify 0 | fgrep RDRAND
cpu0: features1 0x7ffafbff
# cpuctl identify 0 | fgrep RDSEED
cpu0: features5 0x29c6fbf
# rndctl -l
Source Bits Type  Flags
[..]
system-power  0 power estimate, collect, v, t, dt
autoconf  0 ???  estimate, collect, t
seed256 ???  estimate, collect, v
rdrand/rdseed   512 rng  estimate, collect, v


or on a system with another cpu:

# rndctl -l
Source Bits Type  Flags
[..]
ualea0  512 rng  estimate, collect, v
[..]
system-power  0 power estimate, collect, v, t, dt
autoconf  0 ???  estimate, collect, t
seed256 ???  estimate, collect, v


and yet another with no trusted source, but properly seeded (also an
old and non-x86 system):

# rndctl -l
Source Bits Type  Flags
/dev/random   0 ???  estimate, collect, v
ucom7 0 tty  estimate, collect, v, t, dt
ucom6 0 tty  estimate, collect, v, t, dt
ucom5 0 tty  estimate, collect, v, t, dt
ucom4 0 tty  estimate, collect, v, t, dt
ucom3 0 tty  estimate, collect, v, t, dt
ucom2 0 tty  estimate, collect, v, t, dt
ucom1 0 tty  estimate, collect, v, t, dt
ucom0 0 tty  estimate, collect, v, t, dt
sd1   0 disk estimate, collect, v, t, dt
sd0   0 disk estimate, collect, v, t, dt
cpu0  0 vm   estimate, collect, v, t, dv
hardclock 0 skew estimate, collect, t
mvgbe00 net  estimate, v, t, dt
system-power  0 power estimate, collect, v, t, dt
autoconf  0 ???  estimate, collect, t
seed256 ???  estimate, collect, v


Martin

Re: regarding the changes to kernel entropy gathering

[Thomas Klausner]

On Sun, Apr 04, 2021 at 11:14:31AM -0700, John Nemeth wrote:
>  I understand the need for good random sources, and won't argue
> it.  My question is, how can we tell what random sources a system
> actually has, i.e. is there some flag that cpuctl identify shows
> when a system has RDRAND/RDSEED?  Are there other sources that can
> be positively identified as providing randomness?

That should do it:

# cpuctl identify 0 | grep -e RDRAND -e RDSEED 
cpu0: features1 0x7ed8320b
cpu0: features5 0x209c01a9

 Thomas

Re: regarding the changes to kernel entropy gathering

[John Nemeth]

On Apr 4,  9:49, Taylor R Campbell wrote:
} 
} What NetBSD-current is telling you on your Xen system, on a CPU
} predating RDRAND/RDSEED, is the unfortunate truth that there is no
} reliable source of entropy available in your system -- annoying, yes,
} but when you talk about `matters so important as system security and
} integrity' you might prefer to hear about this rather than have it
} swept under the rug.

 I understand the need for good random sources, and won't argue
it.  My question is, how can we tell what random sources a system
actually has, i.e. is there some flag that cpuctl identify shows
when a system has RDRAND/RDSEED?  Are there other sources that can
be positively identified as providing randomness?

}-- End of excerpt from Taylor R Campbell

Re: regarding the changes to kernel entropy gathering

[Robert Elz]

Date:Sun, 4 Apr 2021 15:28:13 +
From:Taylor R Campbell 
Message-ID:  <20210404152814.3c56360...@jupiter.mumble.net>

  | you can let NetBSD take care of it automatically
  | on subsequent boots by running `/etc/rc.d/random_seed stop' to save a
  | seed to disk.)

Is that file encrypted?   If it is, where does the decryption key come from?

If not, what prevents someone from reading (copying) the file from the
system while it is stopped (assessing the storage device via other methods)
and then knowing exactly what the seed is going to be when the system boots?

I think I'd prefer possibly insecure, but difficult to obtain from outside
like disk drive interrupt timing low order bits than that.   Regardless of
how unproven that method might be.

And what's the scheme for cheap low-end devices that have no writable storage?
(The proverbial internet toaster, for example).

Lastly, why would anyone presume that RDRAND generates less predictable
bits (less predictable to someone who knows how it works) than any of
the other methods that are used.   After all, all the chips are more or
less identical, what about them can absolutely guarantee unpredictable
data (a very rare thing for computers) and how can anyone be certain
that it has been correctly implemented without any bugs?

If we want really good security, I'd submit we need to disable
the random seed file, and RDRAND (and anything similar) until we
have proof that they're perfect.

Personally, I'm happy with anything that your average high school
student is unlikely to be able to crack in an hour.   I don't run
a bank, or a military installation, and I'm not the NSA.   If someone
is prepared to put in the effort required to break into my systems,
then let them, it isn't worth the cost to prevent that tiny chance.
That's the same way that my house has ordinary locks - I'm sure they
can be picked by someone who knows what they're doing, and better security
is available, at a price, but a nice happy medium is what fits me best.

kre

Re: regarding the changes to kernel entropy gathering

[Taylor R Campbell]

> Date: Sun, 4 Apr 2021 10:40:22 -0400 (EDT)
> From: Mouse 
> 
> > What NetBSD-current is telling you on your Xen system, on a CPU
> > predating RDRAND/RDSEED, is the unfortunate truth that there is no
> > reliable source of entropy available in your system --
> 
> Not quite.  That there is nothing which NetBSD, independent of the
> sysadmin, is confident is a reliable source of entropy.
> 
> It's entirely possible one or more of those sources actually does
> supply usable entropy, but NetBSD doesn't realize that (and, as I
> understand it, provides no way for the sysadmin to fix that, short of
> hacking on the source).

If the sysadmin knows something NetBSD doesn't, it is easy for the
sysadmin to convince NetBSD to unblock by writing 32 bytes to
/dev/random as root.  No need to hack any source.

(This shouldn't be an automatic recipe, though, because it depends on
specific knowledge of the system in question which the authors of the
device drivers and the rest of the software didn't know.  And if you
have done that once, you can let NetBSD take care of it automatically
on subsequent boots by running `/etc/rc.d/random_seed stop' to save a
seed to disk.)

Re: regarding the changes to kernel entropy gathering

[Mouse]

> What NetBSD-current is telling you on your Xen system, on a CPU
> predating RDRAND/RDSEED, is the unfortunate truth that there is no
> reliable source of entropy available in your system --

Not quite.  That there is nothing which NetBSD, independent of the
sysadmin, is confident is a reliable source of entropy.

It's entirely possible one or more of those sources actually does
supply usable entropy, but NetBSD doesn't realize that (and, as I
understand it, provides no way for the sysadmin to fix that, short of
hacking on the source).

/~\ The ASCII Mouse
\ / Ribbon Campaign
 X  Against HTMLmo...@rodents-montreal.org
/ \ Email!   7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: regarding the changes to kernel entropy gathering

[nia]

I have updated the rndctl(8) documentation so it reflects the current
model in the kernel and is no longer misleading.

It could still use some extra work (e.g. -l could print number of
samples collected).

On Sat, Apr 03, 2021 at 10:03:21PM +0200, Steffen Nurpmeso wrote:
> Btw i track 
> 
>   https://github.com/smuellerDD/jitterentropy-library.git
> 
> for about two years, and i "never" (which is a couple of years at
> least) understood why something like this isn't simply used.

The kernel already mixes in hardclock jitter, among other things,
to the entropy pool, but it doesn't increase the entropy estimate.
Nothing does except hardware RNGs and direct entropy fed from the
host machine, and there have been strong objections to re-introducing
any kind of estimator that is based purely on runtime statistical
gathering rather than models of hardware.

"Potential entropy gathering schemes" have been discussed to 
death. Please stay on-topic.

Re: regarding the changes to kernel entropy gathering

[Taylor R Campbell]

> Date: Sat, 03 Apr 2021 12:24:29 -0700
> From: "Greg A. Woods" 
> 
> Updating a system, even on -current, shouldn't create a long-lived
> situation where the system documentation and the behaviour and actions
> of system commands is completely out of sync with the behaviour of the
> kernel, and in fact lies to the administrator about the abilities of the
> system.

It would help if you could identify specifically what you are calling
a lie.

> @@ -1754,21 +1766,21 @@
>  rnd_add_uint32(struct krndsource *rs, uint32_t value)
>  {
> 
> - rnd_add_data(rs, , sizeof value, 0);
> + rnd_add_data(rs, , sizeof value, sizeof value * NBBY);
>  }

The rnd_add_uint32 function is used by drivers to feed in data from
sources _with no known model for their entropy_.  It's how drivers
toss in data that might be helpful but might totally predictable, and
the driver has no way to know.

Your change _creates_ the lie that every bit of data entered this way
is drawn from a source with independent uniform distribution.

What NetBSD-current is telling you on your Xen system, on a CPU
predating RDRAND/RDSEED, is the unfortunate truth that there is no
reliable source of entropy available in your system -- annoying, yes,
but when you talk about `matters so important as system security and
integrity' you might prefer to hear about this rather than have it
swept under the rug.

What your patch does is shoot yourself in the foot by fantasizing that
_every_ source using rnd_add_uint32, even if it predictably always
supplies all-zero bits, has the maximum entropy possible.