Bug#923675: debian-installer: consider using haveged to gather entropy

[Cyril Brulebois]

Holger Levsen  (2019-04-22):
> heh. what was the reason haveged was choosen and not
> jitterentropy-rngd which was also suggested here?

I have enough things to keep me busy; if the first one I look at can be
turned into something useful in d-i, seems to have reasonable integration
and maintenance costs, and has a maintainer who is happy to let me modify
the packaging accordingly, then it's unlikely I'll spend more time looking
at an alternative.

Not that I wouldn't appreciate such a comparison in theory; in practice,
time is limited. If others want to compare both, feel free to.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#923675: debian-installer: consider using haveged to gather entropy

[Holger Levsen]

Hi Cyril,

On Sat, Apr 20, 2019 at 11:28:23PM +0200, Cyril Brulebois wrote:
> > does that also mean that haveged get's installed on the final system if
> > it's deemed to be useful in d-i or is that still missing?
> There's nothing in what I have written (on this bug report or in the
> code I've quoted or pointed to) that references /target, no.

this was my understanding as well, though I wasn't sure (havent reviewed
the code), thats why I asked, so thanks for pointing this out! (and for
the other feedback as well!)

> TBF I have no idea whether we should do that; the situation is slightly
> different as a non-installer/non-live system can carry over entropy from
> one boot to the next one, which d-i can't do.

right.

> I've focussed on getting entropy issues within d-i fixed, which seemed
> urgently needed. I'm fine with people seeking a consensus through
> debian-boot@ (and maybe debian-devel@) regarding what should happen in
> the installed system.

ok, cool.

though I don't think I have time/energy to drive this discussion right
now :/

> (I almost mentioned the fix would be trivial as it's about pulling an
> extra package, but since we have no rng support in udebs at the moment,
> we would have no rng support in d-i thus haveged running, while the
> installed system could have rng support… Anyway: deciding what to do is
> the important part; implementation should be much more straightforward
> than the haveged udeb addition dance I've just orchestrated.)

heh. what was the reason haveged was choosen and not jitterentropy-rngd
which was also suggested here?


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Bug#923675: debian-installer: consider using haveged to gather entropy

[Cyril Brulebois]

Hi Holger,

Holger Levsen  (2019-04-20):
> On Sat, Apr 20, 2019 at 02:39:49AM +0200, Cyril Brulebois wrote:
> > I've tweaked it a little so that we log whether haveged is available,
> > and whether it should be started, in case we need to investigate:
> >   
> > https://salsa.debian.org/installer-team/rootskel/blob/master/src/lib/debian-installer-startup.d/S50entropy-source
> 
> nice work!
> 
> does that also mean that haveged get's installed on the final system if
> it's deemed to be useful in d-i or is that still missing?

There's nothing in what I have written (on this bug report or in the
code I've quoted or pointed to) that references /target, no.


TBF I have no idea whether we should do that; the situation is slightly
different as a non-installer/non-live system can carry over entropy from
one boot to the next one, which d-i can't do.

I've focussed on getting entropy issues within d-i fixed, which seemed
urgently needed. I'm fine with people seeking a consensus through
debian-boot@ (and maybe debian-devel@) regarding what should happen in
the installed system.

(I almost mentioned the fix would be trivial as it's about pulling an
extra package, but since we have no rng support in udebs at the moment,
we would have no rng support in d-i thus haveged running, while the
installed system could have rng support… Anyway: deciding what to do is
the important part; implementation should be much more straightforward
than the haveged udeb addition dance I've just orchestrated.)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#923675: debian-installer: consider using haveged to gather entropy

[Holger Levsen]

On Sat, Apr 20, 2019 at 02:39:49AM +0200, Cyril Brulebois wrote:
> I've tweaked it a little so that we log whether haveged is available,
> and whether it should be started, in case we need to investigate:
>   
> https://salsa.debian.org/installer-team/rootskel/blob/master/src/lib/debian-installer-startup.d/S50entropy-source

nice work!

does that also mean that haveged get's installed on the final system if
it's deemed to be useful in d-i or is that still missing?


-- 
tschau,
Holger

---
   holger@(debian|reproducible-builds|layer-acht).org
   PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C


signature.asc
Description: PGP signature

Bug#923675: debian-installer: consider using haveged to gather entropy

[Cyril Brulebois]

Control: tag -1 patch pending

Hi,

Ben Hutchings  (2019-04-17):
> Ideally it would only be used if there isn't a hardware RNG available.
> Currently we don't include any hardware RNG modules in udebs, but that
> can be changed.  So please first check that:
> 
> * /sys/devices/virtual/misc/hw_random/rng_current is absent or
>   contains "none"
> * (x86 only) /proc/cpuinfo does not mention rdrand (I can't find an
>   arch-independent way to check for this, and Linux doesn't yet
>   support an equivalent feature on any other architecture)
> 
> Something like this should work:
> 
> if [ "$(cat /sys/devices/virtual/misc/hw_random/rng_current 2>/dev/null || 
> echo none)" = none ] \
>&& ! grep -q '^flags\b.*\brdrand\b' /proc/cpuinfo; then
> # use software entropy daemon
> fi

Many thanks for your input and for the suggested implementation.

I've tweaked it a little so that we log whether haveged is available,
and whether it should be started, in case we need to investigate:
  
https://salsa.debian.org/installer-team/rootskel/blob/master/src/lib/debian-installer-startup.d/S50entropy-source


I think I've tested all cases:
 - when haveged-udeb hasn't been added to src:debian-installer's
   pkg-lists yet
 - with the default Skylake-Client in libvirt, which leads to an rdrand
   CPU flag;
 - with a core2duo CPU instead, which has no such flag;
 - with the same CPU, but with a VirtIO RNG enabled, and those extra
   kernel modules in my netboot-gtk image:
 lib/modules/4.19.0-4-amd64/kernel/drivers/char/hw_random/rng-core.ko
 lib/modules/4.19.0-4-amd64/kernel/drivers/char/hw_random/virtio-rng.ko
 lib/modules/4.19.0-4-amd64/kernel/drivers/virtio/virtio.ko
 lib/modules/4.19.0-4-amd64/kernel/drivers/virtio/virtio_ring.ko
   which leads to a virtio_rng.0 in …/hw_random/rng_current.


So I've just uploaded a new version of rootskel (1.129), and pushed a
new commit to debian-installer:
  
https://salsa.debian.org/installer-team/debian-installer/commit/c470001925d067b42cdf613339634f4d54ed01b6

The haveged-udeb addition was already uploaded and also ACCEPTED from
NEW. I'll keep an eye on the daily builds.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#923675: debian-installer: consider using haveged to gather entropy

[Ben Hutchings]

On Tue, 2019-04-16 at 23:45 +0200, Cyril Brulebois wrote:
[...]
> My initial thought would be to launch it on demand when one is about to
> get to wget calls that needs HTTPS; but we could probably benefit from
> it in case HTTP is requested but redirections to HTTPS happens… There
> are also the obvious keypair generations mentioned above. But then over
> time maybe some other operations could be needing entropy (the
> cryptsetup case is discussed in a separate thread[1]).
> 
>  1. https://lists.debian.org/debian-boot/2019/04/msg00153.html
> 
> So it might be best to start it unconditionally at start-up?

Ideally it would only be used if there isn't a hardware RNG available.
Currently we don't include any hardware RNG modules in udebs, but that
can be changed.  So please first check that:

* /sys/devices/virtual/misc/hw_random/rng_current is absent or
  contains "none"
* (x86 only) /proc/cpuinfo does not mention rdrand (I can't find an
  arch-independent way to check for this, and Linux doesn't yet
  support an equivalent feature on any other architecture)

Something like this should work:

if [ "$(cat /sys/devices/virtual/misc/hw_random/rng_current 2>/dev/null || echo 
none)" = none ] \
   && ! grep -q '^flags\b.*\brdrand\b' /proc/cpuinfo; then
# use software entropy daemon
fi

Ben.

-- 
Ben Hutchings
Make three consecutive correct guesses and you will be considered
an expert.



signature.asc
Description: This is a digitally signed message part

Bug#923675: debian-installer: consider using haveged to gather entropy

[Jonathan Carter]

On 2019/04/16 23:45, Cyril Brulebois wrote:
> I'm no cryptographer so I cannot judge haveged from that angle.

Ditto here, but...

> But from a /proc/sys/kernel/random/entropy_avail standpoint, starting
> the haveged daemon inside d-i, a couple of screens after the graphical
> installer start-up, I'm getting a bump from ~150 to ~2500.
> 
> This needs to be polished before submitting the addition of haveged-udeb
> and of course proper integration needs to happen, with real tests… For
> wget, we're hitting #926315, but it was luckily closed a couple hours
> ago; arm devices that need so much time to generate a keypair should get
> a nice improvement…

Yeah debian-live was unusable without haveged (as in, some sessions
wouldn't start up for hours unless users pounded on the keyboard for a
while). Some people quickly get hand-wavy about haveged, but it seems
like the theory of how it works is reasonably solid and I really tried
to find evidence of it being harmful or not generating enough randomness
in typical use cases, but couldn't find anything, so we went ahead and
included it in the live media and it seems to work for us there.

Debian's official documentation probably just needs a section explaining
what haveged is and that if someone needs to create a mass amount of
keys for commercial applications or such then it's really recommended
that they get a decent hardware RNG or use an external service to seed that.

-Jonathan

Bug#923675: debian-installer: consider using haveged to gather entropy

[Steve McIntyre]

On Tue, Apr 16, 2019 at 11:45:08PM +0200, Cyril Brulebois wrote:
>Cyril Brulebois  (2019-04-16):
>> The former was on my list of things to try; thanks for mentioning the
>> latter.

...

>My initial thought would be to launch it on demand when one is about to
>get to wget calls that needs HTTPS; but we could probably benefit from
>it in case HTTP is requested but redirections to HTTPS happens… There
>are also the obvious keypair generations mentioned above. But then over
>time maybe some other operations could be needing entropy (the
>cryptsetup case is discussed in a separate thread[1]).
>
> 1. https://lists.debian.org/debian-boot/2019/04/msg00153.html
>
>So it might be best to start it unconditionally at start-up?

I'd go with that, yes. What's the down-side?

I'm also pondering doing something similar with "udevadm monitor" -
start it unconditionally, logging to the installer syslog. It'd be a
good extra bit of debug to have.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Google-bait:   http://www.debian.org/CD/free-linux-cd
  Debian does NOT ship free CDs. Please do NOT contact the mailing
  lists asking us to send them to you.

Bug#923675: debian-installer: consider using haveged to gather entropy

[Cyril Brulebois]

Control: retitle -1 debian-installer: consider using haveged to gather entropy

Cyril Brulebois  (2019-04-16):
> The former was on my list of things to try; thanks for mentioning the
> latter.

I'm no cryptographer so I cannot judge haveged from that angle.

But from a /proc/sys/kernel/random/entropy_avail standpoint, starting
the haveged daemon inside d-i, a couple of screens after the graphical
installer start-up, I'm getting a bump from ~150 to ~2500.

This needs to be polished before submitting the addition of haveged-udeb
and of course proper integration needs to happen, with real tests… For
wget, we're hitting #926315, but it was luckily closed a couple hours
ago; arm devices that need so much time to generate a keypair should get
a nice improvement…


My initial thought would be to launch it on demand when one is about to
get to wget calls that needs HTTPS; but we could probably benefit from
it in case HTTP is requested but redirections to HTTPS happens… There
are also the obvious keypair generations mentioned above. But then over
time maybe some other operations could be needing entropy (the
cryptsetup case is discussed in a separate thread[1]).

 1. https://lists.debian.org/debian-boot/2019/04/msg00153.html

So it might be best to start it unconditionally at start-up?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature