Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Thu, 21 Sept 2023 at 09:24, Roberto A. Foglietta wrote: > > I consider this interface one of the brokest seen in the UNIX domain > ever. Despite this, I am reluctant in wishing that it would be > completely changed because breaking the back-compatibility with > 30years old production systems is something that goes beyond a survey. > Everything can be improved and enhanced, obviously and I hope this > will happen for this interface as well, in a smooth and > back-compatible way. > This, as long as we are writing about computer information technology, Linux in this case. If you ask me about society, my opinion is completely different. The reason is pretty simple: computers are "machines" and until the day they will develop an AGI and show to be a sentient self-conscious entityîes, they have no rights but just work to do - no matter how this work is broken or a burden. Put your chips on fire! When we talk about society, we are speaking about people, about human rights, about giving a meaning to our development and progress (aka society). The two concepts "technology" and "people" are not even comparable. I wrote this because I had the sensation that someone confused the two aspects. Some people never change their mind, whatever. While we can always reinstall an old machine to update it. The two concepts "people" and "technology" are way different in good and bad shapes. KISS works for IT, but rarely it works for people. etc. etc. - Just to clarify and to avoid that someone can leverage my writing to extend their meaning out of their specific context to influence other people's opinion or trying to understand my "under the hood" thinking. There is no doubt that I always was, I am and I will always be in favour of progress (a change for the better, a solid true improvement based on facts and not just opinions) when we are talking about people and society, no matter what, no matter who. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Sat, 23 Sept 2023 at 16:54, Roberto A. Foglietta wrote: > > On Sat, 23 Sept 2023 at 16:30, Steffen Nurpmeso wrote: > > > > Roberto A. Foglietta wrote in > > : > > |On Thu, 21 Sept 2023 at 20:05, Steffen Nurpmeso > > wrote: > > |>|IMHO, I vote for /sys rather than /proc/debug. The capability to > > > > There already _is_ a debugfs interface? > > Nothing which is interesting, AFAIK > However, I think that on the new hardware the entropy creation is not anymore an issue https://www.kernel.org/doc/html/latest/admin-guide/hw_random.html In both my laptop and smartphone on which I develop, the /dev/hwrng exists. Well, not totally solve the problem: http://main.lv/writeup/kernel_dev_hwrng.md <> Best regards, R- ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Sat, 23 Sept 2023 at 16:30, Steffen Nurpmeso wrote: > > Roberto A. Foglietta wrote in > : > |On Thu, 21 Sept 2023 at 20:05, Steffen Nurpmeso wrote: > |>|IMHO, I vote for /sys rather than /proc/debug. The capability to > > There already _is_ a debugfs interface? Nothing which is interesting, AFAIK redfishos:~ # find /sys -name rand\* /sys/class/mem/random /sys/devices/virtual/mem/random /sys/module/random redfishos:~ # cat /sys/class/mem/random/dev 1:8 redfishos:~ # cat /sys/devices/virtual/mem/random/uevent MAJOR=1 MINOR=8 DEVNAME=random DEVMODE=0666 redfishos:~ # cat /sys/module/random/parameters/ratelimit_disable 0 Until someone add a patch that creates something like entropy_pool_access_status (RW) cat: disable write append hashing (were hashing is like append but hashing before) entropy_pool_access (RW) cat: number of bytes of data, number of entropy bit > This must be new then. Just a couple more of parameters > I do not have debugfs no more, it is of no > use for me. Sorry if i have missed anything. You are welcome, do not worry about. Best regards, R- ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Roberto A. Foglietta wrote in : |On Thu, 21 Sept 2023 at 20:05, Steffen Nurpmeso wrote: |>|IMHO, I vote for /sys rather than /proc/debug. The capability to There already _is_ a debugfs interface? This must be new then. I do not have debugfs no more, it is of no use for me. Sorry if i have missed anything. ... |>|directly handle the system entropy pool should be an Admin privilege ... |> Access to the pool will never happen again i bet. | |Well, not by /dev/random or by /dev/urandom for a matter of |back-compatibility and I agree on this. Breaking well established |interfaces is not a good policy especially when an alternative is |possible. There was a pool access ioctl that was removed. Must i comment? I have no problem with that, even though one could say "i have a very good entropy generator" and wants to stir the pool her/himself. Direct I/O and even mapping of physical pages in userspace is possible / discussed, so why that not? Concurrent unlocked messing via binary ops surely improves the randomness (:-} ... |but quite cheap. At least for everyone that can pay for a top-gamma |smartphone ($1000-$2000) and with the same money can enter into HP |computing. Even the newest fairphone is only 70% fair iirc. No. Like the wonderful Zarah Leander sang in "Nur nicht aus Liebe weinen" (badly: [anything, but] not crying due to love): the answer is always renunciation. (That was, granted, under the Nazi regime.) (And, it counteracts a bit with Hans Albers, "Hoppla! Jetzt komm' ich", maybe.) But now clear the streets, for me! --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Thu, 21 Sept 2023 at 20:05, Steffen Nurpmeso wrote: > |IMHO, I vote for /sys rather than /proc/debug. The capability to > > There is /proc/sys/kernel/random. > > |directly handle the system entropy pool should be an Admin privilege > |even before being a debug option. As well as disable that handling > |with an > | > |echo 1 > /sys/driver/random/entropy_handle_disable > | > |In such a way that even Admin cannot do something like > | > |echo 1 > /sys/driver/random/entropy_pool_reset > | > |or > | > |dd if=/dev/zero count=128 of=/sys/driver/random/entropy_pool > ... > > Access to the pool will never happen again i bet. Well, not by /dev/random or by /dev/urandom for a matter of back-compatibility and I agree on this. Breaking well established interfaces is not a good policy especially when an alternative is possible. > And i have to concur that with the seedrng approach super fine- > grained and secure locking in a super-parallel super-fast async > boot process is possible. Mixing parallel output streams into a single one like a log daemon does, for a 4 or a 8 cores system is a great source of entropy. On the other side, you can argue that each line of those streams' line carries on a timestamp and this strictly enforces an order. As long as the timestamp is fine-grained, it supplies greatly the lack of arbitrary mixing where arbitrary means when the scheduler decides to switch to another task. > But i do not get his arguments, who writes during the boot, and so > much, and if, just change to /dev/random for credit (i mean, isn't > /dev/urandom the not-so-good thing anyhow). > But if it is not that (random), then a trigger like yours as in > /proc/sys/kernel/random/pool_is_seeded is not a bad idea. Thanks Steffen. After all, it is nothing more that a generalisation of an interface to inject errors which usually would stay in debugfs but because can also have a real-world production advantage - let the admin dealing with the entropy in a straightforward manner - it could be a great way to overcome the limitations of the ioctl() interface, in some advanced system (most of the system can be considered advanced by this definition). Recently I saw a commercial about a mother board carrying up to 8 SoC each of one with a 4-cores ARM CPU, for a total of 32 cores. That board fully equipped was selling for less than $1000. High parallel computing entrance barrier is not anymore as expensive as in the past, but quite cheap. At least for everyone that can pay for a top-gamma smartphone ($1000-$2000) and with the same money can enter into HP computing. Best regards, R- ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Roberto A. Foglietta wrote in
:
|On Thu, 21 Sept 2023 at 02:35, Steffen Nurpmeso wrote:
|>
|> Steffen Nurpmeso wrote in
|> <20230919222910.pn44y%stef...@sdaoden.eu>:
|>|Laurent Bercot wrote in
|>| :
|>||>|IIRC writing to /dev/urandom doesn't do what you want it to do.
|>||>|You have to use an ioctl() to actually set entropy.
|>||>
|>||>And that is the sad point about it.
|>||>Kernel hackers should stand up to allow it again!
|>||
|>|| As Ted Ts'o said[1], and Donenfeld agreed[2], the problem is that
|>||any user can write to /dev/urandom, including malicious users, so
|>||you cannot credit what they write.
|>|
|>|Well i was talking on unlocking the thing upon boot. Only root
|>|there is.
|>|
|>|And then you could easily look who is the writer, or require
...
|>|The thing is plain. Unix "everything is a file" ("except when it
|>|isn't"; except on Plan9, where it is) is broken by that for no
|>|reason. I did not read the files it is too late, but i would have
|>|stood up and have done it like that.
|> ...
|>||[1]: https://lwn.net/ml/linux-kernel/yjqvemckzcu1p...@mit.edu/
|>||[2]: https://lwn.net/ml/linux-kernel/yjqbcqbyhcopg...@zx2c4.com/
|>
|> Now i have read them and [2] indeed brings SYS_ADMIN into play but
|> disregards it with
|>
|> The problem with that approach, though, is that various
|> userspaces might already write garbage into /dev/urandom, not
|> expecting them to be credited -- for example, some userspace
|> hardware configuration component that writes some serial number
|> there. So I'm sort of hesitant to _change_ the behavior that
|> we've had for so long.
...
|> One could have made a survey? Before breaking all non-systemd
|> distributions i mean.
|> Change creditable root writes to /dev/random instead?
|
|I consider this interface one of the brokest seen in the UNIX domain
I want to add that [2] first said
Since we now always use a cryptographic hash function, we can
haphazardly mix whatever any user wants, without too much
concern. The issue is whether we _credit_ those bits. Were we to
credit those bits, a malicious unpriv'd user could
as well as that you removed the quote
Before i had to install haveged because python2 via mailman
consumes so much random you get grazy.
and remark that haveged creates many (many!!) bits of entropy in
the fraction of a second.
Luckily i could remove haveged after the Linux random rewrite.
The problem is (thus) solely, in an asynchronous startup, when the
CRNG reports itself seeded (and security related software that
hang waiting on it for this get served).
...
|IMHO, I vote for /sys rather than /proc/debug. The capability to
There is /proc/sys/kernel/random.
|directly handle the system entropy pool should be an Admin privilege
|even before being a debug option. As well as disable that handling
|with an
|
|echo 1 > /sys/driver/random/entropy_handle_disable
|
|In such a way that even Admin cannot do something like
|
|echo 1 > /sys/driver/random/entropy_pool_reset
|
|or
|
|dd if=/dev/zero count=128 of=/sys/driver/random/entropy_pool
...
Access to the pool will never happen again i bet.
Personally i think it is best effort and that this was achieved.
I was convinced of the rewrite of Donenfeld, including the forward
secrecy that Bernstein was talking about -- i cannot reference,
anything under cr.yp.to/crypto.html seem to have disappeared.
I found this large pool mixing that was used before and is still
used by OpenBSD a bit suspect. On the other hand a larger pool
can store more input. Hm.
(I find it grazy how expensively this is handled even. It happens
all the time while i write this, .. WOW! ...)
One thing is plain, i would not know how to make it any better.
Except for the topic of this thread.
And i have to concur that with the seedrng approach super fine-
grained and secure locking in a super-parallel super-fast async
boot process is possible.
But i do not get his arguments, who writes during the boot, and so
much, and if, just change to /dev/random for credit (i mean, isn't
/dev/urandom the not-so-good thing anyhow).
But if it is not that (random), then a trigger like yours as in
/proc/sys/kernel/random/pool_is_seeded is not a bad idea. Then
a boot sequence could decide when to unlock daemons that yet
accumulated.
This is of course all hypothetical. And in the meantime most
boot sequences adopted a dedicated program to be shipped.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Thu, 21 Sept 2023 at 02:35, Steffen Nurpmeso wrote:
>
> Steffen Nurpmeso wrote in
> <20230919222910.pn44y%stef...@sdaoden.eu>:
> |Laurent Bercot wrote in
> | :
> ||>|IIRC writing to /dev/urandom doesn't do what you want it to do.
> ||>|You have to use an ioctl() to actually set entropy.
> ||>
> ||>And that is the sad point about it.
> ||>Kernel hackers should stand up to allow it again!
> ||
> || As Ted Ts'o said[1], and Donenfeld agreed[2], the problem is that
> ||any user can write to /dev/urandom, including malicious users, so
> ||you cannot credit what they write.
> |
> |Well i was talking on unlocking the thing upon boot. Only root
> |there is.
> |
> |And then you could easily look who is the writer, or require
> |CAP_SYS_ADMIN or whatever when deciding whether "entropy" is to be
> |counted or not. That is just a check that is done a thousand
> |times in the kernel, i would think.
> |
> |Actually, if i recall correctly, it is only for unlocking after
> |the rewrite anyway since entropy is not counted no more? At least
> |the IOCTL which gives the number does not change no more.
> |
> |The thing is plain. Unix "everything is a file" ("except when it
> |isn't"; except on Plan9, where it is) is broken by that for no
> |reason. I did not read the files it is too late, but i would have
> |stood up and have done it like that.
> ...
> ||[1]: https://lwn.net/ml/linux-kernel/yjqvemckzcu1p...@mit.edu/
> ||[2]: https://lwn.net/ml/linux-kernel/yjqbcqbyhcopg...@zx2c4.com/
>
> Now i have read them and [2] indeed brings SYS_ADMIN into play but
> disregards it with
>
> The problem with that approach, though, is that various
> userspaces might already write garbage into /dev/urandom, not
> expecting them to be credited -- for example, some userspace
> hardware configuration component that writes some serial number
> there. So I'm sort of hesitant to _change_ the behavior that
> we've had for so long.
>
> and
>
> Another variation on that would be to do what this current patch
> does, but only crng_pre_init_inject() on CAP_SYS_ADMIN. But this
> has the same pitfall of only working as intended at cnrg_init=0
> but not crng_init=1.
>
> One could have made a survey? Before breaking all non-systemd
> distributions i mean.
> Change creditable root writes to /dev/random instead?
I consider this interface one of the brokest seen in the UNIX domain
ever. Despite this, I am reluctant in wishing that it would be
completely changed because breaking the back-compatibility with
30years old production systems is something that goes beyond a survey.
Everything can be improved and enhanced, obviously and I hope this
will happen for this interface as well, in a smooth and
back-compatible way.
Fortunately, in the years the /sys became a standard and this allows a
creation of a specific device u/random that has a property like
entropy. That property presented like a file, despite the ownership
and rwx bitmask assigned, it will check for CAP_SYSADMIN and allow
root to inject directly in the entropy pool whatever data s/he wants
and/or resetting the pool.
This enanchement is good not only to re-established the principle for
which everything in UNIX is a file and root is the Admin but also
allows people to experiment with application running on a system for
which the entropy pool has been arbitrarily changed including reseted
to a large bounce of zeros. A condition that should never happen in
production but could be very interesting to see in a lab experiment.
After all, it would not be the first time that we design a way to
inject errors into a system to debug / verify the error handlers or
watch how the system and application behave in extreme situations or
corner-cases. Nothing new under the sun. Right?
IMHO, I vote for /sys rather than /proc/debug. The capability to
directly handle the system entropy pool should be an Admin privilege
even before being a debug option. As well as disable that handling
with an
echo 1 > /sys/driver/random/entropy_handle_disable
In such a way that even Admin cannot do something like
echo 1 > /sys/driver/random/entropy_pool_reset
or
dd if=/dev/zero count=128 of=/sys/driver/random/entropy_pool
without enabling the pool handler interface which would come disabled
by default. I also suggest that > (write) will reset it, while >>
(append) will just append. The /sys interface for u/random is
completely new and we can accept to establish a completely new
standard about it. Following the common rules of dealing with files,
it seems to me the most reasonable way to do it.
I hope this helps, R-
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Steffen Nurpmeso wrote in
<20230919222910.pn44y%stef...@sdaoden.eu>:
|Laurent Bercot wrote in
| :
||>|IIRC writing to /dev/urandom doesn't do what you want it to do.
||>|You have to use an ioctl() to actually set entropy.
||>
||>And that is the sad point about it.
||>Kernel hackers should stand up to allow it again!
||
|| As Ted Ts'o said[1], and Donenfeld agreed[2], the problem is that
||any user can write to /dev/urandom, including malicious users, so
||you cannot credit what they write.
|
|Well i was talking on unlocking the thing upon boot. Only root
|there is.
|
|And then you could easily look who is the writer, or require
|CAP_SYS_ADMIN or whatever when deciding whether "entropy" is to be
|counted or not. That is just a check that is done a thousand
|times in the kernel, i would think.
|
|Actually, if i recall correctly, it is only for unlocking after
|the rewrite anyway since entropy is not counted no more? At least
|the IOCTL which gives the number does not change no more.
|
|The thing is plain. Unix "everything is a file" ("except when it
|isn't"; except on Plan9, where it is) is broken by that for no
|reason. I did not read the files it is too late, but i would have
|stood up and have done it like that.
...
||[1]: https://lwn.net/ml/linux-kernel/yjqvemckzcu1p...@mit.edu/
||[2]: https://lwn.net/ml/linux-kernel/yjqbcqbyhcopg...@zx2c4.com/
Now i have read them and [2] indeed brings SYS_ADMIN into play but
disregards it with
The problem with that approach, though, is that various
userspaces might already write garbage into /dev/urandom, not
expecting them to be credited -- for example, some userspace
hardware configuration component that writes some serial number
there. So I'm sort of hesitant to _change_ the behavior that
we've had for so long.
and
Another variation on that would be to do what this current patch
does, but only crng_pre_init_inject() on CAP_SYS_ADMIN. But this
has the same pitfall of only working as intended at cnrg_init=0
but not crng_init=1.
One could have made a survey? Before breaking all non-systemd
distributions i mean.
Change creditable root writes to /dev/random instead?
But regardless, they silently broke random in 2019 (due to
RANDOM_TRUST_CPU addition) so that my VM hang, so i wrote
entropy-saver.c to actually count the entropy i saved/restored
myself.
Before i had to install haveged because python2 via mailman
consumes so much random you get grazy.
At least that nonsense concept is now history.
Or, as Mr. Donenfeld wrote to me in private after the thread on
this list last year, "If you're trying to find some kind of
"cryptographically secure cut-off point" instead, what you want is
256 bits / 32 bytes" (instead of 512 bytes).
So to be out, i think that is terrible and i like how OpenBSD does
it, via shell, aka there the CRUX Linux /etc/rc(.shutdown)? way of
doing things would simply continue to work. I think the Blake2/xy
approach in the kernel and the VM fork detection and reseeding is
great. Getting rid of entropy count is great. (All of that seems
to have been massively touched again i think i read in LKML.)
But it is good that the world is getting more perfect.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Wed, 20 Sept 2023 at 18:52, Didier Kryn wrote: > > It is the rationale of everything /run to be lost on reboot. > Thanks Didier, for having highlighted this. Currently, everything is on RAM in the embedded system I am working on. ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Le 20/09/2023 à 07:51, Roberto A. Foglietta a écrit : The reason because it exists the ioctl() is well explained here https://stackoverflow.com/questions/17118705/using-rndaddentropy-to-add-entropy-to-dev-random which refers to the man page https://man7.org/linux/man-pages/man4/random.4.html Beware: I may have misunderstood something but the last reference must be outdated since it recommends to save the seed under /var/run ... In Debian/Devuan, /var/run points to /run which is mounted on tmpfs and therefore vanishes on shutdown. Better store the seed under /var/lib. It is the rationale of everything /run to be lost on reboot. /var/lib is not so well defined; it contains stuff which must be preserved (eg hardware clock frequency adjustment) and things which don't need (eg file locks). -- Didier ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Wed, 20 Sept 2023 at 07:06, Roberto A. Foglietta
wrote:
> Finally, repeat one more time all together: ioctls() are bad because
> they are a trick to workaround the limitation of "in UNIX everything
> is a file" principle and like every principle it establish some
> limitations for a good reason. Again, again...
The reason because it exists the ioctl() is well explained here
https://stackoverflow.com/questions/17118705/using-rndaddentropy-to-add-entropy-to-dev-random
which refers to the man page
https://man7.org/linux/man-pages/man4/random.4.html
Therefore we can assume that interface is stable and widely
accepted while the comments in stackoverflow add an hint: <> - This means that adding good white noise to /dev/urandom
would be sufficient to avoid it running without a source of entropy.
In my own embedded system /dev/random is moved to /dev/.random and
/dev/urandom takes its place. It is a highly-questionable choice, I
know. This prevents anything blocking the functioning in the hope that
/dev/urandom will be good enough to replace /dev/random. To sustain
this conjecture, every 15 minutes, I feed the /dev/urandom with data
that I consider good quality white noise.
If /dev/random exists, it is supposed to exist for a good reason and
replacing it is a questionable approach. I accept it. The way in which
/dev/random is used in practice is another story and in my embedded
system the SSH connection is a full-day working service while almost
all peripherals are inactive or disabled. Much more than a server with
disks and not just everything in RAM, it risks running low on entropy
very fast. However, SSH is configured to use /dev/urandom by default
therefore it is not a sink of entropy and probably there is no any
sink of entropy at all.
Back to the ioctl() which probably has been introduced before the /sys
exist, now the /sys exist. Time to think about planning to replace
that ioctl() with a file in /sys?
#define BUFSIZE 256
/* WARNING - this struct must match random.h's struct rand_pool_info */
typedef struct {
int bit_count; /* number of bits of entropy in data */
int byte_count; /* number of bytes of data in array */
unsigned char buf[BUFSIZ];
} entropy_t;
Giving a look to the structure does not seem straightforward to
replace it with a file-interface in particular about bit_count.
However, that value is ignored for /dev/urandom. Moreover, everything
in the buffer will be added after being passed through a hash
function. I hate this interface because the main idea behind this
interface is clearly: "you are stupid and therefore I prevent you to
do stupid things". This is the opposite of the UNIX paradigm. You
write a file? I assume that you know what you are doing otherwise
expect to meet your Karma soon.
Best regards, R-
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Tue, 19 Sept 2023 at 16:20, Laurent Bercot wrote:
>
>
> Oh boy. Is it that time of the year again already?
Vote for me, it will be Christmas all the days of the year! *ROTFL*
On Tue, 19 Sept 2023 at 23:58, Laurent Bercot wrote:
>
> >|IIRC writing to /dev/urandom doesn't do what you want it to do.
> >|You have to use an ioctl() to actually set entropy.
> >
> >And that is the sad point about it.
> >Kernel hackers should stand up to allow it again!
>
> As Ted Ts'o said[1], and Donenfeld agreed[2], the problem is that
> any user can write to /dev/urandom, including malicious users, so
> you cannot credit what they write.
>
Not every users, those have the permission to write on /dev/urandom
~/j/hr/ramdisk# ls -al /dev/urandom
crw-rw-rw- 1 root root 1, 9 Sep 17 23:46 /dev/urandom
Usually the allowed user is only root which can execute busybox
seedrng as well and it can successfully deliver an ioctl() because it
has the privileges to do that otherwise not. The sensitive difference
is that seedrng is supposed to deliver good enough random data to the
entropy pool and who escalate its running privilege should also tamper
the data. However, who can escalate the seeding privilege and be able
to tamper its data, can also more easily open a root shell and at that
point everything can happen.
Therefore the idea for which we need a ioctl() to credit data into the
entropy pool - AFAIK, it just prevents a mismatch or a poorly
/dev/urandom permission settings - IMHO, instead of a ioctl() I would
check the permissions and accept as creditable entropy only when the
device /dev/urandom is properly configured (or alternatively in a /sys
instance if the kernel driver cannot control the /dev/urandom
permission settings).
Another point in favour of a iotcl() is a write made for mistake by
root. S/he wants write test data on /tmp/urandom but s/he type
/dev/urandom. Let me say, good luck to all those who wish to ride on
this path because preventing root mistakes is an endless and
counterproductive way to waste our life-time especially at the kernel
level.
On Wed, 20 Sept 2023 at 00:29, Steffen Nurpmeso wrote:
>
> The thing is plain. Unix "everything is a file" ("except when it
> isn't"; except on Plan9, where it is) is broken by that for no
> reason. I did not read the files it is too late, but i would have
> stood up and have done it like that.
>
Instead, this is the best reason because ioctl()s are bad, in general
and in particular in this specific case.
===
However the main question was: why does the kernel not properly
initialize the /dev/urandom entropy pool at boot time considering that
99% of the system generates enough entropy at boot time?
The scripts were a way to show in practice (PoC) how we can generate
that entropy. By the way, seedrng generate a files which is supposed
to be written in a persistent filesystem and used by the kernel at the
time of the next boot. I wish you notice two facts about this way of
doing things: 1. that credit data can be tampered by an attacker that
can escalate the priviledges to write on /dev/urandom and 2. the idea
that the kernel read a file system file in a certain position is far
more complex than hash some chukns of the boot log containing a
fine-timing granularity timestamps into the pool of the /dev/urandom
entropy.
Finally, repeat one more time all together: ioctls() are bad because
they are a trick to workaround the limitation of "in UNIX everything
is a file" principle and like every principle it establish some
limitations for a good reason. Again, again...
Best regards, R-
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Laurent Bercot wrote in
:
|
|>|IIRC writing to /dev/urandom doesn't do what you want it to do.
|>|You have to use an ioctl() to actually set entropy.
|>
|>And that is the sad point about it.
|>Kernel hackers should stand up to allow it again!
|
| As Ted Ts'o said[1], and Donenfeld agreed[2], the problem is that
|any user can write to /dev/urandom, including malicious users, so
|you cannot credit what they write.
Well i was talking on unlocking the thing upon boot. Only root
there is.
And then you could easily look who is the writer, or require
CAP_SYS_ADMIN or whatever when deciding whether "entropy" is to be
counted or not. That is just a check that is done a thousand
times in the kernel, i would think.
Actually, if i recall correctly, it is only for unlocking after
the rewrite anyway since entropy is not counted no more? At least
the IOCTL which gives the number does not change no more.
The thing is plain. Unix "everything is a file" ("except when it
isn't"; except on Plan9, where it is) is broken by that for no
reason. I did not read the files it is too late, but i would have
stood up and have done it like that.
| I tend to trust people who do the work rather than those who stand on
|cardboard boxes.
| Still, since it's related to boot sequence things and I want to be a
|reliable source on boot sequences, I actually studied the thing when
|it came up, and understood the issue enough to come up with my own
|conclusion - and my own conclusion is still that the person who did the
|work, i.e. Jason, is right about this.
|
| I'm sorry. I like the idea of writing stuff to /dev/urandom and have
|it count, too. It's just not a good idea for security. That's just the
|way it is. And it would be nice if all the work and ink that already
|went into it, including mine, could actually be useful to all the people
|who don't care about any of this and just want their systems to work
|and be secure - so it would be nice if disinformation and bad ideas
|stopped being spread.
What a sheer nonsense. Sorry.
|[1]: https://lwn.net/ml/linux-kernel/yjqvemckzcu1p...@mit.edu/
|[2]: https://lwn.net/ml/linux-kernel/yjqbcqbyhcopg...@zx2c4.com/
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
|IIRC writing to /dev/urandom doesn't do what you want it to do. |You have to use an ioctl() to actually set entropy. And that is the sad point about it. Kernel hackers should stand up to allow it again! As Ted Ts'o said[1], and Donenfeld agreed[2], the problem is that any user can write to /dev/urandom, including malicious users, so you cannot credit what they write. I tend to trust people who do the work rather than those who stand on cardboard boxes. Still, since it's related to boot sequence things and I want to be a reliable source on boot sequences, I actually studied the thing when it came up, and understood the issue enough to come up with my own conclusion - and my own conclusion is still that the person who did the work, i.e. Jason, is right about this. I'm sorry. I like the idea of writing stuff to /dev/urandom and have it count, too. It's just not a good idea for security. That's just the way it is. And it would be nice if all the work and ink that already went into it, including mine, could actually be useful to all the people who don't care about any of this and just want their systems to work and be secure - so it would be nice if disinformation and bad ideas stopped being spread. [1]: https://lwn.net/ml/linux-kernel/yjqvemckzcu1p...@mit.edu/ [2]: https://lwn.net/ml/linux-kernel/yjqbcqbyhcopg...@zx2c4.com/ -- Laurent ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Laurent Bercot wrote in
:
...
| The answer's to Roberto's first question is: yes, initializing
|/dev/urandom is necessary, but writing stuff to /dev/urandom will not
|help, even if you consider that stuff random enough. You need a specific
|tool like seedrng.
No.
...
| For the second thing: most of the initialization of a system can happen
|while the seeding of the entropy pool is in progress. However, at some
|point, you need a good source of randomness, e.g. when starting an sshd
|server, and you should have a tool that makes sure the entropy pool is
|full *before* important services start using it to get their random
|data.
|
| seedrng, or rngseed, fill that role. Writing data to /dev/urandom does
|not. So the answer to Roberto's second question is: no, the provided
Why not? _Only_ by definition. The definition is not right.
|script excerpt is *not* suitable for seeding the entropy pool, no matter
|how much compression, or even how much hashing, you use.
That .. i agree with. (I have not really looked i must admit.
This is both truly hairy and totally "exaggerated", in my opinion,
sorry for the bad english. I have read OpenBSD's as well as
Donenfeld's first as well as Tso's random stuff in the past. If
anyone wants to know, in my opinion counting entropy was and is
a miracle to me. NetBSD's CVS HEAD now has a truly sophisticated
approach that is user tunable, in sofar: nice! I myself say: it
is best effort, treat it as "seeded" if
stat("/dev/random")->st_blksize (aka "stat -c %o /dev/random")
bytes have been written to "it", uh, that is a large value!, and
mix in "jitterentropy" and interrupts and what not for sources in
the kernel to be unique.)
That VM-fork stuff of Jason Donenfeld is a good thing!
...
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
David Laight wrote in <1628d46df2fb4580b474fd0ea43a5...@acums.aculab.com>: |> Fine, then use > (write) to reset the entropy and >> (append) to \ |> add entropy. | |IIRC writing to /dev/urandom doesn't do what you want it to do. |You have to use an ioctl() to actually set entropy. And that is the sad point about it. Kernel hackers should stand up to allow it again! It wins nothing to only allow this, sorry, grazy lock/ioctl dance unless you have a super-parallel boot sequence, but maybe even then not (as the device below cares, surely it does, not looking). On my (CRUX) distro with SysV style init everything is synchronous during boot/shutdown, and i fail to see the improvement. The kernel should simply agree to have entropy. (This rhymes!!) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Oh boy. Is it that time of the year again already? In case the /dev/urandom initialisation is a necessity (or a best practice), does it make sense to add it into busybox as an option or as an application? So, there are two different things to address here: 1. the way to add entropy to the kernel's entropy pool in order to make /dev/urandom sufficiently unpredictable; 2. the best way to do this as part of a boot process. For the first thing, the problem is that writing to /dev/urandom adds data to the seed, but does not update the entropy count, because the data you write is not considered creditable (i.e. does not come from a source that is considered random enough to warrant increasing the entropy count). That is the reason for the ioctl: to mark some random data as creditable. That is (a part of) why Jason, who revamped the entire random subsystem in the Linux kernel, wrote seedrng[1]. (Unfortunately, he did not approve the modifications to the version of seedrng that made it into busybox, so use bb seedrng at your own risk. I have an alternative implementation of the same concept, rngseed[2], that follows Jason's design more closely, and has additional options to give more flexibility for e.g. systems with read-only boot disks.) The answer's to Roberto's first question is: yes, initializing /dev/urandom is necessary, but writing stuff to /dev/urandom will not help, even if you consider that stuff random enough. You need a specific tool like seedrng. For the second thing: most of the initialization of a system can happen while the seeding of the entropy pool is in progress. However, at some point, you need a good source of randomness, e.g. when starting an sshd server, and you should have a tool that makes sure the entropy pool is full *before* important services start using it to get their random data. seedrng, or rngseed, fill that role. Writing data to /dev/urandom does not. So the answer to Roberto's second question is: no, the provided script excerpt is *not* suitable for seeding the entropy pool, no matter how much compression, or even how much hashing, you use. On that point, busybox cannot help here. busybox has an implementation of seedrng, but it does not provide a full boot sequence or policy whatsoever; it cannot tell *when* during boot it is appropriate to call seedrng. It is all up to system integrators to write boot sequences that seed the entropy pool correctly and start services securely. [1]: https://git.zx2c4.com/seedrng/about/ [2]: https://skarnet.org/software/s6-linux-utils/rngseed.html -- Laurent ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
RE: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
> Fine, then use > (write) to reset the entropy and >> (append) to add entropy. IIRC writing to /dev/urandom doesn't do what you want it to do. You have to use an ioctl() to actually set entropy. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales) ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Tue, 19 Sept 2023 at 14:06, Roberto A. Foglietta wrote: > > On Tue, 19 Sept 2023 at 13:30, Rob Landley wrote: > As the first init instruction or as the last kernel boot operation, is > THE general answer also when it is not the solution. Some systems need > a more dedicated effort but in no way THE answer puts them in a worse > condition than they were before. Confutations are welcome. :-) Yes, it hurts because on some specific systems the /dev/urandom is filled with a data stream which is constant for each boot or not good enough for seeding it in a secure way. This means that any later effort wil not have an immediate effect because of that initial seeding. Fine, then use > (write) to reset the entropy and >> (append) to add entropy. This will break the back-compatibility with the past and mess-up all the scripts or tools which are in the production nowadays. Fine, then use > (write) to add entropy as usual and >> (append) to reset entropy. Unfortunately this approach leaves behind some specific systems, when they will update the kernel. At that future time, they will choose to deselect the default option to seed the /dev/random OR use >> to overcome the issue. Hoping that those systems have a better way to seed the /dev/urandom engine which can not be the case at all because a low-frequency 1-single core NOMMU system, possibly with "none" as I/O and task scheduler can have a very small quantity of entropy and no means to amplify it. We may agree that those systems - because of their intrinsic limitations - are not sophisticated enough for being exposed in an untrustworthy environment like the Internet or they should integrate/connect a I2C/USB cheap thermal sensor that can provide true white noise, for example. With 1% of the effort, 99% of the work is done. The rest should be dealt with by experts which will select the more appropriate set of features and options case by case. Trying to deal with all the cases including that 1% lead to an impasse. This is wrong because a general solution should not necessarily be a mathematical truth but a the main way for almost all the cases which includes a reasonable exception handling that allows to deal properly also with the un-usual cases. This is a general solution. A mathematical truth is often useless or counterproductive in the real-world. Math says it is wrong in the most general case while practice says it works in almost all cases. Then add an exception handling and go with the practice. ;-) I hope this helps, R- ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Tue, 19 Sept 2023 at 14:06, Roberto A. Foglietta wrote: > > On Tue, 19 Sept 2023 at 13:30, Rob Landley wrote: > dmesg | sha512sum > /dev/urandom Oops, sorry because the sha512sum command-line prints a human readable 16 chars based string dmesg | pigz -4c | dd bs=64 skip=1 > /dev/urandom is a far better option. Obviously, in the kernel sha512sum() provides a binary stream not a text. The good habits to have a basic /dev populated with those few devices that always exist like /dev/urandom allows the seeding even before the mount procedure. Best regards, R- ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Tue, 19 Sept 2023 at 13:30, Rob Landley wrote: > > Modern "software" entropy collection is mostly timing jitter collected very > slowly as the system runs. The theory for embedded devices was if you have > some > writeable space in the device you save a few hundred bytes of /dev/urandom > output on shutdown and write it back on bootup so you're keeping enough > entropy > across reboots to seriously salt your attack surface. The perfect is the enemy > of the good: this just makes different boots behave differently. Even without a small writable space the problem could be addressed because the time jittering at boot kernel time before the system would be ready and expose any kind of service. A general solution that can be completely safe and efficient for all kinds of systems or hardware on which Linux kernel can run is very hard to find. For a subset of systems - fortunately the most used and common to be found on the market - the solution is to amplify the small amount of entropy generated by kernel boot timing jittering and use it to seed the /dev/urandom. Because the amplification process can always be based on mixing multi-task outputs in a single stream in a way that tiny⁻⁴ timings variations (jittering) makes a difference also in lines ordering - in an truly unpredictable way - then the dmesg output provided with a fine-grained time resolution timestamp on each line can be used as input for a hash function or a compression algorithm tailored for discarding de/compression metadata. With this data it is possible to seed the /dev/random. Even In a system that always start with UNIX EPOCH = 0 and is missing an RTC, in the /init execute this instruction dmesg | sha512sum as soon as possible. I bet that you will never find two occurrences that match as far as the dmesg is provided with a enough fine-grained time resolution. In those system for which the time resolution provided by dmseg is too large compared with the timings jittering at boot time, then the sha512sum will ALWAYS provide the same output unless multi-tasking boot mix-up some lines because hardware initialisation timings can varying enough (a mixed case, few or some hashes are possible but not a great variety of them). In all these cases, seeding the /dev/urandom with this procedure DOSN'T hurt. Is the seed constant? At least it is not the same for all the systems but characteristic of that specific hw/vt instance. Is the seed weak because it can spawn among few/some others, at least is not totally constant like providing nothing at all. The seed is good but the cat /proc/cmdline happens like in Android just because they need to do something about? Fine, we cannot prevent it. As you can see the problem is not the problem, the problem is the perception of the problem. dmesg | sha512sum > /dev/urandom As the first init instruction or as the last kernel boot operation, is THE general answer also when it is not the solution. Some systems need a more dedicated effort but in no way THE answer puts them in a worse condition than they were before. Confutations are welcome. :-) ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
This is also a problem with many virtual appliances. There is no easy answer to this kernel bootup entropy problem and it is lamentable that solutions are delegated to user space, and even to scripts, and to non-experts who will do a bad job of it but not know it. You may as well feed what random data you have into rngd which will supply it to the kernel. You may as well feed /dev/urandom back, as has been pointed out, it is well-hashed (though then stretched) original entropy. It's a very poor solution but if there was a good solution it would be built into the kernel. (I remember the days when random state was saved on shutdown to be re-used on bootup). If you have control over the hardware spec, get a good random device on-board, RPMB or a TPM or something that can produce real random data on demand and provides /dev/hwrng against which you can run rngd to keep entropy full. My particular advice is to not simply dump some limited amount of poor "entropy" into the pool, because in corner cases (especially if retry is needed) it may not always be enough for whatever early processes need it. Instead, use rngd to keep the pool full enough until you get past this stage and then kill rngd. Having poor entropy is bad enough, but stalling boot because you don't have enough poor entropy because you guessed wrong... Some processes unexpectedly consume entropy, e.g. openssl has been seen to consume entropy even when decrypting. If you decrypt enough things you can stall on entropy starvation. Sam On Tue, 19 Sept 2023 at 07:12, Michael Conrad wrote: > On 9/19/23 01:36, Roberto A. Foglietta wrote: > > On Tue, 19 Sept 2023 at 03:25, Michael Conrad > wrote: > >> On 9/18/23 06:14, Guillermo Rodriguez Garcia wrote: >> >> everything is compressed with gzip -7. This is the worst scenario. >>> However, even in the worst scenario due to gzip one single bit of >>> difference in the input generates a completely different compressed >>> output: >>> >> >> Compression (or any other deterministic manipulation of data) does not >> add any entropy (or "unpredictability") since the processing is 100% >> reproducible. >> In terms of entropy the output of the function is as good (or as bad) as >> the amount of entropy in the initial seed. >> >> Hi Michel, > >> Even aside from that, using gzip as some sort of hash function is not >> going to be anywhere near as good as using an actual hash function, like >> sha256, sha1 or even md5. >> > > PREMISE > > Hashing functions and compression algorithms are two completely different > kinds of mathematical tools. The most obvious difference is that > > That's a very large response so I'm not sure which part to quote, but I > think you're still missing the point. Assuming your initialization of > randomness is meant to stop people from guessing the RSA/SSL keys you > generate on a small low-power device during startup scripts, the attacker > will probably have a copy of your device and be able to replay any of the > steps you take during your startup script. It will not matter whether you > took a few random bits and a large static text and ran it through pigz > before feeding it to /dev/urandom, because the attacker will start with > guesses of the same few bits and also run them and that same static text > through pigz to potentially get the same output. If they mirror your > initialization of /dev/urandom, then they can try to guess your RSA keys. > The only defense is having more actual bits of entropy, or preserving them > from a previous boot in a manner that the attacker can't read. > > My comment about hashing functions was implying that they would be used > repeatedly, not that the entire input would be condensed into one single > hash. Yes there is a reduction of entropy if you feed a hash function more > than 20 bytes of data and only get 20 out of it. My point was that no > hashing or compression is needed, because the kernel RNG is itself a sort > of hash function and should handle that. Just feed it the raw input bytes. > > -Mike > ___ > busybox mailing list > busybox@busybox.net > http://lists.busybox.net/mailman/listinfo/busybox > ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
El martes, 19 de septiembre de 2023, Roberto A. Foglietta < roberto.foglie...@gmail.com> escribió: > > Guillermo confused the information with the entropy. > > I didn’t confuse anything. Please leave me out of your arguments, thank you. > Guillermo is getting mad and screaming > > :-? You definitely need help. Guillermo -- Guillermo Rodriguez Garcia guille.rodrig...@gmail.com ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On 9/19/23 01:36, Roberto A. Foglietta wrote: On Tue, 19 Sept 2023 at 03:25, Michael Conrad wrote: On 9/18/23 06:14, Guillermo Rodriguez Garcia wrote: everything is compressed with gzip -7. This is the worst scenario. However, even in the worst scenario due to gzip one single bit of difference in the input generates a completely different compressed output: Compression (or any other deterministic manipulation of data) does not add any entropy (or "unpredictability") since the processing is 100% reproducible. In terms of entropy the output of the function is as good (or as bad) as the amount of entropy in the initial seed. Hi Michel, Even aside from that, using gzip as some sort of hash function is not going to be anywhere near as good as using an actual hash function, like sha256, sha1 or even md5. PREMISE Hashing functions and compression algorithms are two completely different kinds of mathematical tools. The most obvious difference is that That's a very large response so I'm not sure which part to quote, but I think you're still missing the point. Assuming your initialization of randomness is meant to stop people from guessing the RSA/SSL keys you generate on a small low-power device during startup scripts, the attacker will probably have a copy of your device and be able to replay any of the steps you take during your startup script. It will not matter whether you took a few random bits and a large static text and ran it through pigz before feeding it to /dev/urandom, because the attacker will start with guesses of the same few bits and also run them and that same static text through pigz to potentially get the same output. If they mirror your initialization of /dev/urandom, then they can try to guess your RSA keys. The only defense is having more actual bits of entropy, or preserving them from a previous boot in a manner that the attacker can't read. My comment about hashing functions was implying that they would be used repeatedly, not that the entire input would be condensed into one single hash. Yes there is a reduction of entropy if you feed a hash function more than 20 bytes of data and only get 20 out of it. My point was that no hashing or compression is needed, because the kernel RNG is itself a sort of hash function and should handle that. Just feed it the raw input bytes. -Mike ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Tue, 19 Sept 2023 at 03:25, Michael Conrad wrote: > On 9/18/23 06:14, Guillermo Rodriguez Garcia wrote: > > everything is compressed with gzip -7. This is the worst scenario. >> However, even in the worst scenario due to gzip one single bit of >> difference in the input generates a completely different compressed >> output: >> > > Compression (or any other deterministic manipulation of data) does not add > any entropy (or "unpredictability") since the processing is 100% > reproducible. > In terms of entropy the output of the function is as good (or as bad) as > the amount of entropy in the initial seed. > > Hi Michel, > Even aside from that, using gzip as some sort of hash function is not > going to be anywhere near as good as using an actual hash function, like > sha256, sha1 or even md5. > PREMISE Hashing functions and compression algorithms are two completely different kinds of mathematical tools. The most obvious difference is that 1. hash produces an output whose size is fixed whatever is the size of the input while compression output size might vary when input size changes 2. compression algorithms (f) have their counterpart (f⁻¹) that reverse the process while hash have not 3. because of point #2 the compression algorithm is bi-univocal functions the same input gave the same output and the same output brings back to the same input 4 We know that hashing functions are always injective functions: the same input gives the same output but the same output can have different input (collisions). Unless a hardware system is provided with a specific hardware component that produces constant entropy (white noise, preferably) the main problem is to create it from few reasonably good random inputs. As you can imagine, we can start a debate about the definition of "reasonably good random inputs" or "entropy". Or at the opposite, we can accept that those definitions are - restricted to our specific sector - simply meaning unpredictable data - unpredictable by an attacker or even better by the root admin of the system. Nanoseconds time granularity cannot be predicted by an attacker and also a system administrator could have a real hard time in doing that without sophisticated external hardware instruments. Unfortunately, not all systems are able to provide a nanosecond timing and the first main reason of this lack depend by the clock frequency: to have nanoseconds granularity (10⁻⁹) is necessary to have a GHz (10⁹) clock. MD5SUM, GZIP AND THE WHITE NOISE A relatively weak hash like MD5SUM is way better to create an unpredictable stream of data than any compression function. Ok, let see it: +:git-restore:recovery2:yamui> echo | md5sum 68b329da9893e34099c7d8ad5cb9c940 - As you can see I have 1/16 chance to guess the right next char in the md5sum output. At this point you notice that I am unfair because I used the output of the md5sum command-line (textual human-friendly representation) instead of the md5sum() binary output stream. Obviously, you are right. Hence, I make you notice that you did the same considering the gzip. You took the whole stream which also contains the information to decompress that stream of data. Modifying the function in a way that decompression information are not sent to the output, the output cannot be reversed anymore (f⁻¹) does not exist anymore. Because of this trick we can have a sort of length variable hashing function. It is a very bad hash function, because the fixed size of the output is a great feature. A great feature for the primary purpose for which hash functions are currently used, not so great for generating entropy. In fact, if we have 8 bit of entropy, the hash can provide us 512 bit of data stream - white noise - but the number of the 512 bit dataset that could provide us remains 256. In other words: O(8-bit-entropy) = 256 = O(sh512sum 8-bit-entropy). This is what Gulliermo wrote proposing as "entropy conservation principle". The entropy of a closed system never remains constant along the time but always increases and this is a currently accepted principle of physics. Guillermo confused the information with the entropy. However, the principle for which information is an immutable constant is not yet established in physics. I suggest abandoning this kind of consideration and remain confined in our specific sector. In our specific sector, the spectral analysis cannot confute this claim: removed all the data which are specifically tailored for decompressing, the compressed data stream can be statistically separated by white noise in a sensitive way. Well, it is not 100% true. The gzip -1 output can be discriminated by this kind of analysis from gzip -9 output with a certain degree of confidence. However, if we do the same with a real-world entropy generator - say based on thermal effects - we notice a slight rose-noise effect, a tiny-tiny-tiny-tiny predominance in low-frequencies. I used the word tiny 4 times because the black-body law has a 4 as an
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On 9/18/23 06:14, Guillermo Rodriguez Garcia wrote: everything is compressed with gzip -7. This is the worst scenario. However, even in the worst scenario due to gzip one single bit of difference in the input generates a completely different compressed output: Compression (or any other deterministic manipulation of data) does not add any entropy (or "unpredictability") since the processing is 100% reproducible. In terms of entropy the output of the function is as good (or as bad) as the amount of entropy in the initial seed. Even aside from that, using gzip as some sort of hash function is not going to be anywhere near as good as using an actual hash function, like sha256, sha1 or even md5. I would expect this all goes into the kernel's own hashing inside the RNG and so gzip or any other hash function before delivering it to the kernel is probably irrelevant. The name of the game is to find actually random bits, which you either need to save from the previous boot, or obtain from hardware somehow. The low bits of thermal sensors and multithreading scheduler timing nanoseconds are probably your best bet if you can't rely on having a hardware entropy generator. -Mike C ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Hi Roberto,
El lun, 18 sept 2023 a las 11:54, Roberto A. Foglietta (<
roberto.foglie...@gmail.com>) escribió:
> On Mon, 18 Sept 2023 at 11:20, Guillermo Rodriguez Garcia
> wrote:
> >
> >> # RAF: seeding the urandom device with some data and a few bits of
> randomness.
> >> # The randomness is put at the beginning of some text data, which
> is going
> >> # to be compressed. It is expected that the whole compressed data
> will be
> >> # way different each time, even if a great part of the input is
> constant.
> >> # Moreover, the size of the randomness changes each time into a
> range of
> >> # [32, 64] bytes, and this adds more unpredictability. Like a
> hash, the
> >> # compression algorithm will produce a way different binary output
> by just
> >> # changing a few bytes and initial conditions.
> >> {
> >> n=$((33 + ${RANDOM:-15}%32))
> >> dd if=/dev/random bs=$n count=1 2>&1
> >> cat /proc/cmdline /proc/*stat /init*
> >> } | pigz -$((1 + n%9))c > /dev/urandom &
> >
>
> Hi Gulliermo,
>
> first of all, thank for the feedback.
>
> > Not sure whether seeding dev/urandom with output from dev/random makes
> much sense, since both use the same source of entropy.
>
> AFAIK, the /dev/random uses a source of entropy related to hardware
> events while /dev/urandom is a pseudo-random generator.
No, this is a common myth but it is not correct. Both random and urandom
use the same PRNG. See: https://www.2uo.de/myths-about-urandom/
> This should
> grant us that there is a difference between the two. immediately after
> a boot, it is supposed that many hardware events took place
Or not. This could be, for example, a cloned VM running on the cloud (which
is a typical use case for busybox).
> and
> therefore reading some bytes from /dev/random would not be such a big
> issue., IMHO.
>
> > Also note that dev/random will block if there is not enough entropy
> left, so doing this in an init script might not be a very good idea --
> specially on systems that don't have a good source of entropy available.
>
> As as you might noticed in the function that I sent later answering
> to Jeff, I do not use anymore the /dev/random but /dev/urandom
>
> udvseed(){ local n=$((33+${RANDOM:-15}%32)) u=/dev/urandom;f(){ dd
> if=$u bs=$n count=1; };(cd /proc;f;cat cmdline *stat;f;) 2>&1|pigz
> -$((1+n%9))c >$u; }
>
> This makes your first statement something to consider. Also $RANDOM if
> available should be considered generated by the /dev/urandom and
> therefore belonging to its entropy pool. If $RANDOM is not available
> then my function is quite weak in term of unpredictability because
> read 48 bytes from /dev/urandom which is not seeded yet and use it
> with some proc data that might change but can be guessed and then
> everything is compressed with gzip -7. This is the worst scenario.
> However, even in the worst scenario due to gzip one single bit of
> difference in the input generates a completely different compressed
> output:
>
Compression (or any other deterministic manipulation of data) does not add
any entropy (or "unpredictability") since the processing is 100%
reproducible.
In terms of entropy the output of the function is as good (or as bad) as
the amount of entropy in the initial seed.
Best regards,
Guillermo Rodriguez Garcia
guille.rodrig...@gmail.com
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Mon, 18 Sept 2023 at 11:20, Guillermo Rodriguez Garcia
wrote:
>
>> # RAF: seeding the urandom device with some data and a few bits of
>> randomness.
>> # The randomness is put at the beginning of some text data, which is
>> going
>> # to be compressed. It is expected that the whole compressed data will
>> be
>> # way different each time, even if a great part of the input is
>> constant.
>> # Moreover, the size of the randomness changes each time into a range of
>> # [32, 64] bytes, and this adds more unpredictability. Like a hash, the
>> # compression algorithm will produce a way different binary output by
>> just
>> # changing a few bytes and initial conditions.
>> {
>> n=$((33 + ${RANDOM:-15}%32))
>> dd if=/dev/random bs=$n count=1 2>&1
>> cat /proc/cmdline /proc/*stat /init*
>> } | pigz -$((1 + n%9))c > /dev/urandom &
>
Hi Gulliermo,
first of all, thank for the feedback.
> Not sure whether seeding dev/urandom with output from dev/random makes much
> sense, since both use the same source of entropy.
AFAIK, the /dev/random uses a source of entropy related to hardware
events while /dev/urandom is a pseudo-random generator. This should
grant us that there is a difference between the two. immediately after
a boot, it is supposed that many hardware events took place and
therefore reading some bytes from /dev/random would not be such a big
issue., IMHO.
> Also note that dev/random will block if there is not enough entropy left, so
> doing this in an init script might not be a very good idea -- specially on
> systems that don't have a good source of entropy available.
As as you might noticed in the function that I sent later answering
to Jeff, I do not use anymore the /dev/random but /dev/urandom
udvseed(){ local n=$((33+${RANDOM:-15}%32)) u=/dev/urandom;f(){ dd
if=$u bs=$n count=1; };(cd /proc;f;cat cmdline *stat;f;) 2>&1|pigz
-$((1+n%9))c >$u; }
This makes your first statement something to consider. Also $RANDOM if
available should be considered generated by the /dev/urandom and
therefore belonging to its entropy pool. If $RANDOM is not available
then my function is quite weak in term of unpredictability because
read 48 bytes from /dev/urandom which is not seeded yet and use it
with some proc data that might change but can be guessed and then
everything is compressed with gzip -7. This is the worst scenario.
However, even in the worst scenario due to gzip one single bit of
difference in the input generates a completely different compressed
output:
redfishos:~ # cat /etc/firmware/touch_module_id_0x82.img | pigz -7c | sha1sum
1f0e7e00a47159708a5877b052d2e2c6e3489788 -
redfishos:~ # { cat /etc/firmware/touch_module_id_0x82.img; echo; } |
pigz -7c | sha1sum
d8db1ae97fc5ac8fa441db5c146d95fc43ac6d2e
In the best scenario $RANDOM provides a number that makes the
procedure change boot. The data read from /dev/urandom can be 32 bytes
or 64 bytes and the compression level can vary between 1 and 9.
Therefore every single bit of the input should be correctly guessed
otherwise the output will be completely different.
Instead of using $RANDOM, it is possible to use a value generated from
/dev/random by a single byte read:
randval=$(dd if=/dev/random bs=1 count=1 status=none | hexdump -ve '1/1 "%d\n"')
Or keep the first variant in which the initial data is read from
/dev/random in a range between 32 and 64 bytes, adding this trick just
in case $RANDOM is not defined.
Best regards, R-
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
El lun, 18 sept 2023 a las 9:42, Roberto A. Foglietta (<
roberto.foglie...@gmail.com>) escribió:
> Hi all,
>
> I am investigating the Android init procedure (one version, one
> device, not in general) and I found an interesting line about the
> initialization of the /dev/urandom (seeding, I suppose).
>
> cat /proc/cmdline > /dev/urandom
>
> Therefore, I developed a more sophisticated way to do that initialisation:
>
> # RAF: seeding the urandom device with some data and a few bits of
> randomness.
> # The randomness is put at the beginning of some text data, which is
> going
> # to be compressed. It is expected that the whole compressed data
> will be
> # way different each time, even if a great part of the input is
> constant.
> # Moreover, the size of the randomness changes each time into a range
> of
> # [32, 64] bytes, and this adds more unpredictability. Like a hash,
> the
> # compression algorithm will produce a way different binary output by
> just
> # changing a few bytes and initial conditions.
> {
> n=$((33 + ${RANDOM:-15}%32))
> dd if=/dev/random bs=$n count=1 2>&1
> cat /proc/cmdline /proc/*stat /init*
> } | pigz -$((1 + n%9))c > /dev/urandom &
>
Not sure whether seeding dev/urandom with output from dev/random makes much
sense, since both use the same source of entropy.
Also note that dev/random will block if there is not enough entropy left,
so doing this in an init script might not be a very good idea -- specially
on systems that don't have a good source of entropy available.
Guillermo
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Mon, 18 Sept 2023 at 10:11, Jeff Pohlmeyer wrote:
>
> On Mon, Sep 18, 2023 at 2:42 AM Roberto A. Foglietta
> wrote:
>
> > In case the /dev/urandom initialisation is a necessity (or a best
> > practice), does it make sense to add it into busybox as an option or
> > as an application?
>
> If you are able to update to a newer version of busybox, you might
> want to check out the recently added "seedrng" applet, which seems to
> be a well-considered means of addressing this issue.
Hi Jeff,
thanks for the anwer:
redfishos:~ # seedrng
seedrng: can't create directory '/var/lib/seedrng': No such file or directory
redfishos:~ # mkdir -p /var/lib/seedrng
redfishos:~ # seedrng
Saving 2048 bits of creditable seed for next boot
I think that the app could create the directory path if it does not
exist. Moreover an option to write on stdout would be nice to have.
> You can find a
> (rather lengthy) discussion here:
>
> http://lists.busybox.net/pipermail/busybox/2022-April/089545.html
About this discussion, I have noticed two main points
1. the RNG can't actually be seeded from a shell script, due to the
reliance on ioctls and the fact that entropy written into the
unprivileged /dev/urandom device is not immediately mixed in, making
subsequent seed reads dangerous.
2. I suppose that the kernel will load the generated file in the
standard folder at the next boot time without further changes but I am
not sure about that. For sure, it will not succeed in my case because
rootfs a volatile filesystem and adding a link to a permanent data
partition is not a general solution (for this system and at the
moment).
IMHO, the best I can do is to seed the /dev/urandom by injecting some
data and then retrieve some data from it. I have no clue how long the
data read from /dev/urandom to be granted that the entropy injected
into it will be mixed as expected.
I have created a function that generates more than 2048 bytes for
seeding the /dev/urandom and read 4Kb after hoping to trigger the mix
of the new entropy.
udvseed(){ local n=$((33+${RANDOM:-15}%32)) u=/dev/urandom;f(){ dd
if=$u bs=$n count=1; };(cd /proc;f;cat cmdline *stat;f;) 2>&1|pigz
-$((1+n%9))c >$u; }
I wrote it in a way to be short. In fact, it is 153 bytes while the
seedrng app in busybox is about 1650. The function and the app are
completely different and it is not a fair comparison. However, it is
not the first time that I noticed that a busybox app can be easily
replaced with a shell script function and this reduces N times the
footprint.
Best regards, R-
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
Re: RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
On Mon, Sep 18, 2023 at 2:42 AM Roberto A. Foglietta wrote: > In case the /dev/urandom initialisation is a necessity (or a best > practice), does it make sense to add it into busybox as an option or > as an application? If you are able to update to a newer version of busybox, you might want to check out the recently added "seedrng" applet, which seems to be a well-considered means of addressing this issue. You can find a (rather lengthy) discussion here: http://lists.busybox.net/pipermail/busybox/2022-April/089545.html - Jeff ___ busybox mailing list busybox@busybox.net http://lists.busybox.net/mailman/listinfo/busybox
RFC: initialize /dev/urandom, is it necessary? Can we do it in a better way?
Hi all,
I am investigating the Android init procedure (one version, one
device, not in general) and I found an interesting line about the
initialization of the /dev/urandom (seeding, I suppose).
cat /proc/cmdline > /dev/urandom
Therefore, I developed a more sophisticated way to do that initialisation:
# RAF: seeding the urandom device with some data and a few bits of randomness.
# The randomness is put at the beginning of some text data, which is going
# to be compressed. It is expected that the whole compressed data will be
# way different each time, even if a great part of the input is constant.
# Moreover, the size of the randomness changes each time into a range of
# [32, 64] bytes, and this adds more unpredictability. Like a hash, the
# compression algorithm will produce a way different binary output by just
# changing a few bytes and initial conditions.
{
n=$((33 + ${RANDOM:-15}%32))
dd if=/dev/random bs=$n count=1 2>&1
cat /proc/cmdline /proc/*stat /init*
} | pigz -$((1 + n%9))c > /dev/urandom &
I wish to ask people here in this m-list, because I know that there
are Linux experts here, two questions:
1. initialise the /dev/urandom is necessary? Or the kernel provided
itself but is it better?
2. In your opinion, the script above can provide a reasonable
unpredictable initialisation?
In case the /dev/urandom initialisation is a necessity (or a best
practice), does it make sense to add it into busybox as an option or
as an application?
Best regards, R-
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox
