Re: why does RAND_add() take "randomness" as a "double"?
On 22/05/2019 19:32, Dennis Clarke wrote: Good options inspired by other cryptographic libraries include: - Number of bits of entropy passed in call (For example, a perfectly balanced coin flipper could provide the 4 byte values "head" or "tail" with an entropy of 1 bit). Let's drop the coin flipper. It was an off hand remark and by now we all know there ain't no such thing as a good coin flip for rng. See Professor Persi Diaconis at Stanford for that : https://www.youtube.com/watch?v=AYnJv68T3MM Bell's theorem and kolmogorov aside get a radiation decay source as that is really the *only* real rng that we know of. Or that I know of. http://www.fourmilab.ch/hotbits/hardware.html The coin flipper, even if theoretically problematic, is the standard statistical example used to describe a 1-bit-at-a-time hardware RNG. It includes a nice conceptual model to discuss hardware bias (using Shannon's entropy formula etc.). Actual 1-bit sources include the classic semiconductor shot noise fed to a comparator and some primitive implementations of radioactive RNGs. Also, radioactive sources are an unacceptable danger in many of the embedded and portable applications most likely to lack floating point support. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: why does RAND_add() take "randomness" as a "double"?
Good options inspired by other cryptographic libraries include: - Number of bits of entropy passed in call (For example, a perfectly balanced coin flipper could provide the 4 byte values "head" or "tail" with an entropy of 1 bit). Let's drop the coin flipper. It was an off hand remark and by now we all know there ain't no such thing as a good coin flip for rng. See Professor Persi Diaconis at Stanford for that : https://www.youtube.com/watch?v=AYnJv68T3MM Bell's theorem and kolmogorov aside get a radiation decay source as that is really the *only* real rng that we know of. Or that I know of. http://www.fourmilab.ch/hotbits/hardware.html -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional ps: see "futility of foresight"
Re: why does RAND_add() take "randomness" as a "double"?
I can understand why you may not want to break existing API. Why not add another similar interface done the right way? On 5/22/19, 1:11 PM, "openssl-users on behalf of Dr. Matthias St. Pierre" wrote: I think nobody of us needs to be convinced anymore that making it a 'double' was a bad idea. But the RAND api is very ancient and changing the argument type would be a breaking change. That's why we didn't dare to touch it when we overhauled the RNG implementation for 1.1.1, because we tried very hard not to add unnecessary breaking changes to the ones made in 1.1.0. Matthias smime.p7s Description: S/MIME cryptographic signature
Re: why does RAND_add() take "randomness" as a "double"?
On 21/05/2019 16:44, Salz, Rich via openssl-users wrote: When I overhauled the RAND mechanism, I tried to deprecate this use of floating point, in favor of just a number from 0 to 100 but was voted down. It *is* stupid. Luckily, on a modern system with system-provided randomness to seed the RNG, you never need this call. Perhaps it would have been more acceptable to use a binary base, instead of a decimal percentage, as there is nothing inherently decimal about this value. Good options inspired by other cryptographic libraries include: - Number of bits of entropy passed in call (For example, a perfectly balanced coin flipper could provide the 4 byte values "head" or "tail" with an entropy of 1 bit). - 256th of bits ditto (for example a coin flipper with a known slight imbalance could report 252/256th of a bit for each flip included in the buffer). - 32th of bits ditto (makes the "100%" case equal to (bytecount << 8)). In each of those 3 cases, the internal measure of "entropy left" would be in that same unit, and a compatibility mapping for the old API would do the conversion of the double as a pure inline macro that doesn't trigger "float used" compiler magics in programs that don't use it. Clarifying notes: - The current limit of considering only 32 bytes of entropy is an artifact of the current set of RNG algorithms, and should not be exposed in the API design. For example future addition of post-quantum algorithms may necessitate having an RNG with an internal state entropy larger than 256 bits. - Future RNG implementations may include logic to safely accumulate obtained entropy into "batches" before updating the RNG state, as this may have cryptographic benefits. - The use of a dummy double to force the alignment of structures and unions to the "highest known" value can be trivially replaced by another type where it is not already treated as "not actually floating point operations" by the compiler. For example by passing "-Ddouble=long long" as a compiler option. - The use of floating point registers in CPU-specific vector unit optimizations can be readily avoided by a no-asm compile. - Floating point calculations in test programs such as "openssl speed" is not relevant to actual library use. - On Linux x86, test programs that avoid all floating point can be checked via the PF_USED_MATH flag or its upcoming Linux 5.x replacement. This may be useful in the test suite. Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded
Re: why does RAND_add() take "randomness" as a "double"?
On 5/21/2019 9:48 PM, Paul Dale wrote: Double makes sense. Entropy is often estimated as a real value. Having a human readable calculation using floating point doesn't (to me) mean that an API argument has to be a double. From what I see in the code, the parameter 'double entropy' is used to increment a value that eventually reaches # define ENTROPY_NEEDED 32. Couldn't the number have been an unsigned long? If more precision was needed, make the units 1/64k and make ENTROPY_NEEDED 32 * 64k. It's a bit more work for the caller, but removes the (perhaps only) place floating point is needed.
Re: why does RAND_add() take "randomness" as a "double"?
On 5/21/2019 10:07 PM, Salz, Rich via openssl-users wrote: >Then just set it to 1.0 and be done with it. That hardly helps on systems that don't have floating point at all. No it doesn't. Such systems aren't supported by OpenSSL. There are many places were floating point is used/supported. Removing the second arg to RAND_add is the least of the problems (look at various asm files) The assembler code can be bypassed on those systems. I see a few places where it's used to force an alignment, but perhaps there's another way. It's used in test programs to report performance. For us, the random number generator was the problem.
Re: why does RAND_add() take "randomness" as a "double"?
>Then just set it to 1.0 and be done with it. >That hardly helps on systems that don't have floating point at all. No it doesn't. Such systems aren't supported by OpenSSL. There are many places were floating point is used/supported. Removing the second arg to RAND_add is the least of the problems (look at various asm files)
Re: why does RAND_add() take "randomness" as a "double"?
"Salz, Rich via openssl-users" skrev: (21 maj 2019 17:27:44 CEST) >>If it's a sarcasm, I'm missing the point. > >I was't being sarcastic, I was trying to show that the team, recently, >still liked the use of floating point. > >>There are use cases when one wants to mix/add extra randomness >from, e.g., an external source (that, for whatever reasons, is trusted >more than what's provided by the system). > >Then just set it to 1.0 and be done with it. That hardly helps on systems that don't have floating point at all. Cheers Richard > > > -- Skickat från min Android-enhet med K-9 Mail. Ursäkta min fåordighet.
RE: why does RAND_add() take "randomness" as a "double"?
Double makes sense. Entropy is often estimated as a real value.
E.g. we have the aforementioned coin flipper feeding data serially.
Adding each bit sequentially means 0.125 bytes of entropy per call.
Not the best example
Pauli
--
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia
-Original Message-
From: Laszlo Ersek [mailto:ler...@redhat.com]
Sent: Wednesday, 22 May 2019 12:15 AM
To: openssl-users@openssl.org
Cc: Jian J Wang ; edk2-devel-groups-io
; Lu, XiaoyuX ; Ard Biesheuvel
Subject: why does RAND_add() take "randomness" as a "double"?
(resending, with my subscription to completed)
Hi OpenSSL Developers,
(cross-posting and ,)
OpenSSL commit [1] changed the representation of the "entropy amount" -- later
renamed to "randomess" in [2] -- from "int" to "double". I've read the commit
message:
commit 853f757ecea74a271a7c5cdee3f3b5fe0d3ae863
Author: Bodo Möller
Date: Sat Feb 19 15:22:53 2000 +
Allow for higher granularity of entropy estimates by using 'double'
instead of 'unsigned' counters.
Seed PRNG in MacOS/GetHTTPS.src/GetHTTPS.cpp.
Partially submitted by Yoram Meroz .
and also checked "MacOS/GetHTTPS.src/GetHTTPS.cpp" at the same commit.
But, I'm none the wiser.
Can someone please explain what is gained by using a floating point type here?
Is it really a relevant use case that entropy is fed from an external source to
OpenSSL such that truncating the amount to a whole number of bits would cause
significant lossage? (Admittedly, it could be relevant if the individual
randomness bit counts were in the (0, 1) interval, both boundaries exclusive.)
Using floating point for randomness representation is a problem for
environments that prefer to avoid floating point altogether, such as
edk2 ("UEFI") firmware
Thanks,
Laszlo
[1] https://github.com/openssl/openssl/commit/853f757ecea7
[2] https://github.com/openssl/openssl/commit/f367ac2b2664
Re: why does RAND_add() take "randomness" as a "double"?
On 5/21/19 3:27 PM, Salz, Rich via openssl-users wrote: If it's a sarcasm, I'm missing the point. I was't being sarcastic, I was trying to show that the team, recently, still liked the use of floating point. There are use cases when one wants to mix/add extra randomness from, e.g., an external source (that, for whatever reasons, is trusted more than what's provided by the system). Then just set it to 1.0 and be done with it. External 300 baud serial attached coin flipper also works well. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional
Re: why does RAND_add() take "randomness" as a "double"?
On 5/21/19, 10:45 AM, "openssl-users on behalf of Salz, Rich via openssl-users" wrote: When I overhauled the RAND mechanism, I tried to deprecate this use of floating point, in favor of just a number from 0 to 100 but was voted down. If it's a sarcasm, I'm missing the point. It *is* stupid. In general, yes, it is. Luckily, on a modern system with system-provided randomness to seed the RNG, you never need this call. Respectfully disagree. There are use cases when one wants to mix/add extra randomness from, e.g., an external source (that, for whatever reasons, is trusted more than what's provided by the system). smime.p7s Description: S/MIME cryptographic signature
Re: why does RAND_add() take "randomness" as a "double"?
>If it's a sarcasm, I'm missing the point. I was't being sarcastic, I was trying to show that the team, recently, still liked the use of floating point. >There are use cases when one wants to mix/add extra randomness from, e.g., > an external source (that, for whatever reasons, is trusted more than what's > provided by the system). Then just set it to 1.0 and be done with it.
Re: why does RAND_add() take "randomness" as a "double"?
When I overhauled the RAND mechanism, I tried to deprecate this use of floating point, in favor of just a number from 0 to 100 but was voted down. It *is* stupid. Luckily, on a modern system with system-provided randomness to seed the RNG, you never need this call.
Re: why does RAND_add() take "randomness" as a "double"?
On 5/21/2019 10:15 AM, Laszlo Ersek wrote:
[snip]
Can someone please explain what is gained by using a floating point type
here?
Is it really a relevant use case that entropy is fed from an external
source to OpenSSL such that truncating the amount to a whole number of
bits would cause significant lossage? (Admittedly, it could be relevant
if the individual randomness bit counts were in the (0, 1) interval,
both boundaries exclusive.)
Using floating point for randomness representation is a problem for
environments that prefer to avoid floating point altogether, such as
edk2 ("UEFI") firmware
I agree, and I reported this back in 2016. We also have an environment
that does not have floating point.
