I composed in VI, and the tabs were a problem. I apologize; there is no good excuse not to have called the function before attaching it.
I had a logic problem. Accumulating the decimal conversion of hex needed to be modular divided to be within range, and I had the name of the hex string (pool) wrong. It runs as a program now. From quickly testing large ranges, it hovers near the mean, but on 10 and 60, it seemed to do ok. I am far less optimistic now. James On Tue, Nov 15, 2022 at 7:56 AM James Johnson <jj126...@gmail.com> wrote: > I am an amateur enthusiast. The random function I found in documentation > was in 2014 or so. I do not have the vocabulary to differentiate between > various prng's. I only knew of one, and I it returned some version of the > bell curve, with three favorite values, and one of THEM a pronounced > preference. > > The scholars here are referencing white papers and using hardware and > software interchangeably (?) I am not a randomization expert, and I do not > seek academic recognition. It's a novel hack is all, and I don't know how > to invoke the quality algorithms you reference. > > Chris A. asked me to "post" a module. I'm pretty sure he means on git-hub, > and I don't know how to use it. I attach a script with a randbelow() > function. > > The kernel of my idea is to use randomness from a hash value, so I > declared the hash globally, and called it in the function. I know this may > not be correct, but I don't know how to declare one momentarily and then > destruct it or release the ram the next minute, between random numbers. > > Normally, I would update the hexdigest outside the function, and pass it > to randbelow(range, hexdigest_string) as two arguments. Chris A wants it to > feed an analysis program that may not agree well with the "permanence" of a > hash between random numbers. > > I am willing to put in effort, but it's a steep learning curve for me. I > DO understand that my first script was a lot of data entry, data checking > and elementary reporting that was not germane. > > I use my fast computer off-line, and connect to the web with a Chromebook, > so online tools would mean changing my setup. I remain interested to help > and learn. > > I hope this is not too bare-bones. > > On Tue, Nov 15, 2022 at 7:17 AM Stephen J. Turnbull < > stephenjturnb...@gmail.com> wrote: > >> James Johnson writes: >> >> > I want to be good natured about it, not combative. I tried whatever >> random >> > function I found in documentation online, and used it. Without >> analyzing >> > the world, I was simply dissatisfied with the results. >> >> On the Python lists, you'll get much better discussion if you're >> specific about your "dissatisfaction". (If it's just the repetition >> problem, that's better handled by composing with a filter than by >> altering the PRNG itself, which should be as close to uniform as >> possible to make generating other distributions as simple as possible.) >> >> Your method *as described* is unclear. Don Knuth tried something >> similar (you have better tools available than he did, but the basic >> idea seems to be similarly "let's compose a few pseudo-randomizing >> transformations"), and got an embarrassing result. (Spoiler in >> footnote [1] if you don't feel like digging through Seminumerical >> Algorithms.) You may have had more method than Knuth did (cf Dave >> Mertz's statement that it "looks like Random123"), but again, it would >> really help communication if you described why you think your PRNG >> will do better than Python's current one. It's a heavily researched >> field for many decades now, and there are quite a few core devs with a >> lot of knowledge and experience in the field (Python apps provide >> quite of bit of the "attack surface" exposed on the Internet, it would >> be irresponsible if there weren't!) >> >> By the way, nobody here is going to laugh if it turns out that your >> thinking was naive. My point is to communicate more effectively, not >> that your idea is bad (despite having a copy of Seminumerical >> Algorithms, I'm not competent to evaluate your algorithm itself :-). >> How you feel about "getting schooled" is a personal thing, but my >> personal experience has been that there's an excellent education to be >> had by posting naive ideas to this list. :-) >> >> Sincere regards, >> >> N.B. Deliberately not trimming so you have to work to see the >> spoiler. :-) >> >> > >> > Here’s what I settled on. I actually “put my thumb on the scales,” to >> rule >> > out repetitions for seven questions, so randomness didn’t ultimately >> answer >> > my question. >> > >> > I’m not sure that the function I “got” was the Mersenne Twister. I do >> know >> > Mersenne was a truly great mathematician. >> > >> > I acknowledge that updating with time.asctime() EVERY time I update the >> > hash would be random; I’m not qualified to know if it would be “more” >> > random. After you ask about the bell curve v square distributions, I am >> > quite out of my depth. >> > >> > Here’s my “wizard” for anyone who wants to sell him cryptocurrency or >> > overcome his objections to the environmentalist agenda, or query him >> about >> > foreign policy. I hope you find it better than average. >> > >> > https://drive.google.com/file/d/1EqQsMfBHDrNpOBQrI7CJxrpbowvKPD2G/ >> > >> > Regards, >> > >> > James J >> > >> > On Tue, Nov 15, 2022 at 4:31 AM Wes Turner <wes.tur...@gmail.com> >> wrote: >> > >> > > While FWIU it is advisable to keep seeding an RNG after startup - >> that's >> > > what e.g. rngd does - the cryptography.io docs do advise to just use >> > > `os.urandom()` (which is not the same as random.SystemRandom()?). >> > > >> > > There probably should be better default random in CPython; though I'm >> > > personally not at all qualified to assess, TIL about NIST 800-22 and >> FIPS >> > > 140-2 for evaluating sources of entropy. >> > > >> > > Differential entropy > Differential entropies for various >> distributions >> > > https://en.wikipedia.org/wiki/Differential_entropy >> > > >> > > >> > > *** >> > > >> > > (TIL that quantum information is never destroyed; so which physical >> > > processes are actually nonreversible like hashing and RNG are >> supposed to >> > > be is up for consideration as classical information is a subset of >> quantum >> > > information. Do black holes shift gamma radiation may actually be >> relevant! >> > > Perhaps a permanent magnet under a (double-jointed?) bar and ball >> pendulum >> > > on a magnetic bearing would produce enough Brownian motion to get >> enough >> > > uniform random to call it sufficiently entropic for purposes of >> CSPRNG? >> > > Nondeterministic NDE fluid calculations diffract into many possible >> > > outcomes, but a brute force combinatorial search of all the possible >> return >> > > values is still deterministically ordered. And the quantum computing >> folks >> > > are working on increasing coherence / reducing error propagation; >> like ECC.) >> > > >> > > - [ ] DOC: The docs could advise regarding which decent enough open >> source >> > > hw RNG would be supported by os.urandom or random.SystemRandom if >> rngd is >> > > not configured to keep nondeterministically seeding with entropy that >> > > should presumably be from a Uniform random entropic process >> > > >> > > There should be better software random in python. >> > > >> > > >> > > On Tue, Nov 15, 2022, 1:19 AM James Johnson <jj126...@gmail.com> >> wrote: >> > > >> > >> Thank you for replying with such specific assistance. I am made >> acutely >> > >> aware that I am only a Python enthusiast, and not an academic. >> > >> >> > >> Hashes are deterministic, not random, but byte by byte, they can be >> very >> > >> random. Please accept the attached script as a "hack," that might >> be novel, >> > >> or a curiosity. My first script was defective several ways. In >> this, I seed >> > >> the hash by hashing the uu-8 ENCODED string value of the time - it >> is not >> > >> random, but it is rarely the same twice. I used 0-9 because it is >> decimal, >> > >> and 0-59, because hours and minutes come in 60's. >> > >> >> > >> I found that taking time.time_nc() gave me far more odds than evens. >> > >> >> > >> I do not know how the scheme I devised of adding hex digits together >> > >> would scale to larger ranges - it is done by adding hex digits that >> vary >> > >> from 0 - 15 repeatedly, but, since the number of hex digits varies >> with the >> > >> range, it could actually work for larger ranges. >> > >> >> > >> The random function in Python is not really adequate for a magic >> eight >> > >> ball program, so as a consumer I am dissatisfied without examining >> it at a >> > >> quantum level, for repeated reference thousands of times each. >> > >> >> > >> Thank you for your kind attention, and I hope the (much improved) >> hack >> > >> can be helpful to other enthusiasts. >> > >> >> > >> James J >> > >> (A.A. Faulkner University) >> > >> >> > >> On Mon, Nov 14, 2022 at 11:23 AM Wes Turner <wes.tur...@gmail.com> >> wrote: >> > >> >> > >>> QRNG "Quantum Random Number Generation" -> Hardware random number >> > >>> generator > Physical phenomena with random properties > Quantum >> random >> > >>> properties >> > >>> >> > >>> >> https://en.wikipedia.org/wiki/Hardware_random_number_generator#Quantum_random_properties >> > >>> >> > >>> >> > >>> FWIW, SciPy and SymPy have various non-CSPRNG random distributions: >> > >>> >> > >>> - https://docs.scipy.org/doc/scipy/reference/stats.html >> > >>> - >> > >>> >> https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.uniform.html#scipy.stats.uniform >> > >>> >> > >>> - https://docs.sympy.org/latest/modules/stats.html >> > >>> - >> https://docs.sympy.org/latest/modules/stats.html#sympy.stats.Uniform >> > >>> - >> > >>> >> https://docs.sympy.org/latest/modules/stats.html#sympy.stats.DiscreteUniform >> > >>> >> > >>> "RFC 8937: Randomness Improvements for Security Protocols" (2020) >> > >>> https://www.rfc-editor.org/rfc/rfc8937.html >> > >>> >> > >>> *** >> > >>> >> > >>> FWIW, >> > >>> UUIDs and W3C DIDs require entropy e.g. from a CSPRNG or better: >> > >>> https://www.w3.org/TR/did-core/#terminology : >> > >>> >> > >>> > Universally Unique Identifier (UUID) >> > >>> > A type of globally unique identifier defined by [RFC4122]. UUIDs >> are >> > >>> similar to DIDs in that they do not require a centralized >> registration >> > >>> authority. UUIDs differ from DIDs in that they are not resolvable >> or >> > >>> cryptographically-verifiable >> > >>> >> > >>> On Mon, Nov 14, 2022, 11:54 AM Wes Turner <wes.tur...@gmail.com> >> wrote: >> > >>> >> > >>>> https://docs.python.org/3/library/random.html : >> > >>>> >> > >>>> > Warning: The pseudo-random generators of this module should not >> be >> > >>>> used for security purposes. For security or cryptographic uses, >> see the >> > >>>> secrets module >> > >>>> >> > >>>> https://docs.python.org/3/library/secrets.html#module-secrets >> > >>>> >> > >>>> PEP 506 – Adding A Secrets Module To The Standard Library >> > >>>> https://peps.python.org/pep-0506/#alternatives >> > >>>> https://github.com/python/peps/blob/main/pep-0506.txt >> > >>>> >> > >>>> PEP 12: new PEP template: >> > >>>> https://github.com/python/peps/blob/main/pep-0012/pep-NNNN.rst >> > >>>> >> > >>>> Pseudorandom number generator > Cryptographic PRNGs >> > >>>> >> > >>>> >> https://en.wikipedia.org/wiki/Pseudorandom_number_generator#Cryptographic_PRNGs >> > >>>> >> > >>>> Random number generator attack > Defenses >> > >>>> >> https://en.wikipedia.org/wiki/Random_number_generator_attack#Defenses >> > >>>> >> > >>>> >> > >>>> /? CSPRNG >> > >>>> https://www.google.com/search?q=CSPRNG >> > >>>> >> > >>>> From "THE LINUX CSPRNG IS NOW GOOD!" >> > >>>> https://words.filippo.io/dispatches/linux-csprng/ : >> > >>>> >> > >>>> > [ get random() is from OpenBSD and LibreSSL ] >> > >>>> >> > >>>> > Performance and ChaCha20 >> > >>>> > Some people would say they needed a userspace CSPRNG for >> PERFORMANCE. >> > >>>> I never really believed most of them, but to be fair Linux was >> using a >> > >>>> kinda slow SHA-1 extractor back then. However, since Linux 4.8 >> (2016) the >> > >>>> default getrandom(2) source is a fast ChaCha20-based CSPRNG, with >> separate >> > >>>> pools per NUMA node to avoid contention. (Just ignore the rude >> comments in >> > >>>> the code about applications not running their own CSPRNG, this is >> still >> > >>>> Linux after all.) >> > >>>> > >> > >>>> > There's even a neat trick XOR'ing some of the CSPRGN output >> back into >> > >>>> the ChaCha20 state to prevent an attacker from recovering any >> past output >> > >>>> from before the time of compromise. >> > >>>> > >> > >>>> > Some of these improvements came along thanks to the Wireguard >> work by >> > >>>> Jason A. Donenfeld >> > >>>> >> > >>>> "Problems emerge for a unified /dev/*random" (2022) >> > >>>> https://lwn.net/Articles/889452/ >> > >>>> >> > >>>> From >> > >>>> >> https://www.redhat.com/en/blog/understanding-red-hat-enterprise-linux-random-number-generator-interface >> > >>>> : >> > >>>> >> > >>>> """ >> > >>>> How does the kernel initialize its CSPRNG? >> > >>>> The kernel has an “entropy pool,” a place where unpredictable >> input >> > >>>> observed by the kernel is mixed and stored. That pool serves as a >> seed to >> > >>>> the internal CSPRNG, and until some threshold of estimated >> entropy is >> > >>>> reached initially, it is considered uninitialized. >> > >>>> >> > >>>> Let’s now see how the kernel initializes its entropy pool. >> > >>>> >> > >>>> 1. After the kernel takes control on power-on, it starts filling >> its >> > >>>> entropy pool by mixing interrupt timing and other unpredictable >> input. >> > >>>> >> > >>>> 2. The kernel gives control to systemd. >> > >>>> >> > >>>> 3. Next, systemd starts and initializes itself. >> > >>>> >> > >>>> 4. Systemd, optionally, loads kernel modules which will improve >> the >> > >>>> kernel's entropy gathering process on a virtual machine (e.g., >> virtio-rng). >> > >>>> >> > >>>> 5. Systemd loads the rngd.service which will gather additional >> input >> > >>>> entropy obtained via a random generator exposed by hardware >> (e.g., the x86 >> > >>>> RDRAND instruction or similar) and jitter entropy1; this entropy >> is fed >> > >>>> back into the kernel to initialize its entropy pool, typically in >> a matter >> > >>>> of milliseconds. >> > >>>> >> > >>>> After the last step, the kernel has its entropy pool initialized, >> and >> > >>>> any systemd services started can take advantage of the kernel’s >> random >> > >>>> generator. >> > >>>> >> > >>>> Note that the virtio-rng kernel module loading in step (3), is an >> > >>>> optional step which improves entropy gathering in a virtual >> machine by >> > >>>> using the host's random generator to initialize the guest systems >> in KVM. >> > >>>> The rngd.service loading at the final step (4) is what ensures >> that the >> > >>>> kernel entropy pools are initialized on every scenario, and >> furthermore it >> > >>>> continues mixing additional data in the kernel pool during system >> runtime. >> > >>>> """ >> > >>>> >> > >>>> https://github.com/nhorman/rng-tools/blob/master/fips.c : >> > >>>> >> > >>>> ```c >> > >>>> /* fips.c -- Performs FIPS 140-1/140-2 RNG tests >> > >>>> ``` >> > >>>> >> > >>>> /? FIPS 140-1/140-2 RNG tests >> > >>>> https://www.google.com/search?q=FIPS+140-1%2F140-2+RNG+tests >> > >>>> >> > >>>> /? CMVP "cprng" >> > >>>> https://www.google.com/search?q=CMVP+%22cprng%22 >> > >>>> https://csrc.nist.gov/publications/detail/fips/140/3/final >> > >>>> >> > >>>> https://www.google.com/search?q=rng+tests >> > >>>> >> > >>>> - https://www.johndcook.com/blog/rng-testing/ : >> > >>>> >> > >>>> > We test RNGs using the standard test suites: PractRand, >> TestU01 >> > >>>> (BigCrush), DIEHARD(ER), NIST SP 800-22. >> > >>>> >> > >>>> Randomness tests: >> > >>>> >> > >>>> >> https://en.wikipedia.org/wiki/Randomness_test#Notable_software_implementations >> > >>>> : >> > >>>> - https://en.wikipedia.org/wiki/Diehard_tests >> > >>>> - https://en.wikipedia.org/wiki/TestU01 >> > >>>> - /? NIST 800-22 https://www.google.com/search?q=nist+800-22 >> > >>>> >> > >>>> /? nist 800-22 site:github.com >> > >>>> https://www.google.com/search?q=nist+800-22+site%3Agithub.com >> > >>>> >> > >>>> - >> > >>>> >> https://github.com/google/paranoid_crypto/blob/main/docs/randomness_tests.md >> > >>>> >> > >>>> >> > >>>> From https://cryptography.io/en/latest/random-numbers/ >> > >>>> >> https://github.com/pyca/cryptography/blob/main/docs/random-numbers.rst >> > >>>> : >> > >>>> >> > >>>> >> > >>>> ```rst >> > >>>> Random number generation >> > >>>> ======================== >> > >>>> >> > >>>> When generating random data for use in cryptographic operations, >> such >> > >>>> as an >> > >>>> initialization vector for encryption in >> > >>>> :class:`~cryptography.hazmat.primitives.ciphers.modes.CBC` mode, >> you do >> > >>>> not >> > >>>> want to use the standard :mod:`random` module APIs. This is >> because >> > >>>> they do not >> > >>>> provide a cryptographically secure random number generator, which >> can >> > >>>> result in >> > >>>> major security issues depending on the algorithms in use. >> > >>>> >> > >>>> Therefore, it is our recommendation to `always use your operating >> > >>>> system's >> > >>>> provided random number generator`_, which is available as >> > >>>> :func:`os.urandom`. >> > >>>> For example, if you need 16 bytes of random data for an >> initialization >> > >>>> vector, >> > >>>> you can obtain them with: >> > >>>> >> > >>>> .. doctest:: >> > >>>> >> > >>>> >>> import os >> > >>>> >>> iv = os.urandom(16) >> > >>>> >> > >>>> This will use ``/dev/urandom`` on UNIX platforms, and >> > >>>> ``CryptGenRandom`` on >> > >>>> Windows. >> > >>>> >> > >>>> If you need your random number as an integer (for example, for >> > >>>> :meth:`~cryptography.x509.CertificateBuilder.serial_number`), you >> can >> > >>>> use >> > >>>> ``int.from_bytes`` to convert the result of ``os.urandom``: >> > >>>> >> > >>>> .. code-block:: pycon >> > >>>> >> > >>>> >>> serial = int.from_bytes(os.urandom(20), byteorder="big") >> > >>>> >> > >>>> In addition, the `Python standard library`_ includes the >> ``secrets`` >> > >>>> module, >> > >>>> which can be used for generating cryptographically secure random >> > >>>> numbers, with >> > >>>> specific helpers for text-based formats. >> > >>>> >> > >>>> .. _`always use your operating system's provided random number >> > >>>> generator`: >> > >>>> >> https://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/ >> > >>>> .. _`Python standard library`: >> > >>>> https://docs.python.org/3/library/secrets.html >> > >>>> >> > >>>> ``` >> > >>>> >> > >>>> >> > >>>> On Mon, Nov 14, 2022, 10:57 AM Barry <ba...@barrys-emacs.org> >> wrote: >> > >>>> >> > >>>>> >> > >>>>> >> > >>>>> > On 14 Nov 2022, at 14:31, James Johnson <jj126...@gmail.com> >> wrote: >> > >>>>> > >> > >>>>> > >> > >>>>> > I wrote the attached python (3) code to improve on existing >> prng >> > >>>>> functions. I used the time module for one method, which resulted >> in >> > >>>>> disproportionate odd values, but agreeable means. >> > >>>>> > >> > >>>>> > I used the hashlib module for the second. It is evident that >> the >> > >>>>> code is amateur, but the program might result in better PRN >> generation. >> > >>>>> > >> > >>>>> > The "app" lends itself to checking, using statistical tools >> (see >> > >>>>> comments.) >> > >>>>> >> > >>>>> Have you used any cryptographic tools to prove the quality of >> your >> > >>>>> PRNG? >> > >>>>> What results did you get? >> > >>>>> How does your PRNG compare to what python already has? >> > >>>>> >> > >>>>> Without that this analysis this will be unlikely to be >> considered as a >> > >>>>> candidate for python stdlib. >> > >>>>> >> > >>>>> Barry >> > >>>>> >> > >>>>> > >> > >>>>> > I remain a fan, >> > >>>>> > >> > >>>>> > James Johnson >> > >>>>> > <testrandom.py> >> > >>>>> > _______________________________________________ >> > >>>>> > Python-ideas mailing list -- python-ideas@python.org >> > >>>>> > To unsubscribe send an email to python-ideas-le...@python.org >> > >>>>> > >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> > >>>>> > Message archived at >> > >>>>> >> https://mail.python.org/archives/list/python-ideas@python.org/message/VENAT2YTVYVHTBSEVHHMIURCU6MG2CEO/ >> > >>>>> > Code of Conduct: http://python.org/psf/codeofconduct/ >> > >>>>> >> > >>>>> _______________________________________________ >> > >>>>> Python-ideas mailing list -- python-ideas@python.org >> > >>>>> To unsubscribe send an email to python-ideas-le...@python.org >> > >>>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> > >>>>> Message archived at >> > >>>>> >> https://mail.python.org/archives/list/python-ideas@python.org/message/HWQV4AKQAUM7CY4NWS2IRIVLRAYMKR3V/ >> > >>>>> Code of Conduct: http://python.org/psf/codeofconduct/ >> > >>>>> >> > >>>> -- >> > Truth causes consequences; consequences bring pain; pain exorcises >> guilt! >> > _______________________________________________ >> > Python-ideas mailing list -- python-ideas@python.org >> > To unsubscribe send an email to python-ideas-le...@python.org >> > https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> > Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/ODL773AIV43GLUSO5UUNIPZYTPYJPQOZ/ >> > Code of Conduct: http://python.org/psf/codeofconduct/ >> >> Footnotes: >> [1] He composed several pseudo-random transformations, mostly linear >> congruential but also including at least one nonlinear one (middle >> square). The resulting sequence always converged to a short cycle >> within a handful of iterations. >> >>
randbelow.py
Description: Binary data
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XOX7RFI75Q3RT6JU357QVKNL3X5GXWD3/ Code of Conduct: http://python.org/psf/codeofconduct/
