I apologize, Maybe I wasn't clear. I'm voting in favor of Victor's PEP 524, i.e. making os.urandom() always blocking, over introducing an exception so rare that it's impossible to provoke without mocking entirely.
We may be trying to steer people away from os.urandom(), but it's a venerable API that has been around and stable since Python 2.4. Telling people "oh, BTW, you must now catch BlockingIOError in your code that calls os.urandom()" is a problem for straddling code, since BlockingIOError doesn't even exist in Python 2 (though its base class, OSError, does). Having to think about the consequences of os.urandom() blocking is necessary regardless of which PEP is implemented, since on some platforms it will block (though rarely). And the correct response is almost always "so let it block in those rare cases, it's just going to be a little hiccup". People working in an asyncio world may want to send secure random calls off to the thread pool using e.g. `loop.run_in_executor(None, os.urandom, 128)` -- but they have to think that way anyways because of other platforms, and they will have to do this for the recommended higher-level secure random APIs too. I really see a much bigger downside to adding the possibility that os.urandom() raises BlockingIOError, compared to accepting the possibility that it may block (which is hardly news). There is one thing that is still really unresolved for me, and that is a good understanding of how likely this feared event, "not having enough entropy" actually is, for environments where Python may actually be used. My main question is, can it occur in situations *other* than during very early startup? What's the answer for various platforms? Once I'm past this boot phase, can I safely assume os.urandom() will never block, or is there still a possibility for a system to run out of entropy later (say, by excessive calls to os.urandom(), possibly in another process)? The text of https://www.python.org/dev/peps/pep-0522/#adding-secrets-wait-for-system-rng suggests that that is *not* a possibility (since it recommends putting that call in __main__). Anyways, if the answer ends up being "yes, some systems may occasionally run out of entropy during normal operation", I would count that as a further point against PEP 522. But, assuming I am asked for a vote, my vote goes to Victor's PEP 524, making os.urandom() occasionally block even on Linux, and adding os.getrandom() on those platforms that have it. --Guido On Sat, Aug 6, 2016 at 7:29 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > On 6 August 2016 at 20:23, Victor Stinner <victor.stin...@gmail.com> wrote: >> An alternative would be to add to my PEP 524 an *optional* >> random.SystemRandomNonblock which is basically the PEP 522 (raise if it >> would block). "Optional"... or maybe make it always available but block(!) >> on some platforms? (Bad idea IMO) > > No, we don't want anything new added to the random module for this - > outside backwards compatibility considerations, random.SystemRandom > should be superseded entirely for security sensitive purposes by the > module level APIs in the secrets module. > >> I dislike the idea of adding 2 new functions to generate random in the same >> PEP (getrandom, SystemRandomNonblock), it's already hard enough to pick to >> right one in Python 3.5... > > With the changes to PEP 522, secrets.token_bytes will be a > cross-platform blocking API regardless of which underlying > implementation model we choose - either inheriting that behaviour from > os.urandom() (PEP 524), or adding it when encountering BlockingIOError > (PEP 522). > > That means the essential question becomes: Should os.urandom() just be > secrets.token_bytes() without a default number of bytes requested? Or > does it make more sense to use it to expose the Linux sys.getrandom() > non-blocking behaviour to Python code in a platform independent way? > > Since we're going to have the two level API anyway (os module vs > secrets), and have two different behaviours we'd like to expose > (blocking vs non-blocking with notification), the latter design is the > one I ended up converging on: high level API with implicit blocking, > low level API that never blocks, but may throw an exception. It's not > where I expected to end up when I first wrote the PEP, but that's the > PEP process for you :) > > Cheers, > Nick. > >> >> Victor >> >> >> Le 6 août 2016 10:46 AM, "Nick Coghlan" <ncogh...@gmail.com> a écrit : >>> >>> On 6 August 2016 at 18:32, Victor Stinner <victor.stin...@gmail.com> >>> wrote: >>> > Le 6 août 2016 04:39, "Guido van Rossum" <gu...@python.org> a écrit : >>> >> 4. Bad code gets cargo-culted (e.g. through StackOverflow). >>> > >>> >> Re (4): With PEP 524, people worried about blocking may be driven to >>> >> unnecessarily write more complicated code using os.getrandom(). With >>> >> PEP 522, people worried about crashes may be driven to unnecessarily >>> >> call secrets.wait_for_system_rng() or put try/except blocks catching >>> >> raise BlockingIOError around all their os.urandom()-based calls. >>> > >>> > What can we do to reduce this issue? Promote the best recipes in the >>> > documentation of the random and/or secrets module? Add Nick's >>> > secrets.wait_for_system_rng()? >>> >>> At the moment, PEP 522 doesn't propose making the secrets API block >>> implicitly. I was already starting to have doubts about that, and >>> given Guido's feedback, I think I should change it so that it does. >>> >>> That would give the following overall outcome: >>> >>> - the random APIs will never block (but shouldn't be used for secrets) >>> - the secrets APIs will block if they need to (including >>> secrets.wait_for_system_rng()) >>> - os.urandom() may raise BlockingIOError if you don't wait for the >>> system RNG first >>> - random.SystemRandom() may raise BlockingIOError if you don't wait >>> for the system RNG first >>> >>> And if in the latter two cases someone is directed to the secrets >>> module to wait for the system RNG to be ready (e.g. in the error >>> message we raise), they may find that secrets offers a higher level >>> API for whatever they were trying to do anyway. >>> >>> Meanwhile, folks that want to do something other than block if the >>> system RNG isn't ready (like log potentially relevant details of the >>> system encountering the lack of entropy) can just catch >>> BlockingIOError, rather than needing to use platform specific APIs >>> like os.getrandom(). >>> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > > > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia -- --Guido van Rossum (python.org/~guido) _______________________________________________ Security-SIG mailing list Security-SIG@python.org https://mail.python.org/mailman/listinfo/security-sig
