On Wed, Feb 14, 2018 at 12:42 AM, Chris Jerdonek
<chris.jerdo...@gmail.com> wrote:
> Thanks, Dima and Andrew, for your suggestions.
> Re: loop.slow_callback_duration, that's interesting. I didn't know
> about that. However, it doesn't seem like that alone would work to
> catch many operations that are normally fast, but strictly speaking
> blocking. For example, it seems like simple disk I/O operations
> wouldn't be caught even with slow_callback_duration set to a small
> value. Dima suggests that such calls are okay. Is there a consensus on
> that?

It's not generally possible to avoid occasional arbitrary blocking,
e.g. due to the GC running, the OS scheduler, page faults, etc.
Basically the problem caused by blocking is when it means that other
tasks are stuck waiting when they could be getting useful work done.
If callbacks are finishing quickly then this isn't happening, so
slow_callback_duration is checking for exactly the right thing.

Where it might fall down is for operations that are only occasionally
slow, so they slip past your testing. E.g. if your disk is fast when
testing on your developer machine, but then in production you run on
some high-occupancy cloud host and a noisy neighbor starts pounding
the disk and suddenly your disk latencies shoot up.

> Dima's mock.patch_all_known_blocking_calls() is an interesting idea
> and seems like it would work for the case I mentioned. Has anyone
> started writing such a method (e.g. for certain standard lib modules)?

The implementation of gevent.monkey.patch_all() is probably not too
far from what you want.


Nathaniel J. Smith -- https://vorpus.org
Async-sig mailing list
Code of Conduct: https://www.python.org/psf/codeofconduct/

Reply via email to