[Async-sig] PR's re: asyncio troubleshooting / tracebacks

Hi folks, I wanted to raise visibility about a few asyncio PR's I'm hoping can be merged before the 3.9 beta deadline -- which is ~10 days away. The four PR's are listed at the end of this message. The PR's are on the simple side compared to others. I think collectively they can make asyncio a

Re: [Async-sig] killing tasks that won't cancel

On Tue, Feb 19, 2019 at 12:33 PM Yury Selivanov wrote: > Unfortunately asyncio isn't super flexible around "cancellation with a > timeout" kind of scenarios. The current assumption is that once the > cancellation is requested, the Task will start cancelling and will do so in > a timely manner.

Re: [Async-sig] killing tasks that won't cancel

caller doesn't want to continue waiting past a certain time. --Chris > The same as generator object technically can swallow GeneratorExit > (but such code is most likely buggy). > > On Tue, Feb 19, 2019 at 9:55 PM Chris Jerdonek > wrote: > > > > I have an asynci

Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under

On Sat, Aug 18, 2018 at 2:13 PM, Nathaniel Smith wrote: > On Sat, Aug 18, 2018, 01:22 Chris Jerdonek wrote: >> >> Also, just to be clear, I think the idea of a library to sniff this >> information is great. >> >> It's just the location of where this inform

Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under

On Fri, Aug 17, 2018 at 12:50 PM, Nathaniel Smith wrote: > On Fri, Aug 17, 2018, 12:12 Chris Jerdonek wrote: >> >> Did you also think about whether it would be possible for a library to >> advertise itself without having to depend on a third-party library >> (e.g. us

Re: [Async-sig] new library: sniffio – Sniff out which async library your code is running under

If I'm reading the docs correctly, it looks like an async library has to depend on sniffio in order to be detected by sniffio: https://sniffio.readthedocs.io/en/latest/#adding-support-to-a-new-async-library Did you also think about whether it would be possible for a library to advertise itself

Re: [Async-sig] "Coroutines" sometimes run without being scheduled on an event loop

It would probably be hard for people to find at this point all the places they might be relying on this behavior (if anywhere), but isn't this a basic documented property of coroutines? From the introduction section on coroutines [1]: > Calling a coroutine does not start its code running – the

Re: [Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

If you're already using PostgreSQL, you might also look at "advisory locks": https://www.postgresql.org/docs/current/static/explicit-locking.html#ADVISORY-LOCKS --Chris On Tue, Apr 17, 2018 at 4:34 AM, Ludovic Gasc wrote: > Hi Nickolai, > > Thanks for your suggestions,

Re: [Async-sig] avoiding accidentally calling blocking code

e: after the fact. >> hack1. run a separate thread/process that inspects event loop thread's >> `wchan` value (Linux) >> hack2. same via taskstats >> hack3. detect event loop lag (false positives are cpu-bound tasks and >> overloaded system, but I think you'd like to d

[Async-sig] avoiding accidentally calling blocking code

Hi, This is a general Python async question I've had that I haven't seen a discussion of. Do people know of any techniques other than manually inspecting code line by line to find out if you're accidentally making a raw call to a blocking function in a coroutine? Related to this, again, outside

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

On Sun, Jan 14, 2018 at 3:33 AM, Nathaniel Smith <n...@pobox.com> wrote: > On Fri, Jan 12, 2018 at 4:17 AM, Chris Jerdonek > <chris.jerdo...@gmail.com> wrote: >> Say you have a complex operation that you want to be able to timeout >> or cancel, but the process of c

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

Thanks, Nathaniel. Very instructive, thought-provoking write-up! One thing occurred to me around the time of reading this passage: > "Once the cancel token is triggered, then all future operations on that token > are cancelled, so the call to ws.close doesn't get stuck. It's a less >

[Async-sig] task.result() and exception traceback display

Hi, I noticed that if a task in asyncio raises an exception, then the displayed traceback can be "polluted" by intermediate calls to task.result(). Also, the calls to task.result() can appear out of order relative to each other and to other lines. Here is an example: import asyncio

Re: [Async-sig] subclassing CancelledError

t; wrote: >> >> IIRC at very early stages Guido van Rossum decided to *freeze* >> `CancelledError`: user code should not derive from the exception. Like you >> never derive from StopIteration. >> >> On Sun, Dec 3, 2017 at 8:00 AM Chris Jerdonek <

[Async-sig] subclassing CancelledError

Hi, I want to ask how people feel about the following. If you raise a subclass of CancelledError from within a task and then call task.result(), CancelledError is raised rather than the subclass. Here is some code to illustrate: class MyCancelledError(CancelledError): pass async def

Re: [Async-sig] awaiting task is not chaining exception

Python? --Chris On Sat, Nov 11, 2017 at 1:47 AM, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > On Sat, Nov 11, 2017 at 12:39 AM, Nathaniel Smith <n...@pobox.com> wrote: >> On Fri, Nov 10, 2017 at 9:52 PM, Chris Jerdonek >> <chris.jerdo...@gmail.com>

Re: [Async-sig] coroutine function vs. function returning awaitable

're asking all this.) > > On Thu, Nov 2, 2017 at 7:49 PM, Chris Jerdonek <chris.jerdo...@gmail.com> > wrote: >> >> Thanks. And how about making asyncio.iscoroutine() return True for the >> return value? It looks like asyncio calls that one in a half-dozen places or &

Re: [Async-sig] coroutine function vs. function returning awaitable

cio calls > `iscouroutinefunction()`, and in both cases it raises a TypeError to remind > the user that a coroutine/awaitable object is required. So, it doesn't > really offer much of an advantage and I wouldn't worry about it. > > On Thu, Nov 2, 2017 at 4:51 PM, Chris Jerdonek <chris.

[Async-sig] coroutine function vs. function returning awaitable

Hi, I have a 2-part question: 1) Say you have a function that returns an awaitable (but not a coroutine object). Are there any advantages or disadvantages to replacing this with a function for which asyncio.iscoroutinefunction() returns True and asyncio.iscoroutine() returns True for the return

Re: [Async-sig] Feedback, loop.load() function

On Sun, Aug 6, 2017 at 3:57 PM, Pau Freixes wrote: > Hi guys, > > I would appreciate any feedback about the idea of implementing a new > load function to ask about how saturated is your reactor. Hi, Would it be possible for you to rephrase what you've done in terms of

Re: [Async-sig] question re: loop.shutdown_asyncgens()

ze asyncio changes in 3.7. > I don't have a concrete ETA for it, but I'll try to get the first draft out > by mid September. > > [1] https://github.com/python/asyncio/pull/465 > > Thanks, > Yury > > On Jul 27, 2017, 11:24 PM -0400, Chris Jerdonek <chris.jerdo...@gmai

Re: [Async-sig] using asyncio in synchronous applications

On Tue, Jul 11, 2017 at 2:35 PM, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > On Tue, Jul 11, 2017 at 1:25 PM, Andrew Svetlov > <andrew.svet...@gmail.com> wrote: >> Hmm. After rethinking I see `set_event_loop()` is required in your design. >> But better to ha

Re: [Async-sig] Optional async method and best practices

On Wed, Jul 12, 2017 at 9:44 AM, Laurent Mazuel via Async-sig wrote: > @property > def future(self): > if self._future is None: > self._future = asyncio.Future() > asyncio.ensure_future(self._poll()) > return self._future > >

Re: [Async-sig] Optional async method and best practices

On Tue, Jul 11, 2017 at 3:26 PM, Laurent Mazuel via Async-sig wrote: > But I got a warning if I decide to do not use this poller, RuntimeWarning: > coroutine 'foo' was never awaited > ... > I found 2 solutions to avoid the warning, and I currently prefer solution 2: > 1-

Re: [Async-sig] using asyncio in synchronous applications

cking point is just whether it should go in the standard library. Thanks, --Chris > > The implementation doesn't touch default loop but `asyncio.get_event_loop()` > call from `coro` returns a running loop instance. > > > On Tue, Jul 11, 2017 at 10:12 PM Chris Jerdonek <chr

Re: [Async-sig] using asyncio in synchronous applications

that doesn't require calling set_event_loop()? --Chris > On Tue, Jul 11, 2017, 17:56 Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >> >> There's something I realized about "creating and destroying" ephemeral >> event loops if you want to create temporary event

Re: [Async-sig] using asyncio in synchronous applications

y on the client side as well. --Chris > > On Sun, Jul 9, 2017 at 8:48 PM, Chris Jerdonek <chris.jerdo...@gmail.com> > wrote: >> >> I have a two-part question. >> >> If my application is single-threaded and synchronous (e.g. a web app >> using Guni

Re: [Async-sig] async testing question

change, then fine :) > If you consider that unexpected breakage, then I dunno what you can do :P > > On 1 July 2017 at 22:06, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >> On Sat, Jul 1, 2017 at 3:49 AM, Dima Tisnek <dim...@gmail.com> wrote: >>> Hi Chr

Re: [Async-sig] async testing question

nnable tasks differently on each run. I hope someone rewrites this > for asyncio) > > Certainty [better] tools are needed, and ultimately it's a tradeoff between > sane/understable/maintainable tests and testing deeper/more corner cases. > > Just my 2c... > > On Jul 1, 2

Re: [Async-sig] question re: asyncio.Condition lock acquisition order

On Tue, Jun 27, 2017 at 3:48 PM, Nathaniel Smith <n...@pobox.com> wrote: > On Tue, Jun 27, 2017 at 4:15 AM, Chris Jerdonek >> Thinking through the requirements I want for my RW synchronization use >> case in more detail, I think I want the completion of any "write"

Re: [Async-sig] "read-write" synchronization

On Tue, Jun 27, 2017 at 3:52 PM, Nathaniel Smith <n...@pobox.com> wrote: > On Mon, Jun 26, 2017 at 6:41 PM, Chris Jerdonek > <chris.jerdo...@gmail.com> wrote: >> I coded up a working version of the pseudo-code I included in an >> earlier email so people ca

Re: [Async-sig] question re: asyncio.Condition lock acquisition order

On Tue, Jun 27, 2017 at 3:29 AM, Nathaniel Smith wrote: > In fact asyncio.Lock's implementation is careful to maintain strict > FIFO fairness, i.e. whoever calls acquire() first is guaranteed to get > the lock first. Whether this is something you feel you can depend on > I'll

[Async-sig] question re: asyncio.Condition lock acquisition order

I have a couple questions about asyncio's synchronization primitives. Say a coroutine acquires an asyncio Condition's underlying lock, calls notify() (or notify_all()), and then releases the lock. In terms of which coroutines will acquire the lock next, is any preference given between (1)

Re: [Async-sig] "read-write" synchronization

nding of async. --Chris > > Cheers, > d. > > On 25 June 2017 at 23:13, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: >> I'm relatively new to async programming in Python and am thinking >> through possibilities for doing "read-write" synchronization

Re: [Async-sig] "read-write" synchronization

, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > On Sun, Jun 25, 2017 at 3:09 PM, Nathaniel Smith <n...@pobox.com> wrote: >> On Sun, Jun 25, 2017 at 2:13 PM, Chris Jerdonek >> <chris.jerdo...@gmail.com> wrote: >>> I'm using asyncio, and the synchroni

Re: [Async-sig] "read-write" synchronization

On Sun, Jun 25, 2017 at 3:09 PM, Nathaniel Smith <n...@pobox.com> wrote: > On Sun, Jun 25, 2017 at 2:13 PM, Chris Jerdonek > <chris.jerdo...@gmail.com> wrote: >> I'm using asyncio, and the synchronization primitives that asyncio >> exposes are relatively simple [1

Re: [Async-sig] "read-write" synchronization

; don't need locks at all. > > On Jun 25, 2017 2:24 PM, "Chris Jerdonek" <chris.jerdo...@gmail.com> wrote: >> >> Thank you. I had seen that, but it seems heavier weight than needed. >> And it also requires locking on reading. >> >> --Chris >&

Re: [Async-sig] "read-write" synchronization

Thank you. I had seen that, but it seems heavier weight than needed. And it also requires locking on reading. --Chris On Sun, Jun 25, 2017 at 2:16 PM, Andrew Svetlov <andrew.svet...@gmail.com> wrote: > There is https://github.com/aio-libs/aiorwlock > > On Mon, Jun 26, 2017 at

[Async-sig] "read-write" synchronization

I'm relatively new to async programming in Python and am thinking through possibilities for doing "read-write" synchronization. I'm using asyncio, and the synchronization primitives that asyncio exposes are relatively simple [1]. Have options for async read-write synchronization already been