This proposal might be of interest to folks here:
https://discuss.python.org/t/preventing-yield-inside-certain-context-managers/1091
I posted the main text there instead of here because it's a change to
the core interpreter, but the motivation is async APIs.
Probably best to post replies there, t
On Wed, Mar 27, 2019 at 1:49 PM Guido van Rossum wrote:
>
> On Wed, Mar 27, 2019 at 1:23 PM Nathaniel Smith wrote:
>>
>> On Wed, Mar 27, 2019 at 10:44 AM Daniel Nugent wrote:
>> >
>> > FWIW, the ayncio_run_encapsulated approach does not work with the
>
On Wed, Mar 27, 2019 at 10:44 AM Daniel Nugent wrote:
>
> FWIW, the ayncio_run_encapsulated approach does not work with the
> transport/protocol apis because the loop needs to stay alive concurrent with
> the connection in order for the awaitables to all be on the same loop.
Yeah, there are two
On Mon, Mar 25, 2019 at 4:37 PM Guido van Rossum wrote:
>
> I also hope Nathaniel has something to say -- I wonder if trio supports
> nested event loops?
Trio does have a similar check to prevent starting a new Trio loop
inside a running Trio loop, and there's currently no way to disable
it:
ht
On Mon, Feb 25, 2019 at 4:15 PM Josh Quigley <0zer...@gmail.com> wrote:
>
> I've realised the error of my ways: because Task separates the scheduling
> from the response handling, you cannot know if an exception is unhandled
> until the task is deleted. So in my example the reference means the ta
I'm not sure what a "higher priority" exception is...raising an
exception is hard to miss.
There are a few things Trio does differently here that might be
relevant, but it depends on why Chris is having trouble cancelling
tasks.
1. Trio's cancellation exception, trio.Cancelled, inherits from
Base
Yeah, 'await' makes it *possible* for the function you're calling to return
control to the event loop, but returning is still an explicit action that
the function has to take.
In asyncio, the only operation that actually returns control to the event
loop is awaiting a Future object.
In Trio, we m
On Sat, Aug 18, 2018 at 2:44 PM, Chris Jerdonek
wrote:
> The kind of alternative I had in mind for a neutral location is
> setting an attribute with an agreed upon name on a module in the
> standard lib, perhaps something like
> `contextvars.current_async_library_cvar` to use your naming. This is
On Sat, Aug 18, 2018, 01:22 Chris Jerdonek wrote:
> Also, just to be clear, I think the idea of a library to sniff this
> information is great.
>
> It's just the location of where this information is being stored that
> I'm focusing in on and asking about. It seems like it should be in a
> "neutr
On Fri, Aug 17, 2018 at 11:44 PM, Chris Jerdonek
wrote:
> On Fri, Aug 17, 2018 at 12:50 PM, Nathaniel Smith wrote:
>> On Fri, Aug 17, 2018, 12:12 Chris Jerdonek wrote:
>>>
>>> Did you also think about whether it would be possible for a library to
>>> advertis
On Fri, Aug 17, 2018, 12:12 Chris Jerdonek wrote:
> If I'm reading the docs correctly, it looks like an async library has
> to depend on sniffio in order to be detected by sniffio:
>
> https://sniffio.readthedocs.io/en/latest/#adding-support-to-a-new-async-library
If you don't want to depend th
On Fri, Aug 17, 2018, 09:09 Alex Grönholm wrote:
> This was my approach:
>
> def _detect_running_asynclib() -> str:
> if 'trio' in sys.modules:
> from trio.hazmat import current_trio_token
> try:
> current_trio_token()
> except RuntimeError:
> p
Hi all,
A number of people are working on packages that support multiple async
backends (e.g., asyncio + trio, or trio + curio, or trio + twisted,
...). So then the question arises... how can I figure out which async
library my user is actually using?
Answer: install sniffio, and then call
sniffi
Using a thread pool is the right answer. There's some discussion here:
https://github.com/python-trio/trio/issues/20
One thing Python could do to make this more effective would be to add
a "buffer-only" mode for operations on file objects:
https://bugs.python.org/issue32561
-n
On Sat, Jun 9, 201
On Sun, May 13, 2018 at 1:40 AM, twisteroid ambassador
wrote:
> Repo: https://github.com/twisteroidambassador/async_stagger
> Docs: http://async-stagger.readthedocs.io/en/latest/
>
> Provides near drop-in replacements for open_connection() and
> create_connection() using Happy Eyeballs. Also expos
On Wed, Apr 25, 2018 at 3:17 AM, Antoine Pitrou wrote:
> On Wed, 25 Apr 2018 02:24:15 -0700
> Nathaniel Smith wrote:
>> Hi all,
>>
>> I just posted another essay on concurrent API design:
>>
>> https://vorpus.org/blog/notes-on-structured-concurrency-or-go-stat
On Wed, Apr 25, 2018 at 9:43 PM, Guido van Rossum wrote:
> Now there's a PEP I'd like to see.
Which part?
-n
--
Nathaniel J. Smith -- https://vorpus.org
___
Async-sig mailing list
Async-sig@python.org
https://mail.python.org/mailman/listinfo/async-si
On Thu, Apr 26, 2018 at 7:55 PM, Dima Tisnek wrote:
> My 2c after careful reading:
>
> restarting tasks automatically (custom nursery example) is quite questionable:
> * it's unexpected
> * it's not generally safe (argument reuse, side effects)
> * user's coroutine can be decorated to achieve same
On Wed, Apr 25, 2018, 06:34 Mark E. Haase wrote:
>
> This is also an interesting approach that I will experiment with. I guess
> this solves problem #1 (works on Windows) but not #2 (task stuck in loop),
> right? (The latter is a feature of all cooperative multitasking systems,
> yeah?)
>
If a t
Hi all,
I just posted another essay on concurrent API design:
https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
This is the one that finally gets at the core reasons why Trio exists;
I've been trying to figure out how to write it for at least a year
now.
On Tue, Apr 24, 2018 at 2:25 PM, Mark E. Haase wrote:
> My mental model of how the event loop works is pretty poor, but I roughly
> understand that the event loop is responsible for driving coroutines. It
> appears
> here that the event loop has stopped driving my main() coroutine, and so the
> on
Pretty sure you want to add a try/finally around that yield, so you release
the lock on errors.
On Tue, Apr 17, 2018, 14:39 Ludovic Gasc wrote:
> 2018-04-17 15:16 GMT+02:00 Antoine Pitrou :
>
>>
>>
>> You could simply use something like the first 64 bits of
>> sha1("myapp:")
>>
>
> I have follow
On Wed, Feb 14, 2018 at 12:42 AM, Chris Jerdonek
wrote:
> Thanks, Dima and Andrew, for your suggestions.
>
> Re: loop.slow_callback_duration, that's interesting. I didn't know
> about that. However, it doesn't seem like that alone would work to
> catch many operations that are normally fast, but s
On Sun, Jan 14, 2018 at 6:33 PM, Dima Tisnek wrote:
> Perhaps the latter is what `shield` should do? That is detach computation as
> opposed to blocking the caller past caller's deadline?
Well, it can't do that in trio :-). One of trio's core design
principles is: no detached processes.
And even
On Sun, Jan 14, 2018 at 2:45 PM, Nick Badger wrote:
>> However, I think this is probably a code smell. Like all code smells,
>> there are probably cases where it's the right thing to do, but when
>> you see it you should stop and think carefully.
>
> Huh. That's a really good point. But I'm not su
On Sun, Jan 14, 2018 at 5:11 AM, Chris Jerdonek
wrote:
> On Sun, Jan 14, 2018 at 3:33 AM, Nathaniel Smith wrote:
>> On Fri, Jan 12, 2018 at 4:17 AM, Chris Jerdonek
>> wrote:
>>> Say you have a complex operation that you want to be able to timeout
>>> or c
On Fri, Jan 12, 2018 at 4:17 AM, Chris Jerdonek
wrote:
> Thanks, Nathaniel. Very instructive, thought-provoking write-up!
>
> One thing occurred to me around the time of reading this passage:
>
>> "Once the cancel token is triggered, then all future operations on that
>> token are cancelled, so t
On Thu, Jan 11, 2018 at 7:49 PM, Dima Tisnek wrote:
> Very nice read, Nathaniel.
>
> The post left me wondering how cancel tokens interact or should
> logically interact with async composition, for example:
>
> with move_on_after(10):
> await someio.gather(a(), b(), c())
>
> or
>
> with move_o
Hi all,
Folks here might be interested in this new blog post:
https://vorpus.org/blog/timeouts-and-cancellation-for-humans/
It's a detailed discussion of pitfalls and design-tradeoffs in APIs
for timeout and cancellation, and has a proposal for handling them in
a more Pythonic way. Any feedback
I haven't thought about this enough to have an opinion about whether
this is correct or how it could be improved, but I can explain why
you're seeing what you're seeing :-).
The traceback is really a trace of where the exception went after it
was raised, with new lines added to the top as it bubbl
Hi all,
I'm proud to announce the release of Trio v0.2.0. Trio is a new async
concurrency library for Python that's obsessed with usability and
correctness -- we want to make it easy to get things right. This is
the second public release, and it contains major new features and
bugfixes from 14 con
On Tue, Nov 14, 2017 at 6:54 AM, Mark E. Haase wrote:
> If an exception is thrown while the `asyncio` event loop is running, the
> stack trace is pretty complicated. Here's an example:
>
[...]
>
> I'm posting here to get constructive criticism on the concept and would also
> like to hear if Curio
On Tue, Nov 14, 2017 at 2:00 PM, Roger Pate wrote:
> On Tue, Nov 14, 2017 at 9:54 AM, Mark E. Haase wrote:
> ...
>> print('Async Traceback (most recent call last):')
>> for frame in traceback.extract_tb(tb):
>> head, tail = os.path.split(frame.filename)
>>
On Sun, Nov 12, 2017 at 7:27 AM, Guido van Rossum wrote:
> On Sun, Nov 12, 2017 at 2:53 AM, Chris Jerdonek
> wrote:
>>
>> By the way, since we're already on the subject of asyncio tasks and
>> (truncated) stack traces, this looks like a good opportunity to ask a
>> question that's been on my mind
On Fri, Nov 10, 2017 at 9:52 PM, Chris Jerdonek
wrote:
> Hi, I recently encountered a situation with asyncio where the stack
> trace is getting truncated: an exception isn't getting chained as
> expected.
>
> I was able to reduce it down to the code below.
>
> The reduced case seems like a pattern
On Mon, Aug 21, 2017 at 9:15 AM, Yury Selivanov wrote:
> Hi Pau,
>
> I personally don't think we need this in asyncio. While the function has a
> relatively low overhead, it's still an overhead, it's a couple more syscalls
> on each loop iteration, and it's a bit of a technical debt.
>
> The bar
It looks like your "load average" is computing something very different
than the traditional Unix "load average". If I'm reading right, yours is a
measure of what percentage of the time the loop spent sleeping waiting for
I/O, taken over the last 60 ticks of a 1 second timer (so generally
slightly
On Tue, Jul 11, 2017 at 3:26 PM, Laurent Mazuel via Async-sig
wrote:
> Hello,
>
> I’m working currently with Brett Cannon to bring asyncio support to our SDK.
> We wanted to check with you one of the scenario, since we got a lng
> discussion on it together 😊. And we want to do it using the b
On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm wrote:
> The real question is: why doesn't vanilla Sphinx have any kind of support
> for async functions which have been part of the language for quite a while?
Because no-one's sent them a PR, I assume. They're pretty swamped AFAICT.
One of the mai
On Jul 1, 2017 3:11 AM, "Chris Jerdonek" wrote:
I have a question about testing async code.
Say I have a coroutine:
async def do_things():
await do_something()
await do_more()
await do_even_more()
And future:
task = ensure_future(do_things())
Is there a way to
If we're citing curio and sphinxcontrib-asyncio I guess I'll also
mention sphinxcontrib-trio [1], which was inspired by both of them
(and isn't in any way specific to trio). I don't know if the python
docs can use third-party sphinx extensions, though, and it is a bit
opinionated (in particular it
On Mon, Jun 26, 2017 at 6:41 PM, Chris Jerdonek
wrote:
> On Mon, Jun 26, 2017 at 12:37 PM, Dima Tisnek wrote:
>> Chris, here's a simple RWLock implementation and analysis:
>> ...
>> Obv., this code could be nicer:
>> * separate context managers for read and write cases
>> * .unlock can be automat
On Tue, Jun 27, 2017 at 4:15 AM, Chris Jerdonek
wrote:
> On Tue, Jun 27, 2017 at 3:29 AM, Nathaniel Smith wrote:
>> In fact asyncio.Lock's implementation is careful to maintain strict
>> FIFO fairness, i.e. whoever calls acquire() first is guaranteed to get
>> the l
On Tue, Jun 27, 2017 at 12:29 AM, Chris Jerdonek
wrote:
> I have a couple questions about asyncio's synchronization primitives.
>
> Say a coroutine acquires an asyncio Condition's underlying lock, calls
> notify() (or notify_all()), and then releases the lock. In terms of
> which coroutines will a
I actually thought that async generators already guarded against this using
their ag_running attribute. If I try running Dima's example with
async_generator, I get:
sending user-1
received user-1
sending user-2
sending user-0
Traceback (most recent call last):
[...]
ValueError: async generator alr
On Mon, Jun 26, 2017 at 12:37 PM, Dima Tisnek wrote:
> Note that `.unlock` cannot validate that it's called by same coroutine
> as `.lock` was.
> That's because there's no concept for "current_thread" for coroutines
> -- there can be many waiting on each other in the stack.
This is also a surpris
On Sun, Jun 25, 2017 at 2:13 PM, Chris Jerdonek
wrote:
> I'm relatively new to async programming in Python and am thinking
> through possibilities for doing "read-write" synchronization.
>
> I'm using asyncio, and the synchronization primitives that asyncio
> exposes are relatively simple [1]. Hav
SSLObject.unwrap has the contract that if it finishes successfully, then
the SSL connection has been cleanly shut down and both sides remain in
sync, and can continue to use the socket in unencrypted mode. When asyncio
calls unwrap before the handshake has completed, then this contract is
impossibl
A few quick thoughts:
You might find these notes interesting:
https://github.com/python-trio/trio/issues/32
It sounds like a more precise description of your scheduler would be
"hierarchical FIFO"? i.e., there's a top-level scheduler that selects
between "child" schedulers in round-robin/FIFO fas
On Thu, Jun 8, 2017 at 3:32 PM, manuel miranda wrote:
> Hello everyone,
>
> After using asyncio for a while, I'm struggling to find information about
> how to support both synchronous and asynchronous use cases for the same
> library.
>
> I.e. imagine you have a package for http requests and you w
[dropped python-announce from CC list]
On Fri, May 12, 2017 at 9:23 AM, Yury Selivanov wrote:
> I like it!
>
> Do you have support for hybrid iterators/context managers:
>
> async with con.transaction() as tr
> # or
> tr = await con.transaction()
>
> and
>
> async for row in con.cursor(’S
[dropped python-announce from CC list]
On Fri, May 12, 2017 at 9:17 AM, Brett Cannon wrote:
> So are you going to try to upstream this? ;)
Realistically, for me this is a side-project to a side-project, so it
may require someone else do the integration work, but:
https://github.com/sphinx-doc/s
Hi all,
I just released a new package, sphinxcontrib-trio:
https://sphinxcontrib-trio.readthedocs.io/
It makes it easier to document many kinds of functions/methods in
sphinx, including async functions, abstract methods, generators, etc.
I originally wrote it for the trio [1] project, hence
Hi async-sig,
Some of you have probably already seen this, but in case it's of
interest, I just posted some discussion of signal handling and how I
approach it in trio:
https://vorpus.org/blog/control-c-handling-in-python-and-trio/
If nothing else at least it's the signal handling documentatio
On Mar 13, 2017 11:04, "Brett Cannon" wrote:
I assume the name is a reference to the fact that there's now a three-way
competition for ultimate async library? :)
Eh, there's at least twisted and gevent they it's not really even a
competitor to.
I came up with some justification for the name ab
Hi all,
I'd like to announce the initial release of Trio, a new
permissively-licensed async I/O library for Python 3.5+.
Blog post with more details:
https://vorpus.org/blog/announcing-trio/
Or you can jump straight to the repo:
https://github.com/python-trio/trio/
Cheers,
-n
--
Natha
On Nov 26, 2016 2:07 AM, "Alex Grönholm" wrote:
>
> 26.11.2016, 09:47, Nathaniel Smith kirjoitti:
>>
>> On Fri, Nov 25, 2016 at 10:46 AM, Alex Grönholm
wrote:
>>>
>>> 25.11.2016, 12:09, Nathaniel Smith kirjoitti:
>>>>
>>
On Fri, Nov 25, 2016 at 10:46 AM, Alex Grönholm
wrote:
> 25.11.2016, 12:09, Nathaniel Smith kirjoitti:
>>
>> On Thu, Nov 24, 2016 at 11:59 PM, Alex Grönholm
>> wrote:
>>>
>>> 25.11.2016, 09:25, Nathaniel Smith kirjoitti:
>>>>
>>&
On Fri, Nov 25, 2016 at 12:03 AM, Alex Grönholm
wrote:
> 24.11.2016, 23:23, Nathaniel Smith kirjoitti:
>
> On Nov 23, 2016 11:29 PM, "Alex Grönholm" wrote:
>>
>> 23.11.2016, 01:34, Nathaniel Smith kirjoitti:
>>>
>>> On Tue, Nov 22, 2016 at 2:22
On Thu, Nov 24, 2016 at 11:59 PM, Alex Grönholm
wrote:
> 25.11.2016, 09:25, Nathaniel Smith kirjoitti:
>>
>> On Thu, Nov 24, 2016 at 1:23 PM, Nathaniel Smith wrote:
>> [...]
>>>>
>>>> One thing I noticed is that there seems to be no way to
On Thu, Nov 24, 2016 at 1:23 PM, Nathaniel Smith wrote:
[...]
>> One thing I noticed is that there seems to be no way to detect async
>> generator functions in your implementation. That is something I would want
>> to have before switching.
>
> Good point. That should
On Nov 23, 2016 11:29 PM, "Alex Grönholm" wrote:
>
> 23.11.2016, 01:34, Nathaniel Smith kirjoitti:
>>
>> On Tue, Nov 22, 2016 at 2:22 PM, Alex Grönholm
wrote:
>> > I'm not sure where asyncio_extras's async generator implementation
assumes
>>
ious approach would be for
async_extras to just depend on async_generator, since AFAICT
async_generator is pretty much complete, and like you say, there's not much
point in carrying around two copies of the same thing. But if you have
another suggestion I'd be interested to hear it...
-n
>
On Nov 22, 2016 6:22 AM, "Alex Grönholm" wrote:
>
> Does this library offer any advantages over the same functionality in
asyncio_extras?
You would probably know better than me, but last time I looked, the
generator code in asyncio_extras didn't support asend/athrow/aclose, didn't
support yield f
What's the use case for the async generator version? Could the yield be
replaced by 'await loop.shutting_down()'?
On Nov 16, 2016 10:12 AM, "Yury Selivanov" wrote:
> One of the remaining problems with the event loop in asyncio is
> bootstrapping/finalizing asyncio programs.
>
> Currently, there
Hi all,
I just released v1.2 of my async_generator package:
https://pypi.org/pypi/async_generator/
This package makes it easy to write PEP 525-style "async generators",
even on Python 3.5. It looks like:
from async_generator import async_generator, yield_, yield_from_
@async_generato
I just posted a long blog/essay that's probably of interest to folks here:
https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/
The short version: I think curio something important to teach us; I
tried to figure out what that is and how we can learn from
On Sun, Oct 9, 2016 at 9:50 AM, Guido van Rossum wrote:
> I think I'm convinced by Amber's argument. Coroutines are something
> you can look up in e.g. Knuth (or Wikipedia) and you'll find something
> a pretty good match. All queries for "async functions" seem to go to
> some ECMAScript proposal (
t maybe it's worth expanding the scope
> to python-ideas? Not too many people read async-sig. (Or should that
> be coroutine-sig? :-)
>
> On Sat, Oct 8, 2016 at 8:50 PM, Nathaniel Smith wrote:
>> On Sat, Oct 8, 2016 at 7:48 PM, Guido van Rossum wrote:
>>> I've h
On Sat, Oct 8, 2016 at 9:10 PM, Amber "Hawkie" Brown
wrote:
>
>> On 9 Oct. 2016, at 14:50, Nathaniel Smith wrote:
>>
>> On Sat, Oct 8, 2016 at 7:48 PM, Guido van Rossum wrote:
>>> I've heard people call it an "async def" too.
>>
On Sat, Oct 8, 2016 at 7:48 PM, Guido van Rossum wrote:
> I've heard people call it an "async def" too.
>
> I don't think it's quite as dramatic as you worry about. People also
> talk about generators (not generator functions) and even though
> there's a further ambiguity between the function and
I've found that when talking about async/await stuff recently, I've mostly
dropped the word "coroutine" from my vocabulary and replaced it with "async
function". I'm writing to suggest that we might want to make this switch as
a community, and do it now, before the next 10x increase in the async/aw
On Wed, Jul 6, 2016 at 6:17 PM, Yury Selivanov wrote:
>
>> ...does it actually work to re-enter a main loop from inside a __del__
>> callback? It seems like you could get into really nasty states with
>> multiple nested __del__ calls, or if a single sweep detects multiple
>> pieces of garbage with
On Wed, Jul 6, 2016 at 5:47 PM, Glyph Lefkowitz wrote:
>
> On Jul 6, 2016, at 5:25 PM, Yury Selivanov wrote:
>
> The problem is that the GC can’t execute async code, and we don’t have any
> control over GC. What if we add a mechanism to control how async generators
> (AG) are destructed. Let’s
On Wed, Jul 6, 2016 at 1:12 PM, Yury Selivanov wrote:
> This is an interesting idea, but I wonder if instead of using ‘async with’ we
> can actually augment ‘async for’ to do the async finalization.
>
> We can add an __aiter_close__ special method which will return an awaitable.
> In this case,
On Wed, Jul 6, 2016 at 6:42 AM, Cory Benfield wrote:
>
>> On 6 Jul 2016, at 13:09, David Beazley wrote:
>>
>> Curio uses asynchronous context managers for much more than closing sockets
>> (which frankly is the least interesting thing). For example, they're used
>> extensively with synchroniz
So here's an interesting issue I just discovered while experimenting
with async generators. It caught me by surprise, anyway. Maybe this
was already obvious to everyone else. But I wanted to get some more
perspectives.
Starting axiom: async functions / async generators need to be prepared
for the
77 matches
Mail list logo
