Re: [Async-sig] Asynchronous cleanup is a problem

2016-07-06 Thread Nathaniel Smith
On Wed, Jul 6, 2016 at 6:42 AM, Cory Benfield wrote: > >> On 6 Jul 2016, at 13:09, David Beazley wrote: >> >> Curio uses asynchronous context managers for much more than closing sockets >> (which frankly is the least interesting thing). For example, they're

[Async-sig] [ANN] async_generator v1.2 released

2016-11-14 Thread Nathaniel Smith
Hi all, I just released v1.2 of my async_generator package: https://pypi.org/pypi/async_generator/ This package makes it easy to write PEP 525-style "async generators", even on Python 3.5. It looks like: from async_generator import async_generator, yield_, yield_from_

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-25 Thread Nathaniel Smith
On Fri, Nov 25, 2016 at 10:46 AM, Alex Grönholm <alex.gronh...@nextday.fi> wrote: > 25.11.2016, 12:09, Nathaniel Smith kirjoitti: >> >> On Thu, Nov 24, 2016 at 11:59 PM, Alex Grönholm >> <alex.gronh...@nextday.fi> wrote: >>> >>> 25.11.2016, 09:2

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-26 Thread Nathaniel Smith
On Nov 26, 2016 2:07 AM, "Alex Grönholm" <alex.gronh...@nextday.fi> wrote: > > 26.11.2016, 09:47, Nathaniel Smith kirjoitti: >> >> On Fri, Nov 25, 2016 at 10:46 AM, Alex Grönholm <alex.gronh...@nextday.fi> wrote: >>> >>> 25.11.2016, 12:09,

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-22 Thread Nathaniel Smith
s to just depend on async_generator, since AFAICT async_generator is pretty much complete, and like you say, there's not much point in carrying around two copies of the same thing. But if you have another suggestion I'd be interested to hear it... -n > > 22.11.2016, 22:32, Nathaniel Smith kirjoitti: &

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-24 Thread Nathaniel Smith
On Nov 23, 2016 11:29 PM, "Alex Grönholm" <alex.gronh...@nextday.fi> wrote: > > 23.11.2016, 01:34, Nathaniel Smith kirjoitti: >> >> On Tue, Nov 22, 2016 at 2:22 PM, Alex Grönholm <alex.gronh...@nextday.fi> wrote: >> > I'm not sure where asynci

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-24 Thread Nathaniel Smith
On Thu, Nov 24, 2016 at 1:23 PM, Nathaniel Smith <n...@pobox.com> wrote: [...] >> One thing I noticed is that there seems to be no way to detect async >> generator functions in your implementation. That is something I would want >> to have before switching. > > Goo

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-25 Thread Nathaniel Smith
On Thu, Nov 24, 2016 at 11:59 PM, Alex Grönholm <alex.gronh...@nextday.fi> wrote: > 25.11.2016, 09:25, Nathaniel Smith kirjoitti: >> >> On Thu, Nov 24, 2016 at 1:23 PM, Nathaniel Smith <n...@pobox.com> wrote: >> [...] >>>> >>>> One thing I

Re: [Async-sig] [ANN] async_generator v1.2 released

2016-11-25 Thread Nathaniel Smith
On Fri, Nov 25, 2016 at 12:03 AM, Alex Grönholm <alex.gronh...@nextday.fi> wrote: > 24.11.2016, 23:23, Nathaniel Smith kirjoitti: > > On Nov 23, 2016 11:29 PM, "Alex Grönholm" <alex.gronh...@nextday.fi> wrote: >> >> 23.11.2016, 01:34, Nathaniel Smith kirj

Re: [Async-sig] Adding asyncio.run() function in Python 3.6

2016-11-16 Thread Nathaniel Smith
What's the use case for the async generator version? Could the yield be replaced by 'await loop.shutting_down()'? On Nov 16, 2016 10:12 AM, "Yury Selivanov" wrote: > One of the remaining problems with the event loop in asyncio is > bootstrapping/finalizing asyncio

Re: [Async-sig] Trio: async I/O for humans and snake people

2017-03-15 Thread Nathaniel Smith
On Mar 13, 2017 11:04, "Brett Cannon" wrote: I assume the name is a reference to the fact that there's now a three-way competition for ultimate async library? :) Eh, there's at least twisted and gevent they it's not really even a competitor to. I came up with some

[Async-sig] Trio: async I/O for humans and snake people

2017-03-10 Thread Nathaniel Smith
Hi all, I'd like to announce the initial release of Trio, a new permissively-licensed async I/O library for Python 3.5+. Blog post with more details: https://vorpus.org/blog/announcing-trio/ Or you can jump straight to the repo: https://github.com/python-trio/trio/ Cheers, -n --

Re: [Async-sig] Feedback, loop.load() function

2017-08-11 Thread Nathaniel Smith
It looks like your "load average" is computing something very different than the traditional Unix "load average". If I'm reading right, yours is a measure of what percentage of the time the loop spent sleeping waiting for I/O, taken over the last 60 ticks of a 1 second timer (so generally slightly

Re: [Async-sig] Optional async method and best practices

2017-07-12 Thread Nathaniel Smith
On Tue, Jul 11, 2017 at 3:26 PM, Laurent Mazuel via Async-sig wrote: > Hello, > > I’m working currently with Brett Cannon to bring asyncio support to our SDK. > We wanted to check with you one of the scenario, since we got a lng > discussion on it together . And we

Re: [Async-sig] async documentation methods

2017-07-01 Thread Nathaniel Smith
If we're citing curio and sphinxcontrib-asyncio I guess I'll also mention sphinxcontrib-trio [1], which was inspired by both of them (and isn't in any way specific to trio). I don't know if the python docs can use third-party sphinx extensions, though, and it is a bit opinionated (in particular it

Re: [Async-sig] async documentation methods

2017-07-04 Thread Nathaniel Smith
On Mon, Jul 3, 2017 at 11:49 PM, Alex Grönholm wrote: > The real question is: why doesn't vanilla Sphinx have any kind of support > for async functions which have been part of the language for quite a while? Because no-one's sent them a PR, I assume. They're pretty

Re: [Async-sig] async testing question

2017-07-01 Thread Nathaniel Smith
On Jul 1, 2017 3:11 AM, "Chris Jerdonek" wrote: I have a question about testing async code. Say I have a coroutine: async def do_things(): await do_something() await do_more() await do_even_more() And future: task =

Re: [Async-sig] question re: asyncio.Condition lock acquisition order

2017-06-27 Thread Nathaniel Smith
On Tue, Jun 27, 2017 at 4:15 AM, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > On Tue, Jun 27, 2017 at 3:29 AM, Nathaniel Smith <n...@pobox.com> wrote: >> In fact asyncio.Lock's implementation is careful to maintain strict >> FIFO fairness, i.e. whoever calls ac

Re: [Async-sig] "read-write" synchronization

2017-06-27 Thread Nathaniel Smith
On Mon, Jun 26, 2017 at 6:41 PM, Chris Jerdonek wrote: > On Mon, Jun 26, 2017 at 12:37 PM, Dima Tisnek wrote: >> Chris, here's a simple RWLock implementation and analysis: >> ... >> Obv., this code could be nicer: >> * separate context managers for

Re: [Async-sig] [ann] sphinxcontrib-trio: make sphinx better at documenting functions/methods, esp. for async/await code

2017-05-12 Thread Nathaniel Smith
[dropped python-announce from CC list] On Fri, May 12, 2017 at 9:17 AM, Brett Cannon wrote: > So are you going to try to upstream this? ;) Realistically, for me this is a side-project to a side-project, so it may require someone else do the integration work, but:

Re: [Async-sig] [ann] sphinxcontrib-trio: make sphinx better at documenting functions/methods, esp. for async/await code

2017-05-12 Thread Nathaniel Smith
[dropped python-announce from CC list] On Fri, May 12, 2017 at 9:23 AM, Yury Selivanov wrote: > I like it! > > Do you have support for hybrid iterators/context managers: > > async with con.transaction() as tr > # or > tr = await con.transaction() > > and > > async

[Async-sig] [ann] sphinxcontrib-trio: make sphinx better at documenting functions/methods, esp. for async/await code

2017-05-12 Thread Nathaniel Smith
Hi all, I just released a new package, sphinxcontrib-trio: https://sphinxcontrib-trio.readthedocs.io/ It makes it easier to document many kinds of functions/methods in sphinx, including async functions, abstract methods, generators, etc. I originally wrote it for the trio [1] project,

Re: [Async-sig] async/sync library reusage

2017-06-08 Thread Nathaniel Smith
On Thu, Jun 8, 2017 at 3:32 PM, manuel miranda wrote: > Hello everyone, > > After using asyncio for a while, I'm struggling to find information about > how to support both synchronous and asynchronous use cases for the same > library. > > I.e. imagine you have a package

Re: [Async-sig] Cancelling SSL connection

2017-06-21 Thread Nathaniel Smith
SSLObject.unwrap has the contract that if it finishes successfully, then the SSL connection has been cleanly shut down and both sides remain in sync, and can continue to use the socket in unencrypted mode. When asyncio calls unwrap before the handshake has completed, then this contract is

Re: [Async-sig] "read-write" synchronization

2017-06-26 Thread Nathaniel Smith
On Mon, Jun 26, 2017 at 12:37 PM, Dima Tisnek wrote: > Note that `.unlock` cannot validate that it's called by same coroutine > as `.lock` was. > That's because there's no concept for "current_thread" for coroutines > -- there can be many waiting on each other in the stack.

Re: [Async-sig] async generator confusion or bug?

2017-06-26 Thread Nathaniel Smith
I actually thought that async generators already guarded against this using their ag_running attribute. If I try running Dima's example with async_generator, I get: sending user-1 received user-1 sending user-2 sending user-0 Traceback (most recent call last): [...] ValueError: async generator

Re: [Async-sig] "read-write" synchronization

2017-06-25 Thread Nathaniel Smith
On Sun, Jun 25, 2017 at 2:13 PM, Chris Jerdonek wrote: > I'm relatively new to async programming in Python and am thinking > through possibilities for doing "read-write" synchronization. > > I'm using asyncio, and the synchronization primitives that asyncio > exposes are

Re: [Async-sig] Feedback, loop.load() function

2017-08-21 Thread Nathaniel Smith
On Mon, Aug 21, 2017 at 9:15 AM, Yury Selivanov wrote: > Hi Pau, > > I personally don't think we need this in asyncio. While the function has a > relatively low overhead, it's still an overhead, it's a couple more syscalls > on each loop iteration, and it's a bit of a

Re: [Async-sig] awaiting task is not chaining exception

2017-11-12 Thread Nathaniel Smith
On Sun, Nov 12, 2017 at 7:27 AM, Guido van Rossum wrote: > On Sun, Nov 12, 2017 at 2:53 AM, Chris Jerdonek > wrote: >> >> By the way, since we're already on the subject of asyncio tasks and >> (truncated) stack traces, this looks like a good

Re: [Async-sig] Simplifying stack traces for tasks?

2017-11-14 Thread Nathaniel Smith
On Tue, Nov 14, 2017 at 2:00 PM, Roger Pate wrote: > On Tue, Nov 14, 2017 at 9:54 AM, Mark E. Haase wrote: > ... >> print('Async Traceback (most recent call last):') >> for frame in traceback.extract_tb(tb): >> head, tail =

Re: [Async-sig] Simplifying stack traces for tasks?

2017-11-14 Thread Nathaniel Smith
On Tue, Nov 14, 2017 at 6:54 AM, Mark E. Haase wrote: > If an exception is thrown while the `asyncio` event loop is running, the > stack trace is pretty complicated. Here's an example: > [...] > > I'm posting here to get constructive criticism on the concept and would also >

Re: [Async-sig] awaiting task is not chaining exception

2017-11-11 Thread Nathaniel Smith
On Fri, Nov 10, 2017 at 9:52 PM, Chris Jerdonek wrote: > Hi, I recently encountered a situation with asyncio where the stack > trace is getting truncated: an exception isn't getting chained as > expected. > > I was able to reduce it down to the code below. > > The

[Async-sig] ANN: Trio v0.2.0 released

2017-12-07 Thread Nathaniel Smith
Hi all, I'm proud to announce the release of Trio v0.2.0. Trio is a new async concurrency library for Python that's obsessed with usability and correctness -- we want to make it easy to get things right. This is the second public release, and it contains major new features and bugfixes from 14

Re: [Async-sig] async_stagger: Happy Eyeballs implementation in pure asyncio

2018-05-15 Thread Nathaniel Smith
On Sun, May 13, 2018 at 1:40 AM, twisteroid ambassador wrote: > Repo: https://github.com/twisteroidambassador/async_stagger > Docs: http://async-stagger.readthedocs.io/en/latest/ > > Provides near drop-in replacements for open_connection() and >

Re: [Async-sig] async file IO library

2018-06-09 Thread Nathaniel Smith
Using a thread pool is the right answer. There's some discussion here: https://github.com/python-trio/trio/issues/20 One thing Python could do to make this more effective would be to add a "buffer-only" mode for operations on file objects: https://bugs.python.org/issue32561 -n On Sat, Jun 9,

Re: [Async-sig] New blog post: Notes on structured concurrency, or: Go statement considered harmful

2018-04-26 Thread Nathaniel Smith
On Thu, Apr 26, 2018 at 7:55 PM, Dima Tisnek wrote: > My 2c after careful reading: > > restarting tasks automatically (custom nursery example) is quite questionable: > * it's unexpected > * it's not generally safe (argument reuse, side effects) > * user's coroutine can be

Re: [Async-sig] New blog post: Notes on structured concurrency, or: Go statement considered harmful

2018-04-26 Thread Nathaniel Smith
On Wed, Apr 25, 2018 at 9:43 PM, Guido van Rossum wrote: > Now there's a PEP I'd like to see. Which part? -n -- Nathaniel J. Smith -- https://vorpus.org ___ Async-sig mailing list Async-sig@python.org

Re: [Async-sig] New blog post: Notes on structured concurrency, or: Go statement considered harmful

2018-04-26 Thread Nathaniel Smith
On Wed, Apr 25, 2018 at 3:17 AM, Antoine Pitrou <solip...@pitrou.net> wrote: > On Wed, 25 Apr 2018 02:24:15 -0700 > Nathaniel Smith <n...@pobox.com> wrote: >> Hi all, >> >> I just posted another essay on concurrent API design: >> >> https://vorpu

Re: [Async-sig] task.result() and exception traceback display

2017-12-25 Thread Nathaniel Smith
I haven't thought about this enough to have an opinion about whether this is correct or how it could be improved, but I can explain why you're seeing what you're seeing :-). The traceback is really a trace of where the exception went after it was raised, with new lines added to the top as it

[Async-sig] Blog post: Timeouts and cancellation for humans

2018-01-11 Thread Nathaniel Smith
Hi all, Folks here might be interested in this new blog post: https://vorpus.org/blog/timeouts-and-cancellation-for-humans/ It's a detailed discussion of pitfalls and design-tradeoffs in APIs for timeout and cancellation, and has a proposal for handling them in a more Pythonic way. Any feedback

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

2018-01-16 Thread Nathaniel Smith
On Sun, Jan 14, 2018 at 6:33 PM, Dima Tisnek wrote: > Perhaps the latter is what `shield` should do? That is detach computation as > opposed to blocking the caller past caller's deadline? Well, it can't do that in trio :-). One of trio's core design principles is: no detached

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

2018-01-14 Thread Nathaniel Smith
On Sun, Jan 14, 2018 at 5:11 AM, Chris Jerdonek <chris.jerdo...@gmail.com> wrote: > On Sun, Jan 14, 2018 at 3:33 AM, Nathaniel Smith <n...@pobox.com> wrote: >> On Fri, Jan 12, 2018 at 4:17 AM, Chris Jerdonek >> <chris.jerdo...@gmail.com> wrote: >>> Say

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

2018-01-14 Thread Nathaniel Smith
On Sun, Jan 14, 2018 at 2:45 PM, Nick Badger wrote: >> However, I think this is probably a code smell. Like all code smells, >> there are probably cases where it's the right thing to do, but when >> you see it you should stop and think carefully. > > Huh. That's a really good

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

2018-01-14 Thread Nathaniel Smith
On Fri, Jan 12, 2018 at 4:17 AM, Chris Jerdonek wrote: > Thanks, Nathaniel. Very instructive, thought-provoking write-up! > > One thing occurred to me around the time of reading this passage: > >> "Once the cancel token is triggered, then all future operations on that

Re: [Async-sig] Blog post: Timeouts and cancellation for humans

2018-01-13 Thread Nathaniel Smith
On Thu, Jan 11, 2018 at 7:49 PM, Dima Tisnek wrote: > Very nice read, Nathaniel. > > The post left me wondering how cancel tokens interact or should > logically interact with async composition, for example: > > with move_on_after(10): > await someio.gather(a(), b(), c()) > >

Re: [Async-sig] avoiding accidentally calling blocking code

2018-02-14 Thread Nathaniel Smith
On Wed, Feb 14, 2018 at 12:42 AM, Chris Jerdonek wrote: > Thanks, Dima and Andrew, for your suggestions. > > Re: loop.slow_callback_duration, that's interesting. I didn't know > about that. However, it doesn't seem like that alone would work to > catch many operations

Re: [Async-sig] Cancelling a coroutine from a signal handler?

2018-04-24 Thread Nathaniel Smith
On Tue, Apr 24, 2018 at 2:25 PM, Mark E. Haase wrote: > My mental model of how the event loop works is pretty poor, but I roughly > understand that the event loop is responsible for driving coroutines. It > appears > here that the event loop has stopped driving my main()

[Async-sig] New blog post: Notes on structured concurrency, or: Go statement considered harmful

2018-04-25 Thread Nathaniel Smith
Hi all, I just posted another essay on concurrent API design: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ This is the one that finally gets at the core reasons why Trio exists; I've been trying to figure out how to write it for at least a year

Re: [Async-sig] Cancelling a coroutine from a signal handler?

2018-04-25 Thread Nathaniel Smith
On Wed, Apr 25, 2018, 06:34 Mark E. Haase wrote: > > This is also an interesting approach that I will experiment with. I guess > this solves problem #1 (works on Windows) but not #2 (task stuck in loop), > right? (The latter is a feature of all cooperative multitasking

Re: [Async-sig] asyncio.Lock equivalent for multiple processes

2018-04-17 Thread Nathaniel Smith
Pretty sure you want to add a try/finally around that yield, so you release the lock on errors. On Tue, Apr 17, 2018, 14:39 Ludovic Gasc wrote: > 2018-04-17 15:16 GMT+02:00 Antoine Pitrou : > >> >> >> You could simply use something like the first 64 bits