Re: [python-tulip] using coroutines within protocols?

2014-01-23 Thread Glyph Lefkowitz
On Jan 23, 2014, at 9:47 AM, Guido van Rossum wrote: > So I think we should just note that > protocols are a lower-level concept than StreamReader. (And more > versatile -- that's usually how lower-level concepts work. :-) +1. Even within Twisted, which follows the callback idiom thoroughly,

Re: [python-tulip] Re: change yield x to yield From(x) in Trollius?

2014-01-23 Thread Yury Selivanov
On 1/23/2014, 4:42 PM, Victor Stinner wrote: 2014/1/23 Yury Selivanov : What would be From(x)? A dummy function just returning x? def From(x): return x In production yes, but I would also suggest that in debug mode it should return a wrapper object, that is checked in your trampoline. Which

Re: [python-tulip] Re: change yield x to yield From(x) in Trollius?

2014-01-23 Thread Victor Stinner
2014/1/23 Yury Selivanov : >> What would be From(x)? A dummy function just returning x? >> >> def From(x): return x >> > In production yes, but I would also suggest that in debug mode > it should return a wrapper object, that is checked in your > trampoline. Which kind of check? Task._step() alrea

Re: [python-tulip] Re: change yield x to yield From(x) in Trollius?

2014-01-23 Thread Yury Selivanov
On 1/23/2014, 4:33 PM, Victor Stinner wrote: What would be From(x)? A dummy function just returning x? def From(x): return x In production yes, but I would also suggest that in debug mode it should return a wrapper object, that is checked in your trampoline. And in general, it's a great advic

[python-tulip] Re: change yield x to yield From(x) in Trollius?

2014-01-23 Thread Victor Stinner
What would be From(x)? A dummy function just returning x? def From(x): return x Victor 2014/1/23 Guido van Rossum : > I propose to change the yield convention in Trollius to "yield > From(x)". This is similar to "raise Return(x)". The reason is that > this way it will be easier to convert Trolli

Re: [python-tulip] Mac OS X, kqueue and PTY

2014-01-23 Thread Jonathan Slenders
Fine for me too to skip older OS X versions. Could you also take a look at the following patch? This is the fix for pty writes, while the other was for pty reads only. Probably, you should do the same here with: support.requires_mac_ver(10, 9) https://codereview.appspot.com/54130043 Jonathan

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Phil Schaf
Am Donnerstag, 23. Januar 2014 18:16:05 UTC+1 schrieb Guido van Rossum: I would have preferred a solution without multiple inheritance but in > this case it seems pretty benign, since SubprocessProtocol is just an > interface class, while StreamReaderProtocol is an implementation > class. A w

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Guido van Rossum
Well, that's water under the bridge. I'm sure the current design is perfectly usable. On Thu, Jan 23, 2014 at 9:46 AM, Gustavo Carneiro wrote: > On 23 January 2014 17:12, Guido van Rossum wrote: >> >> loop.subprocess_xxx() will give give your protocol a callback when the >> process exits (which

Re: [python-tulip] using coroutines within protocols?

2014-01-23 Thread Guido van Rossum
Oh, I apologize for not following the links in the original question. Indeed in this case StreamReader is the solution. I was answering a different question that gets asked a lot too (though perhaps the underlying use case is the same -- apparently StreamReader is not advertised well enough): "Why

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Gustavo Carneiro
On 23 January 2014 17:12, Guido van Rossum wrote: > loop.subprocess_xxx() will give give your protocol a callback when the > process exits (which may be earlier or later than when the pipes are > closed). Sure. But my feeling is that waiting for the pipe to be closed is good enough for most ap

Re: [python-tulip] using coroutines within protocols?

2014-01-23 Thread Guido van Rossum
Thanks for the docs update; I didn't realize these were little-known. Streams shouldn't have callbacks, the existing stream convention for detecting EOF (empty read) should be fine. However I agree that readexactly() should raise an exception. A subclass of EOFError is fine. On Thu, Jan 23, 2014

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Guido van Rossum
I would have preferred a solution without multiple inheritance but in this case it seems pretty benign, since SubprocessProtocol is just an interface class, while StreamReaderProtocol is an implementation class. A way to avoid the multiple inheritance would be to instantiate a StreamReaderProtocol

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Guido van Rossum
loop.subprocess_xxx() will give give your protocol a callback when the process exits (which may be earlier or later than when the pipes are closed). It also manages connecting the pipes for you. But you don't *have* to use it. On Thu, Jan 23, 2014 at 8:40 AM, Gustavo Carneiro wrote: > It's funny

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Gustavo Carneiro
It's funny that I've been using subprocesses without even being aware of loop.subprocess_exec(). I just followed the child_process.py example in Tulip, and it works fine. Why should we use loop.subprocess_xxx instead of plain old subprocess.Popen followed by connecting the pipes to asyncio stream

Re: [python-tulip] Mac OS X, kqueue and PTY

2014-01-23 Thread Victor Stinner
2014/1/23 Guido van Rossum : > No, they would just always call > set_event_loop(SelectorEventLoop(selector)). You are proposing to make > that the default for everyone which I think would be a really bad > step, since few people need PTYs. Ok, I agree. Kqueue is more efficient. Skipping the test i

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Phil Schaf
Am Donnerstag, 23. Januar 2014 16:48:46 UTC+1 schrieb Guido van Rossum: Read the source code of asyncio/streams.py. There are helper classes > that should let you do it. Please post the solution here. > -- > --Guido van Rossum (python.org/~guido) > i’m deep inside that source for some hours

Re: [python-tulip] Mac OS X, kqueue and PTY

2014-01-23 Thread Guido van Rossum
No, they would just always call set_event_loop(SelectorEventLoop(selector)). You are proposing to make that the default for everyone which I think would be a really bad step, since few people need PTYs. Let's just skip the PTY unit tests on OS X before Mavericks. On Thu, Jan 23, 2014 at 6:30 AM,

Re: [python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Guido van Rossum
Read the source code of asyncio/streams.py. There are helper classes that should let you do it. Please post the solution here. On Thu, Jan 23, 2014 at 7:04 AM, Phil Schaf wrote: > hi, i opened a subprocess using > > transport, proto = yield from loop.subprocess_exec(SubprocessProtocol, > *mycmd)

Re: [python-tulip] StreamReader.readexactly(n) may returns less than n bytes

2014-01-23 Thread Guido van Rossum
Agreed. On Thu, Jan 23, 2014 at 3:36 AM, Gustavo Carneiro wrote: > > > > On 23 January 2014 11:30, Victor Stinner wrote: >> >> Hi, >> >> I found the StreamReader.readexactly(n) function which has a strange >> behaviour. It reads exactly n bytes... but it returns less if the end >> of stream is r

Re: [python-tulip] Tulip unit tests: use asyncio module, not submodules

2014-01-23 Thread Guido van Rossum
This is a good point. It made more sense in the early days when I wanted to avoid importing modules that you don't need. But __init__.py imports them all anyways. So go ahead and change this in the tests. (However, inside the asyncio package, please don't use "import asyncio" -- recursive imports

[python-tulip] change yield x to yield From(x) in Trollius?

2014-01-23 Thread Guido van Rossum
I propose to change the yield convention in Trollius to "yield From(x)". This is similar to "raise Return(x)". The reason is that this way it will be easier to convert Trollius code to Tulip code automatically. Without this, you have to look at each yield in the code and decide whether it's in a co

[python-tulip] reading lines from subprocess in a coroutine

2014-01-23 Thread Phil Schaf
hi, i opened a subprocess using transport, proto = yield from loop.subprocess_exec(SubprocessProtocol, *mycmd) now, how do i write something to stdin asynchronously, and then react to lines being written? basically: stdin = transport.get_pipe_transport(0) stdin.write(msg) # this is backgroun

Re: [python-tulip] Mac OS X, kqueue and PTY

2014-01-23 Thread Victor Stinner
2014/1/14 Jonathan Slenders : > Jonathans-MacBook-Pro:~ jonathan$ python /tmp/test_pty.py > kqueue seems to support PTY > Jonathans-MacBook-Pro:~ jonathan$ sw_vers -productVersion > 10.9 So kqueue was improved in OS X Maverick (10.9). Guido wrote: > If someone really wants to use PTYs on affected

Re: [python-tulip] using coroutines within protocols?

2014-01-23 Thread Victor Stinner
2014/1/23 Glyph : > On Jan 22, 2014, at 7:33 AM, Guido van Rossum wrote: > >> The solution is simple: write that logic as a separate method marked >> with @coroutine, and fire it off in data_received() using async() (== >> Task(), in this case). E.g. > > Wouldn't that process each subsequent TCP s

Re: [python-tulip] StreamReader.readexactly(n) may returns less than n bytes

2014-01-23 Thread Gustavo Carneiro
On 23 January 2014 11:30, Victor Stinner wrote: > Hi, > > I found the StreamReader.readexactly(n) function which has a strange > behaviour. It reads exactly n bytes... but it returns less if the end > of stream is reached. > > Why not raising an exception? When I use such function, I expected to

[python-tulip] StreamReader.readexactly(n) may returns less than n bytes

2014-01-23 Thread Victor Stinner
Hi, I found the StreamReader.readexactly(n) function which has a strange behaviour. It reads exactly n bytes... but it returns less if the end of stream is reached. Why not raising an exception? When I use such function, I expected to get... exactly... n bytes, not "almost" n bytes :-) It would

[python-tulip] Tulip unit tests: use asyncio module, not submodules

2014-01-23 Thread Victor Stinner
Hi, I don't understand why Tulip unit tests import submodules instead of using the public API. Example: "from asyncio import tasks" and then use "tasks.Task()". What do you think of only importing asyncio and use the asyncio module instead? It would ensure that all required symbols are exported.

Re: [python-tulip] wait_for() and cancelling the waited-for task

2014-01-23 Thread Victor Stinner
Hi, 2014/1/20 Gustavo Carneiro : > I spent a few hours chasing a nasty bug in either tulip or my code. I got > this traceback: > > Traceback (most recent call last): > ... > File "tests/450-api-place-hide-exchange.py", line 35, in wait_for_line > line = yield from api.read_async_line() >

Re: [python-tulip] using coroutines within protocols?

2014-01-23 Thread Glyph
On Jan 22, 2014, at 7:33 AM, Guido van Rossum wrote: > The solution is simple: write that logic as a separate method marked > with @coroutine, and fire it off in data_received() using async() (== > Task(), in this case). E.g. Wouldn't that process each subsequent TCP segment in a parallel corou