which async framework? - a summary

2014-03-22 Thread Chris Withers
On 14/03/2014 00:36, Tim Chase wrote: On 2014-03-14 00:25, Chris Withers wrote: I've been pleasantly surprised by the succinct, well reasoned and respectful replies from each of the communities! As one who doesn't lurk on the other lists, is there a nice executive summary of their responses?

Re: which async framework?

2014-03-13 Thread Chris Withers
On 11/03/2014 19:41, Terry Reedy wrote: I suspect I'll just end up cross-posting to the various mailing lists, Bad idea. Post separately if you must. which I hope won't cause too much offence or kick off any flame wars. It would do both. Ye of little faith :-P I've been pleasantly

Re: which async framework?

2014-03-13 Thread Tim Chase
On 2014-03-14 00:25, Chris Withers wrote: I've been pleasantly surprised by the succinct, well reasoned and respectful replies from each of the communities! As one who doesn't lurk on the other lists, is there a nice executive summary of their responses? -tkc --

Re: which async framework?

2014-03-12 Thread Ian Kelly
On Tue, Mar 11, 2014 at 5:38 PM, Chris Angelico ros...@gmail.com wrote: On Wed, Mar 12, 2014 at 10:18 AM, Marko Rauhamaa ma...@pacujo.net wrote: Chris Angelico ros...@gmail.com: What corner cases are there with threads that you don't have with anything else? There are numerous. Here's one

Re: which async framework?

2014-03-12 Thread Neal Becker
Grant Edwards wrote: On 2014-03-11, Antoine Pitrou solip...@pitrou.net wrote: Sturla Molden sturla.molden at gmail.com writes: Chris Withers chris at simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from:

Re: which async framework?

2014-03-11 Thread Chris Withers
On 10/03/2014 21:57, Terry Reedy wrote: I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most likely spinning off a thread for each one.

Re: which async framework?

2014-03-11 Thread Sturla Molden
Chris Withers ch...@simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted Looking at Tornado's examples on the web I find this: tornado.ioloop.IOLoop.instance().start()

Re: which async framework?

2014-03-11 Thread Sturla Molden
Chris Withers ch...@simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to learn a framework? You will

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Sturla Molden sturla.mol...@gmail.com: Looking at Tornado's examples on the web I find this: [...] (1) This was written by some Java guys. I have written several Python async frameworks starting from select.epoll(). It's only a handful of lines of code (plus an AVL tree implementation for

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Sturla Molden sturla.mol...@gmail.com: I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to learn a framework? You will find epoll and kqueue/kevent in the select module and iocp in pywin32. You beat me to it. However, I'm hoping asyncio will steer the Python faithful away

Re: which async framework?

2014-03-11 Thread Sturla Molden
Marko Rauhamaa ma...@pacujo.net wrote: Now, I've taken a brief look at the new asyncio and it looks as if it has everything one would hope for (and then some). You'd still need to supply the protocol implementations yourself. Tulip (the new async module) is nice. But I am a bit confused as to

Re: which async framework?

2014-03-11 Thread Ian Kelly
On Tue, Mar 11, 2014 at 4:58 AM, Marko Rauhamaa ma...@pacujo.net wrote: Sturla Molden sturla.mol...@gmail.com: I'd go for using iocp, epoll and kqueue/kevent directly. Why bother to learn a framework? You will find epoll and kqueue/kevent in the select module and iocp in pywin32. You beat

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Ian Kelly ian.g.ke...@gmail.com: eventlet has 115k downloads from PyPI over the last month. gevent has 143k. Twisted has 147k. Tornado has 173k. I'd say that a lot of Python users are already doing non-blocking network I/O, in one form or another. There aren't so many network developers in

Re: which async framework?

2014-03-11 Thread Oscar Benjamin
On 11 March 2014 11:54, Marko Rauhamaa ma...@pacujo.net wrote: Ian Kelly ian.g.ke...@gmail.com: eventlet has 115k downloads from PyPI over the last month. gevent has 143k. Twisted has 147k. Tornado has 173k. I'd say that a lot of Python users are already doing non-blocking network I/O, in

Re: which async framework?

2014-03-11 Thread Antoine Pitrou
Chris Withers chris at simplistix.co.uk writes: The protocols are all financial (do we really not have a pure-python FIX library?!) but none are likely to have existing python implementations. If you are mostly writing protocol implementations (aka parsers and serializers), then you should

Re: which async framework?

2014-03-11 Thread Antoine Pitrou
Sturla Molden sturla.molden at gmail.com writes: Chris Withers chris at simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted I'd go for using iocp, epoll and

Re: which async framework?

2014-03-11 Thread Giampaolo Rodola'
On Tue, Mar 11, 2014 at 11:17 AM, Sturla Molden sturla.mol...@gmail.comwrote: Chris Withers ch...@simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted Looking at

Re: which async framework?

2014-03-11 Thread Grant Edwards
On 2014-03-11, Antoine Pitrou solip...@pitrou.net wrote: Sturla Molden sturla.molden at gmail.com writes: Chris Withers chris at simplistix.co.uk wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip -

Re: which async framework?

2014-03-11 Thread jkn
Hi Grant On Tuesday, 11 March 2014 16:52:18 UTC, Grant Edwards wrote: [...] And don't bother with device drivers for the network adapters either. Just map their PCI regions in to user-space and twiddle the reigisters directly! ;) [I do that when testing PCI boards with C code, and one

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 3:01 AM, Antoine Pitrou solip...@pitrou.net wrote: Yes, why use a library when you can rewrite it all yourself? Actually, you should probably issue system calls to the kernel directly, the libc is overrated (as is portability, I suppose). It's a trade-off, of course. I

Re: which async framework?

2014-03-11 Thread Sturla Molden
Antoine Pitrou solip...@pitrou.net wrote: Yes, why use a library when you can rewrite it all yourself? This assumes something equivalent to the library will have to be written. But if it can be replaced with something very minimalistic it is just bloat. I would also like to respond that the

Re: which async framework?

2014-03-11 Thread Antoine Pitrou
Sturla Molden sturla.molden at gmail.com writes: Antoine Pitrou solipsis at pitrou.net wrote: Yes, why use a library when you can rewrite it all yourself? This assumes something equivalent to the library will have to be written. But if it can be replaced with something very minimalistic

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 5:43 AM, Antoine Pitrou solip...@pitrou.net wrote: This is the usual assumption that high-level libraries are made of useless cruft piled up by careless programmers. But there are actual reasons why these frameworks have a significant amount of code, and people who

Re: which async framework?

2014-03-11 Thread Terry Reedy
On 3/11/2014 3:53 AM, Chris Withers wrote: On 10/03/2014 21:57, Terry Reedy wrote: I'd like to be able to serve the rest of the web api using a pyramid wsgi app if possible, and I'd like to be able to write the things that process requests in and validation out in a synchronous fashion, most

Re: which async framework?

2014-03-11 Thread Sturla Molden
Antoine Pitrou solip...@pitrou.net wrote: This is the usual assumption that high-level libraries are made of useless cruft piled up by careless programmers. It often is the case, particularly in network programming. But in this case the programmer is Guido, so it doesn't apply. :) What

Re: which async framework?

2014-03-11 Thread Gregory Ewing
Sturla Molden wrote: Another thing is that co-routines and yield from statements just makes it hard to follow the logic of the program. I still have to convince myself that a library for transforming epoll function calls into co-routines is actually useful. It's not epoll function calls that

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Gregory Ewing greg.ew...@canterbury.ac.nz: It's not epoll function calls that the coroutine style is intended to replace, it's complex systems of chained callbacks. They're supposed to make that kind of logic *easier* to follow. If you haven't had that experience, it may be because you've

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Antoine Pitrou solip...@pitrou.net: This is the usual assumption that high-level libraries are made of useless cruft piled up by careless programmers. But there are actual reasons why these frameworks have a significant amount of code, and people who decide to ignore those reasons are simply

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 9:28 AM, Marko Rauhamaa ma...@pacujo.net wrote: If you can't write your own event loop, you probably can't be trusted with any multithreaded code, which has much more baffling corner cases. I'm not sure about that. Threads are generally easier to handle, because each one

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Chris Angelico ros...@gmail.com: Yep. Now how is that not a problem when you use some other model, like an event loop? The standard methods of avoiding deadlocks (like acquiring locks in strict order) work exactly the same for all models, and are just as necessary. I was simply saying that

Re: which async framework?

2014-03-11 Thread Chris Angelico
On Wed, Mar 12, 2014 at 10:18 AM, Marko Rauhamaa ma...@pacujo.net wrote: Chris Angelico ros...@gmail.com: What corner cases are there with threads that you don't have with anything else? There are numerous. Here's one example: deadlocks due to two threads taking locks in a different order.

Re: which async framework?

2014-03-11 Thread Marko Rauhamaa
Chris Angelico ros...@gmail.com: What corner cases are there with threads that you don't have with anything else? There are numerous. Here's one example: deadlocks due to two threads taking locks in a different order. The problem crops up naturally with two intercommunicating classes. It can

which async framework?

2014-03-10 Thread Chris Withers
Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few network protocols, both classic tcp and

Re: which async framework?

2014-03-10 Thread Terry Reedy
On 3/10/2014 4:38 PM, Chris Withers wrote: Hi All, I see python now has a plethora of async frameworks and I need to try and pick one to use from: - asyncio/tulip - tornado - twisted From my side, I'm looking to experimentally build a network testing tool that will need to speak a fair few