Re: [python-tulip] Asynchronous console and interfaces

[Vincent Michel]

> At the end, you got the idea. 
> It is readable, portable, generic, etc, etc.. 
> "This" should be underlined in asyncio documentation.
> Not the protocols, callbacks, futures, transports...

I think they're all very different objects:
- callbacks, futures and coroutines are the core of asyncio; you cannot 
write asyncio code without directly using one of those three.
- protocols and transports are low-level abstraction for communication 
channel; you usually don't have to use them directly.
- streams are high-level abstraction; you can do without them but they 
really make your life easier.

Also, I updated the project to make it easier to add a backdoor to a server 
without touching its code:
$ apython --serve localhost: -m some.asyncio.server --some options

A python console is then accessible through:
$ nc localhost  

Re: [python-tulip] Asynchronous console and interfaces

[Gustavo Carneiro]

On 3 December 2015 at 14:25, Imran Geriskovan 
wrote:

> On 12/3/15, Vincent Michel  wrote:
> > there might be other, more powerful tools around to serve this purpose.
> So
> > I'm also interested in comments from people with a wider view of the
> > asyncio ecosystem.
>
> Regarding the "stream style", I can share some of my
> experience:
>
> For code below, other than some limited portion
> and awaits, can you see any line specific to asyncio? No?
>
> Well that's the point. With some little search replace,
> and tweaks it can easily be transformed into
> equivalent multithreaded version. Even to a C++ version.
>

I don't understand the point you're trying to make.  You are saying this
can be easily transformed to use threads.  Are you also saying that it
*should* be using threads?  If that's what you mean, then you are totally
missing the point of asyncio.

I think current form of python asyncio documentation
> leads to "unnecessarily asyncio" nized projects.
> With lots of "protocols, callbacks, futures, transports"..
>

It's not unnecessary.  There are several valid reasons for implementing a
protocol in asyncio:

  1. the existing implementations are not compatible with the asyncio event
loop, and so would need to be executed in a thread pool.  I think it is
clear why this is very suboptimal;

  2. the existing implementations can be made compatible with asyncio (e.g.
Tornado and Twisted protocols can possibly be adapted to asyncio), but then
the API they will present to the programmer will be very much
callback-oriented, which is not as nice as coroutines;

  3. implementing a protocol completely using asyncio coroutines and
streams makes the implementation itself more readable, so it can be a goal
in itself.


>
> Regards,
> Imran
>
> from lib.http import Httpd
> from lib.tcp import Tcpd
> from handler.man import man
>

Where does this code even come from?

I'm confused...


>
> handlers = [('/man/', man)]
> servers = [('127.0.0.1', 90, Httpd(handlers))]
> Tcpd().run(servers)
>
>
> class Tcpd:
> def __init__(self):
> self.loop = l = get_event_loop()
> l.add_signal_handler(SIGINT, l.stop)
> l.add_signal_handler(SIGTERM, l.stop)
>
> def run(self, servers):
> for ip, port, s in servers:
> self.loop.create_task(start_server(s.handle, ip, port,
> family=AF_INET, limit=4096))
> try:
> self.loop.run_forever()
> finally:
> self.loop.close()
>
>
> class TcpStream:
>
> def __init__(self, r, w):
> self.r = r
> self.w = w
>
> # Read Functions
> async def read(self, l, b):
> e = b.extend
> while l:
> try:
> rd = await wait_for(self.r.read(min(l, bsize)), 20)
> except:
> rd = None
> if not rd:
> self.w.close()
> return 1
> e(rd)
> l -= len(rd)
>
> async def readline(self, tout=20):
> try:
> rd = await wait_for(self.r.readline(), tout)
> except:
> rd = None
> if not rd:
> self.w.close()
> return rd
>
>
> class Httpd:
>
> def __init__(self, handlers):
> self.handlers = handlers
>
> async def handle(self, r, w):
> s = TcpStream(r, w)
> hs = self.handlers
> while 1:
> req = HttpReq(s)
> try:
> if await req.read():
> return
>
> path = req.url.path
> for mp, h in hs:
> l = len(mp)
> if path[0:l] == mp:
> req.hpath = path[l:]
> await h(req)
> break
> else:
> async with req.send():
> pass
> except:
> s.w.close()
> print_exc()
> return
>



-- 
Gustavo J. A. M. Carneiro
Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert

[python-tulip] Asynchronous console and interfaces

[Vincent Michel]

Hi all,

Last month I've been playing with asyncio and I ended up writing an 
asynchronous console. It's a python console just like the one in the 'code' 
module, but running in an asyncio event loop instead. That means it lets 
you interact with the program environment while it's running other 
asynchronous tasks. It also supports the `await` syntax so it is possible 
to wait for a task to complete:

>>> await asyncio.sleep(1, result=3)
# Wait 1 sec
3

I then realized It could easily be adapted to support asyncio streams in 
order to serve the console over the network. But since sharing a direct 
access to your machine is probably not the best idea, I thought it'd be 
nice to replace the arbitrary code execution with a limited set of 
commands. Those commands are described using 'argparse' parsers, leading to 
a fairly simple way to create and share a command line interface over the 
network.

All this work is available in a github repo. The README file covers most of 
the features and includes step-by-step examples:
https://github.com/vxgmichel/python3-apython

I could definitely use some feedback on this project. Some parts of the 
code are pretty hackish (especially for the asynchronous exec part), and 
I'm not sure I handle the standard streams correctly. Although it was a fun 
project to work on, it is still unclear what it could be useful for, and 
what the use cases would be. I think debugging might be one thing, though 
there might be other, more powerful tools around to serve this purpose. So 
I'm also interested in comments from people with a wider view of the 
asyncio ecosystem.

Thanks,

Vincent

Re: [python-tulip] Asynchronous console and interfaces

[Imran Geriskovan]

>> Regarding the "stream style", I can share some of my
>> experience:
>>
>> For code below, other than some limited portion
>> and awaits, can you see any line specific to asyncio? No?
>>
>> Well that's the point. With some little search replace,
>> and tweaks it can easily be transformed into
>> equivalent multithreaded version. Even to a C++ version.

>   3. implementing a protocol completely using asyncio coroutines and
> streams makes the implementation itself more readable, so it can be a goal
> in itself.

At the end, you got the idea.
It is readable, portable, generic, etc, etc..
"This" should be underlined in asyncio documentation.
Not the protocols, callbacks, futures, transports...