[issue17986] Alternative async subprocesses (pep 3145)

[STINNER Victor]

STINNER Victor  added the comment:

Well, this issue is basically inactive for 5 years. It doesn't sound like a 
common feature request. I close it.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[Łukasz Langa]

Łukasz Langa  added the comment:

Should this still be open given that PEP 3145 is withdrawn and asyncio 
subprocesses have been used in production for five Python releases now?

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[STINNER Victor]

STINNER Victor added the comment:

As Tulip and subprocdev, starting such project outside the Python stdlib may 
help to get feedback, find and fix bugs faster. What do you think?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[Martin Panter]

Changes by Martin Panter vadmium...@gmail.com:


--
nosy: +vadmium

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[STINNER Victor]

STINNER Victor added the comment:

Subprocess support of asyncio has nice features and is efficient:

- async read from stdout and stderr
- async write into stdin
- async wait for the process exit
- async communicate()
- timeout on any async operation
- support running multiple child processes in parallel
- epoll, kqueue, devpoll or IOCP selector
- On POSIX, fast child watcher waiting for SIGCHLD signal

If possible, I would prefer to not write a second async subprocess.

It's possible to write a blocking API on top of asyncio using 
loop.run_until_complete().

sync_proc_asyncio.py: incomplete proof-of-concept (no working select() method).

--
Added file: http://bugs.python.org/file38123/sync_proc_asyncio.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[Charles-François Natali]

Charles-François Natali added the comment:

I'm not familiar with windows, but if I understand the patch
correctly, you can only select from a single subprocess at a time,
which is IMO an important limitation.

Also, the fact that close() can fail with BlockingIOError is really a
pain, and makes writing portable code complicated, but I know you
can't do anything about it ;-)

In short, that would be nice, but Windows seems to make it so
limited/complicated that I'm not sure that would be really useful, but
once again I don't use Windows, so I can't make a call.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[STINNER Victor]

Changes by STINNER Victor victor.stin...@gmail.com:


--
nosy: +haypo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[Richard Oudkerk]

New submission from Richard Oudkerk:

In the attached file is an experimental implementation of an AsyncPopen
class.  It should work for Python 3.3, 3.4 on Unix and Windows.  Unlike

http://code.google.com/p/subprocdev

(see #1191964) this does not depend on using time.sleep() and polling.
It also uses the stdlib's _winapi instead of ctypes.

It lets one do Unix-style multiplexing of stdin, stdout, stderr on
Windows (by using overlapped IO).

Differences from normal Popen:

* File objects created for stdin/stdout/stderr using ...=PIPE are
  non-blocking.

* There are no options for buffering or for universal line endings
  (or unicode).

* There is an extra method

  select(file_list, timeout=None) - ready_file_list

  which can be used to wait for stdin, stdout, stderr to be ready.
  file_list should be a sublist of [self.stdin, self.stdout, self.stderr]
  with no None elements.  Note that there is no separation between
  readers and writers the way there is for the normal select()
  function.  On Unix this is implemented using select() or poll().

* On Windows close() can fail with BlockingIOError.  To prevent this
  one must use select() to wait for self.stdin to be ready.

As an example, communicate() can be reimplemented using select() as follows:

def communicate(p, input):
buf = memoryview(input)
collected = collections.defaultdict(list)
registered = [f for f in (p.stdin, p.stdout, p.stderr)
  if f is not None]

while registered:
for f in p.select(registered):
if f is p.stdin:
if not buf:
f.close()
registered.remove(f)
else:
n = f.write(buf)
if n is not None:
buf = buf[n:]
elif f in (p.stdout, p.stderr):
s = f.read(8192)
if s == b'':
f.close()
registered.remove(f)
elif s is not None:
collected[f].append(s)
else:
raise RuntimeError('should not get here')

return (b''.join(collected[p.stdout]) if p.stdout else None,
b''.join(collected[p.stderr]) if p.stderr else None)

--
files: asyncsubprocess.py
messages: 189304
nosy: sbt
priority: normal
severity: normal
status: open
title: Alternative async subprocesses (pep 3145)
versions: Python 3.4
Added file: http://bugs.python.org/file30269/asyncsubprocess.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17986] Alternative async subprocesses (pep 3145)

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +neologix

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17986
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com