Re: Need feedback on subprocess-using function
In pan.2009.10.07.03.27.32.844...@nowhere.com Nobody nob...@nowhere.com writes: You could always lift the code from Popen._communicate(), which uses threads for Windows and select() for POSIX. Thanks. A lot of useful advice in your replies. G. -- http://mail.python.org/mailman/listinfo/python-list
Re: Need feedback on subprocess-using function
On Oct 4, 9:46 pm, Nobody nob...@nowhere.com wrote: On Sat, 03 Oct 2009 13:21:00 +, gb345 wrote: I'm relatively new to Python, and I'm trying to get the hang of using Python's subprocess module. As an exercise, I wrote the Tac class below, which can prints output to a file in reverse order, by piping it through the Unix tac utility. (The idea is to delegate the problem of managing the memory for an arbitrarily large task to tac.) self.pipe = subprocess.Popen(['tac'], stdout=out, stdin=subprocess.PIPE, stderr=subprocess.PIPE) This works OK, as far as I can tell, but I'm not sure that I've dotted all the i's and crossed all the t's... E.g., I had to add the line p.stdin.close() to the close method when I when I ran into sporadic deadlock at the p.stderr.read() statement. Are there other similar problems lurking in this code? Yep. If the process writes more than a buffer-full of data to stderr, it will deadlock. tac will block trying to write to stderr, and won't be reading its stdin, so your program will block trying to write to tac. This is why the POSIX popen() call only lets you attach a pipe to stdin or stdout but not both. If you want a double-ended slave process, you need to use polling or non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of any single solution which works on all platforms. The easiest way around this problem is to redirect stderr to a temporary file and read in the file's contents in the close() method. There is also Popen.communicate(): http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate -- http://mail.python.org/mailman/listinfo/python-list
Re: Need feedback on subprocess-using function
On Sat, 03 Oct 2009 13:21:00 +, gb345 wrote: I'm relatively new to Python, and I'm trying to get the hang of using Python's subprocess module. As an exercise, I wrote the Tac class below, which can prints output to a file in reverse order, by piping it through the Unix tac utility. (The idea is to delegate the problem of managing the memory for an arbitrarily large task to tac.) self.pipe = subprocess.Popen(['tac'], stdout=out, stdin=subprocess.PIPE, stderr=subprocess.PIPE) This works OK, as far as I can tell, but I'm not sure that I've dotted all the i's and crossed all the t's... E.g., I had to add the line p.stdin.close() to the close method when I when I ran into sporadic deadlock at the p.stderr.read() statement. Are there other similar problems lurking in this code? Yep. If the process writes more than a buffer-full of data to stderr, it will deadlock. tac will block trying to write to stderr, and won't be reading its stdin, so your program will block trying to write to tac. This is why the POSIX popen() call only lets you attach a pipe to stdin or stdout but not both. If you want a double-ended slave process, you need to use polling or non-blocking I/O or asynchronous I/O or multiple threads. I'm not aware of any single solution which works on all platforms. The easiest way around this problem is to redirect stderr to a temporary file and read in the file's contents in the close() method. -- http://mail.python.org/mailman/listinfo/python-list
Need feedback on subprocess-using function
I'm relatively new to Python, and I'm trying to get the hang of
using Python's subprocess module. As an exercise, I wrote the Tac
class below, which can prints output to a file in reverse order,
by piping it through the Unix tac utility. (The idea is to delegate
the problem of managing the memory for an arbitrarily large task
to tac.)
class Tac(object):
def __init__(self, path):
out = open(path, 'w')
self.pipe = subprocess.Popen(['tac'], stdout=out,
stdin=subprocess.PIPE,
stderr=subprocess.PIPE)
def prn(self, string):
try:
self.pipe.stdin.write('%s\n' % string)
except:
self.close()
raise
def close(self):
p = self.pipe
p.stdin.close()
err = p.stderr.read()
if err:
raise OSError(err)
This works OK, as far as I can tell, but I'm not sure that I've
dotted all the i's and crossed all the t's... E.g., I had to add
the line p.stdin.close() to the close method when I when I ran
into sporadic deadlock at the p.stderr.read() statement. Are there
other similar problems lurking in this code? Also, there's no
robust mechanism for invoking this close method in case of an
exception (unless the exception happens during the execution of
prn).
Any comments and suggestions would be greatly appreciated.
G
--
http://mail.python.org/mailman/listinfo/python-list
