On 11/08/2010 11:32 AM, Von Fugal wrote:
> I still love ruby plenty! :) However, I would never recommend it as a
> scripting language. I've tried that numerous times myself when I was
> still in ruby honeymoon. It's more trouble than it's worth. Use
> something scritpy like perl or bash. I'm not sure I'd even recommend
> python for scripting. No, interpreted != scripting language.
I find that Python works pretty well for scripting, but it was missing
an easy way to run a command and then get the return code, the standard
out and standard-err from the process. I can run a command a get stdout
with the os.system() call, which is, IIRC, like the backticks in perl.
But that's not enough sometimes. So I wrote some wrapper code that
makes it easy to spawn a process, feed it input, and get the
above-mentioned output. I've attached it here if anyone wants it. If
you combine this simple module with generators you won't miss piping
processes together; it's not needed 90% of the time. If you do need to
do some piping, you can always run your processes in a sub-shell (one of
the options in my function).
Note that this will give you all the output in one chunk so if you want
to capture a lot of output, use the subprocess module directly and
operate on the pipe. And of course with pipes you can set up generators
which is pretty awesome and almost as fast as bash piping.
Anyway, it's not as quick and dirty as bash by any means, but it's
sufficient to pretty cleanly do what I need in a shell script most of
the time.
#!/usr/bin/python
import subprocess
import sys
import fcntl
import os
import select
import StringIO
"""
This module contains a single function run() that spawns a
process with arguments, and returns it's errcode, stdout,
and stderr.
It currently has a number of deficiencies, including the lack
of a way to kill the child.
"""
def _make_non_blocking(fd):
"""
Takes a file descriptor and sets it to non-blocking IO mode,
allowing us to use the posix select() on it. Used by this
module to allow us to read from a process as it writes to
stderr and stdout
@type fd: number
@param fd: file descriptor number to set to nonblocking
"""
fl = fcntl.fcntl(fd, fcntl.F_GETFL)
fcntl.fcntl(fd, fcntl.F_SETFL, fl | os.O_NONBLOCK)
def run(cmd, **kwargs):
"""
Runs a command using the subprocess.Popen class, and
returns the err, stdout, and stderr of the process.
Optionally it can print to stdout the process's stderr
and stdout, and can also call callbacks when stdout and
stderr is available from the process. The following
kwargs are useful:
- print_stdout - if set to True will print process stdout
to stdout as data arrives
- print_stderr - if set to True will print process stderr
to stdout as data arrives
- stdin - Text to feed to process's stdin
- stdout_callback - function to call with stdout data
- stderr_callback - function to call with stderr data
@type cmd: text or list
@param cmd: the command to run, use a list if args are passed
@type kwargs: keyword args
@param kwargs: optional keyword arguments (see above)
@rtype: tuple
@return: Tuple of error code, stdout text, stderr text
"""
# Set flags depending on kwargs
if "print_stdout" in kwargs:
print_stdout=True
else:
print_stdout=False
if "print_stderr" in kwargs:
print_stderr=True
else:
print_stderr=False
if "stdout_callback" in kwargs:
stdout_callback=kwargs["stdout_callback"]
print "setting stdout callback"
else:
stdout_callback=None
if "stdout_callback" in kwargs:
stderr_callback=kwargs["stderr_callback"]
print "setting stderr callback"
else:
stderr_callback=None
if "shell" in kwargs:
shell=kwargs["shell"]
else:
shell=False
# create process object, set up pipes
child = subprocess.Popen(cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
bufsize=0,
shell=shell
)
# give the process any stdin that we might have, then
# close stdin to prevent deadlocks
if "stdin" in kwargs:
child.stdin.write(kwargs["stdin"])
child.stdin.close()
# get output pipes, make them non-blocking
fo=child.stdout
fe=child.stderr
_make_non_blocking(fo.fileno())
eof=False
stdout=StringIO.StringIO()
stderr=StringIO.StringIO()
# Loop, checking for data on process's stdout and stderr
# until we've reached an eof condition
to_check = [fo,fe] * (not eof)
while True:
# check for data
ready = select.select(to_check,[],[],.05)
# process any stdout
if fo in ready[0]:
data=fo.read()
if data == '':
eof =True
else:
if print_stdout:
sys.stdout.write(data)
sys.stdout.flush()
#print "got data"
#print data
if stdout_callback:
stdout_callback(data)
stdout.write(data)
stdout.flush()
# process any stderr
if fe in ready[0]:
data=fe.read()
if not data == '':
if print_stderr:
sys.stdout.write(data)
sys.stdout.flush()
#print data,
if stderr_callback:
stderr_callback(data)
stderr.write(data)
stderr.flush()
if eof:
break
# clean up child and get its errcode
err=child.wait()
# Get the data in string form
stdout=stdout.getvalue()
stderr=stderr.getvalue()
return (err, stdout, stderr)
if __name__=="__main__":
"""
Random test cases. I think you pass a command as the first arg
and the second arg is used as stdin.
"""
lines=sys.stdin.readlines()
if lines:
err, stdout, stderr = run(sys.argv[1:],stdin=''.join(lines),
print_stdout=True)
else:
err, stdout, stderr = run(sys.argv[1:], print_stdout=True)
print "Process ended with %d" % err
print "Standard out:"
print stdout
print stdout.split('\n')
print "Standard err:"
print stderr
/*
PLUG: http://plug.org, #utah on irc.freenode.net
Unsubscribe: http://plug.org/mailman/options/plug
Don't fear the penguin.
*/
