On 27May2014 21:01, Adam Gold <a...@gmx.com> wrote:
I'm trying to run the following unix command from within Python as
opposed to calling an external Bash script (the reason being I'm doing
it multiple times within a for loop which is running through a list):
"dd if=/home/adam/1 bs=4k conv=noerror,notrunc,sync | pbzip2 > 1.img.bz2"
First off, one expedient way to do this is to generate a shell script and pipe
into "sh" (or "sh -uex", my preferred error sensitive invocation).
p1 = subprocess.Popen(["sh", "-uex"], stdin=PIPE)
for num in range(1,11):
print("dd if=/home/adam/%d bs=4k conv=noerror,notrunc,sync | pbzip2 >
%d.img.bz2",
% (num, num), file=p1.stdin)
p1.stdin.close()
p1.wait()
Any quoting issues aside, this is surprisingly useful. Let the shell do what it
is good it.
And NOTHING you've said here requires using bash. Use "sh" and say "sh", it is
very portable and bash is rarely needed for most stuff.
However, I gather beyond expediency, you want to know how to assemble pipelines
using subprocess anyway. So...
The first thing I do is break it into two assignments (I know this isn't
strictly necessary but it makes the code easier to deal with):
ddIf = shlex.split("dd if=/home/adam/1 bs=4k conv=noerror,notrunc,sync")
compress = shlex.split("pbzip2 > /home/adam/1.img.bz2")
This is often worth doing regardless. Longer lines are harder to read.
I have looked at the docs here (and the equivalent for Python 3)
https://docs.python.org/2/library/subprocess.html. I can get a 'simple'
pipe like the following to work:
p1 = subprocess.Popen(["ps"], stdout=PIPE)
p2 = subprocess.Popen(["grep", "ssh"], stdin=p1.stdout, stdout=subprocess.PIPE)
p1.stdout.close()
output = p2.communicate()[0]
If you don't care about the stdout of p2 (and you don't, based on your
"dd|pbzip2" example above) and you have left p2's stdout alone so that it goes
to your normal stdout (eg the terminal) then you don't need to waste time with
.communicate. I almost never use it myself. As the doco says, prone to
deadlock. I prefer to just do the right thing explicitly myself, as needed.
I then try to adapt it to my example:
p1 = subprocess.Popen(ddIf, stdout=subprocess.PIPE)
p2 = subprocess.Popen(compress, stdin=p1.stdout, stdout=subprocess.PIPE)
p1.stdout.close()
output = p2.communicate()[0]
I get the following error:
pbzip2: *ERROR: File [>] NOT found! Skipping...
-------------------------------------------
pbzip2: *ERROR: Input file [/home/adam/1.img.bz2] already has a .bz2
extension! Skipping
I think that the '>' redirect needs to be dealt with using the
subprocess module as well but I can't quite put the pieces together.
I'd appreciate any guidance. Thanks.
It is as you expect. Consider what the shell does with:
pbzip2 > 1.img.bz2
It invokes the command "pbzip2" (no arguments) with its output attached to the
file "1.img.bz2".
So first up: stay away form "shlex". It does _not_ do what you need.
Shlex knows about shell string quoting. It does not know about redirections.
It is handy for parsing minilanguages on your own concoction where you want to
be able to quote strings with spaces. It is not a full on shell parser.
So it (may) serve you well for the dd invocation because there are no
redirections. But for your usage, so would the .split() method on a string, or
even better: don't you already know the arguments for your "dd"? Just fill them
out directly rather than backtracking from a string.
However, your recipe is very close. Change:
p2 = subprocess.Popen(compress, stdin=p1.stdout, stdout=subprocess.PIPE)
into:
p2 = subprocess.Popen(["pbzip2"], stdin=p1.stdout, stdout=open("1.img.bz2",
"w"))
Because p2 is writing to "1.img.bz2" you don't need to much about with
.communicate either. No output to collect, no input to supply.
See where that takes you.
Cheers,
Cameron Simpson <c...@zip.com.au>
_______________________________________________
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
