Re: Issue combining gzip and subprocess

2009-07-22 Thread Scott David Daniels 

Piet van Oostrum wrote:

...
f = gzip.open(filename, 'w')
proc = subprocess.Popen(['ls','-la'], stdout=subprocess.PIPE)
while True:
line = proc.stdout.readline()
if not line: break
f.write(line)
f.close()


Or even:
proc = subprocess.Popen(['ls','-la'], stdout=subprocess.PIPE)
with gzip.open(filename, 'w') as dest:
for line in iter(proc.stdout, ''):
f.write(line)

--Scott David Daniels
scott.dani...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list

Re: Issue combining gzip and subprocess

2009-07-22 Thread Piet van Oostrum 
 Scott David Daniels scott.dani...@acm.org (SDD) schreef:

SDD Piet van Oostrum wrote:
 ...
 f = gzip.open(filename, 'w')
 proc = subprocess.Popen(['ls','-la'], stdout=subprocess.PIPE)
 while True:
 line = proc.stdout.readline()
 if not line: break
 f.write(line)
 f.close()

SDD Or even:
SDD proc = subprocess.Popen(['ls','-la'], stdout=subprocess.PIPE)
SDD with gzip.open(filename, 'w') as dest:
SDD for line in iter(proc.stdout, ''):
SDD f.write(line)

If it would work.

1) with gzip... is not supported in Python  3.1
2) for line in iter(proc.stdout), i.e. no second argument.
3) dest == f should be the same identifier.

Lesson: if you post code either: 
- test it and copy verbatim from your test, or
- write a disclaimer 
-- 
Piet van Oostrum p...@cs.uu.nl
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Issue combining gzip and subprocess

2009-07-21 Thread Iwan Vosloo 
Hi there,

We tried to gzip the output of a shell command, but this results in a
strange error: the resulting file seems to be the concatenation of the
plaintext file with the zipped content.

For example:

f = gzip.open(filename, 'w')
subprocess.check_call(['ls','-la'], stdout=f)
f.close()

Using a normal file works as expected, but a GzipFile results in a file
containing what looks like the unzipped data, followed by the zipped
data.

I suspect this may have something to do with limitations of GzipFile
such as it not implementing truncate().

Does anyone have an explanation / can you suggest a nice solution for
doing what we are trying to do?

Regards
- Iwan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Issue combining gzip and subprocess

2009-07-21 Thread Piet van Oostrum 
 Iwan Vosloo i...@reahl.org (IV) wrote:

IV Hi there,
IV We tried to gzip the output of a shell command, but this results in a
IV strange error: the resulting file seems to be the concatenation of the
IV plaintext file with the zipped content.

IV For example:

IV f = gzip.open(filename, 'w')
IV subprocess.check_call(['ls','-la'], stdout=f)
IV f.close()

IV Using a normal file works as expected, but a GzipFile results in a file
IV containing what looks like the unzipped data, followed by the zipped
IV data.

IV I suspect this may have something to do with limitations of GzipFile
IV such as it not implementing truncate().

IV Does anyone have an explanation / can you suggest a nice solution for
IV doing what we are trying to do?

stdout (and the others) must be None, PIPE or a real file object or file
descriptor, not a file like object. In your case the solution would be
to use PIPE, and then read the output and write in to the GzipFile
yourself. 

f = gzip.open(filename, 'w')
proc = subprocess.Popen(['ls','-la'], stdout=subprocess.PIPE)
while True:
line = proc.stdout.readline()
if not line: break
f.write(line)
f.close()

-- 
Piet van Oostrum p...@cs.uu.nl
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list