[issue3548] subprocess.pipe function

[Mike Frysinger]

Mike Frysinger  added the comment:

this would have been nice to have more than once in my personal projects, and 
in build/infra tooling for Chromium OS.  our alternatives so far have been the 
obvious ones: use subprocess.run to capture & return the output, then manually 
feed that as the input to the next command, and so on.  we know it has obvious 
overhead so we've avoided with large output.

we strongly discourage/ban attempts to write shell code, so the vast majority 
of our commands are argv style (list of strings), so the pipes module wouldn't 
help us.

handling of SIGPIPE tends to be where things get tricky.  that'll have to be 
handled by the API explicitly i think.

--
nosy: +vapier

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[desbma]

Changes by desbma :


--
nosy: +desbma

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Matheus Vieira Portela]

Matheus Vieira Portela added the comment:

Oh, I see... The pipes module uses os.system and os.popen to execute commands 
under a /bin/sh environment. So you are proposing to re-implement it with 
subprocess in order to execute commands without shell and, also, extend by 
accepting argv-style parameters. Is this right this time?

How would an argv-style pipe look like?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[R. David Murray]

R. David Murray added the comment:

Exactly like the string pipes API, but cmd would be a list of arguments instead 
of a string.  That would then become the argument list to the subprocess stage.

It is an interesting question whether we'd want to allow mixing stage types.  
I'd say no (at least initially, if people complain we can add it as a feature 
later, but we can't subtract it if we implement it now and decide it is a bad 
idea after putting it in the field).

Now, keep in mind that you've only got my opinion so far that this is even a 
good idea :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Matheus Vieira Portela]

Matheus Vieira Portela added the comment:

Sure. I'll leave this issue for a while before others can emit their opinions.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Matheus Vieira Portela]

Matheus Vieira Portela added the comment:

Let me check whether I understood you suggestion...

What you are saying is that it is already possible to pipeline shell commands 
through subprocess, such as in the following line?

subprocess.call('ls -l | grep Music', shell=True)

However, this requires the command to be executed in a shell environment. 
Hence, it would be a good idea to extend it to non-shell command execution. Is 
this right?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[R. David Murray]

R. David Murray added the comment:

Thanks for being willing to work on it.

If what is wanted is a way to pipeline shell commands, Python already has that 
functionality in the pipes module.

So the interesting thing here would be pipelining *non* shell commands, to 
avoid the shell exploits that using a shell pipeline invites.

The pipes module already has a worked out API, so perhaps it would be useful to 
see about re-implementing pipe's command execution using subprocess, and expand 
the API to allow for argv style command specification that would be fed to 
subprocess using the default shell=False.  This would also presumably allow 
pipes to be used when there's no bash shell available.

The downside is that we might break current uses of pipes if we replace the 
shell version of the pipelining with subprocess shell=True, while using 
subprocess only if the command specifications are argv lists would result in 
code with a split personality.  But if I were working on it I'd experiment with 
that approach to see if it made sense.

Other core devs may have other opinions on this :)

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Matheus Vieira Portela]

Matheus Vieira Portela added the comment:

I just found this open issue and I can work on it. What is left to do before 
closing it?

--
nosy: +matheus.v.portela

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

I think this would be more useful if you could pass an optional input string 
(as in communicate()) and if it returned a (stdout, stderr) tuple. Or perhaps 
even a (return code, stdout, stderr) tuple; alternately, non-zero return codes 
could raise an exception.

--
nosy: +pitrou
versions: +Python 3.3 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Miki Tebeka]

Changes by Miki Tebeka miki.teb...@gmail.com:


--
nosy:  -tebeka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

pipe.patch looks interesting to me.  I would replace **kwargs with a 
keyword-only argument named stderr, since that’s the only key used.  This 
requires more tests and docs.

--
nosy: +eric.araujo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Anh Hai Trinh]

Anh Hai Trinh anh.hai.tr...@gmail.com added the comment:

I've written a package which can do this with arbitrary redirection in all 
subcommands (and some more).

You can, for example, do this:

 Pipe(Sh('echo -n foo 2', {2: 1}), Sh('cat; echo ERROR 2', {2: 
os.devnull})).capture(1).read()
'foo'

The package is at: http://github.com/aht/pc.py

--
nosy: +aht

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Ernesto Menéndez]

Changes by Ernesto Menéndez pya...@gmail.com:


--
nosy: +Netto

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Georg Brandl]

Changes by Georg Brandl ge...@python.org:


--
versions: +Python 3.2 -Python 2.7, Python 3.1

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Legoll Vincent]

Legoll Vincent [EMAIL PROTECTED] added the comment:

On Mon, Sep 1, 2008 at 5:13 PM, Amaury Forgeot d'Arc
[EMAIL PROTECTED] wrote:
 Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:

 Vincent,
 GPL licenced code is incompatible with the inclusion into python.
 And if I am correct, you should sign a contributor agreement. Then the
 licence text is not necessary.

This is not a patch against python, this is a standalone script, just
implementing the same functionality as the original bug report in a
slightly different way...

But thanks for the input anyways.

If this functionality is really interesting people and agreed to be integrated
into upstream subprocess module, I can rework it the right way, or work
from the original patch from the bug report.

--
nosy: +vlegoll

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Vincent Legoll]

Vincent Legoll [EMAIL PROTECTED] added the comment:

- Added shut pylint up comment for ** keyword expansion
- Added Copyright  license header

Added file: http://bugs.python.org/file11330/pipeline.py

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:

Vincent,
GPL licenced code is incompatible with the inclusion into python.
And if I am correct, you should sign a contributor agreement. Then the
licence text is not necessary.

--
nosy: +amaury.forgeotdarc

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Vincent Legoll]

Vincent Legoll [EMAIL PROTECTED] added the comment:

Hello,

I was searching for a bug in subprocess
module when I saw your patch.

I was implementing the exact same functionality
and mixed some of your ideas in what I use now,
which is attached...

Feel free to use it

--
nosy: +vincent.legoll
Added file: http://bugs.python.org/file11262/toto.py

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Vincent Legoll]

Vincent Legoll [EMAIL PROTECTED] added the comment:

Here's a clean version with doc  test

enjoy !

Added file: http://bugs.python.org/file11263/pipeline.py

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Antoine Pitrou]

Changes by Antoine Pitrou [EMAIL PROTECTED]:


--
priority:  - normal
versions: +Python 2.7, Python 3.1 -Python 2.6, Python 3.0

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Miki Tebeka]

New submission from Miki Tebeka [EMAIL PROTECTED]:

Attached is a patch that add pipe command to the subprocess module.

pipe([ls], [grep, test_]) will return the output of 
ls | grep test_.

--
components: Library (Lib)
files: pipe.patch
keywords: patch
messages: 71062
nosy: tebeka
severity: normal
status: open
title: subprocess.pipe function
type: feature request
versions: Python 2.6, Python 3.0
Added file: http://bugs.python.org/file11104/pipe.patch

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3548] subprocess.pipe function

[Miki Tebeka]

Miki Tebeka [EMAIL PROTECTED] added the comment:

Not sure about the name, maybe chain will be better?

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3548
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com