Re: [Python-ideas] Running Python commands from a Shell
On 2/1/19, Steven D'Aprano wrote: > On Fri, Feb 01, 2019 at 07:21:47PM -0600, eryk sun wrote: > >> As soon as "pipe" is mentioned, anyone familiar with the REPL's >> behavior with pipes should know that making this work will require the >> -i command-line option to force interactive mode. Otherwise stdout >> will be fully buffered. For example: > [...] > > I wonder... could Python automatically detect when it is connected to > pipes and switch buffering off? In most cases we want full buffering when standard I/O is a pipe or disk file. It's more efficient to read/write large chunks from/to the OS. In another message I saw -u mentioned to disable buffering. But that's not sufficient. We need -i to force running the built-in REPL over a pipe, and optionally -q to quiet the initial banner message. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell
On Fri, Feb 01, 2019 at 07:21:47PM -0600, eryk sun wrote:
> bash coproc runs a process in the background with stdin and stdout
> redirected to pipes. The file descriptors for our end of the pipes are
> available in an array with the given name (e.g. P3). The default array
> name is COPROC.
Thanks for the explanation.
> As soon as "pipe" is mentioned, anyone familiar with the REPL's
> behavior with pipes should know that making this work will require the
> -i command-line option to force interactive mode. Otherwise stdout
> will be fully buffered. For example:
[...]
I wonder... could Python automatically detect when it is connected to
pipes and switch buffering off?
> > And are we supposed to know what ">&${P3[1]}" does? It looks like your
> > cat walked over your keyboard.
>
> It redirects the command's standard output (>) to the file descriptor
> (&) in index 1 of the P3 array (${P3[1]}), which is our end of the
> pipe that's connected to stdin of the co-process.
And this is why I don't program in bash :-)
--
Steven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell
On 2/1/19, Steven D'Aprano wrote:
> On Fri, Feb 01, 2019 at 04:28:25PM -0600, Dan Sommers wrote:
>
>> As I indicated in what you quoted, shell co-processes allow you to run a
>> command in the background and interact with that command from your
>> shell.
>
> Okay, but what does that mean in practice? What does it require to make
> it work with Python? What is your expected input and output?
bash coproc runs a process in the background with stdin and stdout
redirected to pipes. The file descriptors for our end of the pipes are
available in an array with the given name (e.g. P3). The default array
name is COPROC.
As soon as "pipe" is mentioned, anyone familiar with the REPL's
behavior with pipes should know that making this work will require the
-i command-line option to force interactive mode. Otherwise stdout
will be fully buffered. For example:
$ coproc P3 { python3 -qi 2>&1; }
[1] 16923
$ echo 'import sys; print(sys.version)' >&${P3[1]}
$ read -t 1 <&${P3[0]} && echo $REPLY
>>> 3.6.7 (default, Oct 22 2018, 11:32:17)
$ read -t 1 <&${P3[0]} && echo $REPLY
[GCC 8.2.0]
$ read -t 1 <&${P3[0]} && echo $REPLY
$ echo 'sys.exit(42)' >&${P3[1]}
$
[1]+ Exit 42 coproc P3 { python3 -qi 2>&1; }
> And are we supposed to know what ">&${P3[1]}" does? It looks like your
> cat walked over your keyboard.
It redirects the command's standard output (>) to the file descriptor
(&) in index 1 of the P3 array (${P3[1]}), which is our end of the
pipe that's connected to stdin of the co-process.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell
On Fri, Feb 01, 2019 at 04:28:25PM -0600, Dan Sommers wrote:
> On 2/1/19 3:48 PM, Steven D'Aprano wrote:
> > On Fri, Feb 01, 2019 at 02:38:43PM -0600, Dan Sommers wrote:
> >
> >> So why not turn that around? ksh (since way back when) and
> >> bash (since 2008, according to what I read somewhere online)
> >> have "co-processes," which allow you to run a command "in
> >> the background," and send commands and receive replies from
> >> it. So I tried it with Python, but it didn't work:
> >>
> >> $ coproc P3 { python3; }
> >> $ echo 'import sys; print(sys.version)' >&${P3[1]}
> >> $ read v <&${P3[0]}
> >> [the read command just waits forever]
> >
> > This is another good example of the problem James was referring to in
> > the thread about clearer communication. Don't assume we all know what
> > coproc does.
>
> As I indicated in what you quoted, shell co-processes allow you to run a
> command in the background and interact with that command from your
> shell.
Okay, but what does that mean in practice? What does it require to make
it work with Python? What is your expected input and output?
This is a Python forum. You can assume your readers have a good level of
knowledge about Python. Outside of that, you can expect to lose a
significant portion of your audience if you start talking about
features, practices etc that aren't supported by the Python language,
stdlib and perhaps a few of the better-known third-party libraries.
E.g. me, I have no idea what you mean by "interact with that command
from your shell". If you're a sys admin, you might do that fifty times a
day for all I know, but I've never knowingly done it.
And are we supposed to know what ">&${P3[1]}" does? It looks like your
cat walked over your keyboard.
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell
On 2/1/19 3:48 PM, Steven D'Aprano wrote:
> On Fri, Feb 01, 2019 at 02:38:43PM -0600, Dan Sommers wrote:
>
>> So why not turn that around? ksh (since way back when) and
>> bash (since 2008, according to what I read somewhere online)
>> have "co-processes," which allow you to run a command "in
>> the background," and send commands and receive replies from
>> it. So I tried it with Python, but it didn't work:
>>
>> $ coproc P3 { python3; }
>> $ echo 'import sys; print(sys.version)' >&${P3[1]}
>> $ read v <&${P3[0]}
>> [the read command just waits forever]
>
> This is another good example of the problem James was referring to in
> the thread about clearer communication. Don't assume we all know what
> coproc does.
As I indicated in what you quoted, shell co-processes allow you to run a
command in the background and interact with that command from your
shell.
>> A pile of experiments and examples from web pages later, I
>> think it's Python and not me. My example, with suitable
>> changes to the literal in the echo command, works with sbcl
>> and erl, but not python3. If I start python3 as follows:
>
> What are sbcl and erl?
>
> I'm guessing you don't mean antimony pentachloride and a municipality in
> Austria. Possibly Steel Bank Common Lisp and Erlang? But I'm not
> confident about that.
Yes, the Steel Bank Common Lisp and Erlang REPLs, respectively.
> Does your example work with more well-known interpreted languages with
> interactive interpreters such as Ruby, Lua, Javascript (outside of the
> browser), etc?
I don't know (I don't write software in any of those languages, and I
don't have them imstalled on my computer), but adding the "-i" flag to
my python3 command makes it work (thanks to ChrisA for suggesting "-u";
it was a short leap from there to "-i.")
Dan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell (was: Option of running shell/console commands inside the REPL)
On Fri, Feb 01, 2019 at 02:38:43PM -0600, Dan Sommers wrote:
> So why not turn that around? ksh (since way back when) and
> bash (since 2008, according to what I read somewhere online)
> have "co-processes," which allow you to run a command "in
> the background," and send commands and receive replies from
> it. So I tried it with Python, but it didn't work:
>
> $ coproc P3 { python3; }
> $ echo 'import sys; print(sys.version)' >&${P3[1]}
> $ read v <&${P3[0]}
> [the read command just waits forever]
This is another good example of the problem James was referring to in
the thread about clearer communication. Don't assume we all know what
coproc does.
> A pile of experiments and examples from web pages later, I
> think it's Python and not me. My example, with suitable
> changes to the literal in the echo command, works with sbcl
> and erl, but not python3. If I start python3 as follows:
What are sbcl and erl?
I'm guessing you don't mean antimony pentachloride and a municipality in
Austria. Possibly Steel Bank Common Lisp and Erlang? But I'm not
confident about that.
Does your example work with more well-known interpreted languages with
interactive interpreters such as Ruby, Lua, Javascript (outside of the
browser), etc?
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell (was: Option of running shell/console commands inside the REPL)
On Sat, Feb 2, 2019 at 7:39 AM Dan Sommers
<2qdxy4rzwzuui...@potatochowder.com> wrote:
>
> On 2/1/19 2:26 AM, João Matos wrote:
> > Hello,
> >
> >
> > Consider adding the option of running shell/console commands inside the
> > REPL.
> > Something like
> !dir
(This might be better on python-list rather than here)
> I first ran into this in the days of teletypes and dumb
> terminals, where other programs let you run shell commands
> from inside them. Now the shoe appears to be on the other
> foot.
>
> So why not turn that around? ksh (since way back when) and
> bash (since 2008, according to what I read somewhere online)
> have "co-processes," which allow you to run a command "in
> the background," and send commands and receive replies from
> it. So I tried it with Python, but it didn't work:
>
> $ coproc P3 { python3; }
> $ echo 'import sys; print(sys.version)' >&${P3[1]}
> $ read v <&${P3[0]}
> [the read command just waits forever]
Looks like you may have a problem with output buffering. Try running
"python3 -u" to tell it to run stdout/stderr unbuffered - otherwise,
Python assumes that its output is going to a file and it won't matter.
Alternatively, explicitly flush stdout after output, which could be
done with a display hook:
_old_hook = sys.displayhook
def display(obj):
_old_hook(obj)
sys.stdout.flush()
sys.displayhook = display
But unless you're expecting a lot of output, it's probably easier and
just as effective to simply use "-u".
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Running Python commands from a Shell (was: Option of running shell/console commands inside the REPL)
On 2/1/19 2:26 AM, João Matos wrote:
Hello,
Consider adding the option of running shell/console commands inside the REPL.
Something like
!dir
I first ran into this in the days of teletypes and dumb
terminals, where other programs let you run shell commands
from inside them. Now the shoe appears to be on the other
foot.
So why not turn that around? ksh (since way back when) and
bash (since 2008, according to what I read somewhere online)
have "co-processes," which allow you to run a command "in
the background," and send commands and receive replies from
it. So I tried it with Python, but it didn't work:
$ coproc P3 { python3; }
$ echo 'import sys; print(sys.version)' >&${P3[1]}
$ read v <&${P3[0]}
[the read command just waits forever]
A pile of experiments and examples from web pages later, I
think it's Python and not me. My example, with suitable
changes to the literal in the echo command, works with sbcl
and erl, but not python3. If I start python3 as follows:
$ coproc P3 { python3 | tee /tmp/P3; }
then I can see the empty /tmp/P3 file, and the python3 and
tee processes, but /tmp/P3 remains empty.
Any ideas as to why not?
Thanks,
Dan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
