[issue29829] Documentation lacks clear warning of subprocess issue with pythonw

[Eryk Sun]

Eryk Sun  added the comment:

I forgot about the much older issue for this problem. I'm closing this issue, 
because the broken behavior should be fixed, not documented.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> subprocess failing in GUI applications on Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29829] Documentation lacks clear warning of subprocess issue with pythonw

[Eryk Sun]

Eryk Sun  added the comment:

This problem can be addressed most easily in _get_handles() by combining the 
case of an invalid standard handle with that of a NULL standard handle, for 
which a file handle is validated via GetFileType(). However, that leaves a 
small window for the handle to become invalid before duplicating it. Instead, 
_get_handles() could deal with a failed _make_inheritable() call by 
substituting a pipe handle.

--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.6, Python 
3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29829] Documentation lacks clear warning of subprocess issue with pythonw

[Eryk Sun]

Eryk Sun added the comment:

An exception (not a crash) is possible if a standard handle is invalid and a 
Popen call tries to replace one or two of the other standard handles (e.g. 
stdin is invalid and you pass the argument stdout=PIPE). Note that subprocess 
uses the standard handles directly, so this problem is unrelated to Python's 
sys.std* streams. IMO, the presence of an invalid handle in the standard 
handles should not cause Popen to fail. This is an old problem, and it should 
have been addressed a long time ago by substituting a handle for \\.\NUL.

In Windows 8+ this isn't particular to running pythonw.exe. In older versions 
such as Windows 7, if you start pythonw.exe from a console application, then 
the parent's standard handle values are copied to the pythonw.exe child. But 
these handles are invalid because pythonw.exe doesn't automatically attach to 
its parent's console. (You have to manually call AttachConsole(-1) or 
AllocConsole(), but that's complicated because you may need to reset the CRT's 
standard file descriptors and FILE streams and Pythons sys.std* streams.)

Note that pythonw.exe doesn't delete or close the standard streams. They're 
normally set to None because, in the default case, the Windows standard handles 
aren't valid in pythonw.exe, and thus the CRT maps its standard file 
descriptors (0, 1, and 2) to INVALID_HANDLE_VALUE (i.e. -1 cast as a pointer). 
For example:

C:\Temp>pyw -2 -c "import msvcrt; open('test.txt', 
'w').write(str(msvcrt.get_osfhandle(1)))"

C:\Temp>type test.txt
18446744073709551614

You can override this by starting pythonw.exe with explicit standard handles. 
For example, the following redirects stderr (2) to stdout (1) and redirects 
stdout to a pipe:

C:\>set "script=import sys; print(sys.executable); print(sys.stdout); 
print(sys.stderr)"

C:\>2>&1 pyw -2 -c "%script%" | more
C:\Program Files\Python27\pythonw.exe
', mode 'w' at 0x021FB0C0>
', mode 'w' at 0x021FB150>

C:\>2>&1 pyw -3 -c "%script%" | more
C:\Program Files\Python36\pythonw.exe
<_io.TextIOWrapper name='' mode='w' encoding='cp1252'>
<_io.TextIOWrapper name='' mode='w' encoding='cp1252'>

--
components: +Windows
nosy: +eryksun, paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29829] Documentation lacks clear warning of subprocess issue with pythonw

[R. David Murray]

R. David Murray added the comment:

A warning is not appropriate (we reserve those for things that are security 
related, pretty much). A sentence might be, though.  For example, we could 
change the initial discussion of the run function to say:

   This does not capture stdout or stderr by default. To do so, pass PIPE for 
the stdout and/or stderr arguments.  You *must* do this if you run your python 
program with pythonw on windows, since pythonw closes the standard streams.

Here I'm adding the windows sentence to the existing "This does not capture" 
sentence.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29829] Documentation lacks clear warning of subprocess issue with pythonw

[Steve Barnes]

New submission from Steve Barnes:

When running under pythonw, or pyinstaller with the -w flag, modules that use 
subprocess calls such as popen, run, etc. will crash if the default 
`stdout=None, stderr=None` behaviour is used rather than PIPE. This is an 
obscure problem which is very hard to debug yet there is no warning in the 
documentation on this.

I would like to suggest adding a :warning:`stdout=None, stderr=None` must not 
be used in any of the calls in this module when running under pythonw due to 
the lack of sys.stdout & sys.stderr in that case. Please use `stdout=PIPE, 
stderr=PIPE` instead.

A patch against the default branch would be:
diff -r 4243df51fe43 Doc/library/subprocess.rst
--- a/Doc/library/subprocess.rstFri Feb 10 14:19:36 2017 +0100
+++ b/Doc/library/subprocess.rstThu Mar 16 16:56:24 2017 +
@@ -33,6 +33,13 @@
 function for all use cases it can handle. For more advanced use cases, the
 underlying :class:`Popen` interface can be used directly.
 
+.. warning:: Do not use default parameters on Windows with pythonw.
+
+   As pythonw deletes `sys.stdout` & `sys.stderr` the use of the default 
+   parameters, `stdout=None, stderr=None,`, which defaults to being
+   `stdout=sys.stdout, stderr=sys.stderr,` may cause unexpected crashes
+   it is recommended to use `stdout=PIPE, stderr=PIPE,` instead.
+
 The :func:`run` function was added in Python 3.5; if you need to retain
 compatibility with older versions, see the :ref:`call-function-trio` section.

--
assignee: docs@python
components: Documentation
messages: 289722
nosy: Steve Barnes, docs@python
priority: normal
severity: normal
status: open
title: Documentation lacks clear warning of subprocess issue with pythonw
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com