[issue3905] subprocess failing in GUI applications on Windows

[Eryk Sun]

Eryk Sun  added the comment:

For whatever the reason and Windows version, it's still the case that 
_get_handles() should work around any bad standard handles in the current 
process. In bpo-25492, I suggested checking os.get_handle_inheritable(). That's 
too permissive. It should require a valid file handle, checked via 
_winapi.GetFileType(). For example:

if stdin is None:
p2cread = _winapi.GetStdHandle(_winapi.STD_INPUT_HANDLE)
if p2cread is not None:
try:
_winapi.GetFileType(p2cread)
except OSError:
p2cread = None

--
components: +Library (Lib)
versions: +Python 3.10, Python 3.9 -Python 2.7, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> A current test on a Win 7 machine similar to what I did  
> would still be good.  

Python 3.8 is the last version to support Windows 7 (i.e. NT 6.1). It should 
get bug fixes through Spring 2021, so we have a couple more years to fix this.

We should constrain the fix to just older versions of Windows (prior to 6.2) 
and the classic console file handles (i.e. the lower 2 bits are set such as 3, 
7, 11) that they use. Otherwise a bad standard handle should fail the call.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Since Win 10 was released a year after Jun 2014, I must have still been on Win 
7 when I saw the failure. I believe there have some patches to filenos and 
subprocess since, but I don't know the relation between filenos and file 
handles.  A current test on a Win 7 machine similar to what I did  would still 
be good.  Perhaps it would have been better to reopen #25492, but I am adding 
the current Windows people nosy here.

--
nosy: +paul.moore, steve.dower, tim.golden, zach.ware -georg.brandl, jmfauth
stage:  -> test needed
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> I am closing this as out of date because the problem seems to have been 
> fixed elsewise.

This is a problem only in Windows 7, which we should have addressed years ago. 
It's common enough that we shouldn't leave it as just an unresolved third-party 
issue. I discussed the problem and suggested a workaround in issue 25492, which 
I closed as a duplicate of this issue four years ago.

--
resolution: out of date -> 
stage: resolved -> 
type:  -> behavior
versions: +Python 3.7, Python 3.8 -Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I reran test with IDLE started 4 ways:
I am closing this as out of date because the problem seems to have been fixed 
elsewise.

3.8.0a2+ 32 bit debug python, built today, opened from python icon, then idle 
started with 'import idlelib.idle'
3.8.0a2 64 bit installed, directly from IDLE icon
3.7.2 python -m idlelib
3.7.2 pythonw -m idlelib

#  and it ran correctly all 4 times.
>>> import subprocess
>>> p = subprocess.Popen(["python", "-c", "print(32)"], stdin=None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> p.communicate()
(b'32\r\n', b'')

Anyone with the same or similar failure on current Python (now 3.7, 3.8) should 
open a new issue and give careful details -- release or build, system, startup 
procedure, and a reproducing example only using stdlib modules (no PythonWin).

--
resolution:  -> out of date
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Eryk Sun]

Change by Eryk Sun :


--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Boris]

Boris added the comment:

Probably the same issue: everything works when called from command line, but 
when called via a desktop shortcut I get 
...
_winapi.DUPLICATE_SAME_ACCESS)
OSError: [WinError 6] The handle is invalid

Changing pythonw to python solved the problem. ...this could be expected 
behavior. If anyone is interested I can attach my test files.

--
nosy: +borisompol
versions: +Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Mark Lawrence]

Mark Lawrence added the comment:

I cannot reproduce this problem with 3.4.1 or 3.5.a0 on Windows 7.

--
nosy: +BreamoreBoy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Terry J. Reedy]

Terry J. Reedy added the comment:

Mark, you did not specify which of the variations you tried ;-) I reproduce as 
I said in my last message.

C:\Programs\Python34pythonw -m idlelib

Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [
 import subprocess
 p = subprocess.Popen([python, -c, print(32)], stdin=None,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Traceback (most recent call last):
...
_winapi.DUPLICATE_SAME_ACCESS)
OSError: [WinError 6] The handle is invalid

Change startup command from pythonw to python, no error, and it runs.
 p.communicate()
(b'32\r\n', b'')

--
stage: test needed - needs patch
versions: +Python 3.4, Python 3.5 -Python 3.2, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Mark Lawrence]

Mark Lawrence added the comment:

*facepalm* looks like I've run it with the wrong exe, sorry about the noise :-(

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Tim Lesher]

Changes by Tim Lesher tles...@gmail.com:


--
nosy: +tlesher

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Terry J. Reedy]

Terry J. Reedy tjre...@udel.edu added the comment:

In 3.2.2, starting IDLE from a Command Prompt, I get the same error as Denver. 
Starting IDLE 'normally', from the icon, there is no error. So even though 
Command Prompt detaches pythonw in some way and gives me a new prompt to do 
something else, there is something different about the pythonw process that 
trips up subprocess.

Changing [python, -c, print(32)] to [notepad] gives the same result. 
Running in the interactive interpreted eliminated the error. So does changing 
stdin=None to stdin=subprocess.PIPE eliminates the error and brings up Notepad, 
which seems to operate normally. (I am guessing that it ignores stdin, etc.).

I see that that the latter was known in the original post and plausibly 
diagnosed by Trent Mick, but never fixed because it was not obvious how to do 
so.

Can the WindowsError be caught in subproccess, and if it is this error (6), 
follow the alternate patch that is used for NULL returns? Could that cause 
worse problems that the current situation?

If this is unfixable due to Windows behavior, perhaps we should say something 
in the doc.

--
nosy: +terry.reedy
stage:  - test needed
versions: +Python 2.7, Python 3.2, Python 3.3 -Python 3.0

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Christian Häggström]

Changes by Christian Häggström paj...@kalvdans.no-ip.org:


--
nosy: +chn

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Todd Whiteman]

Todd Whiteman twhit...@yahoo.com.au added the comment:

I still see this problem (in Python 2.7.1 and Python 3.1.2).

When testing using idle, you'll need to launch using pythonw.exe, i.e.:

pythonw.exe lib\idlelib\idle.py

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Denver Coneybeare]

Denver Coneybeare denver.coneybe...@gmail.com added the comment:

Ahh okay.  I've reproduced it in trunk at changeset 053bc5ca199b.

As suggested, I ran: PCBuild\pythonw.exe lib\idlelib\idle.py

Python 3.3a0 (default, Apr  2 2011, 21:55:40) [MSC v.1500 32 bit (Intel)] on 
win32
Type copyright, credits or license() for more information.
 import subprocess
 subprocess.Popen([python, -c, print(32)], stdin=None, 
 stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Traceback (most recent call last):
  File pyshell#1, line 1, in module
subprocess.Popen([python, -c, print(32)], stdin=None, 
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File C:\dev\cpython\cpython\lib\subprocess.py, line 732, in __init__
errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File C:\dev\cpython\cpython\lib\subprocess.py, line 903, in _get_handles
p2cread = self._make_inheritable(p2cread)
  File C:\dev\cpython\cpython\lib\subprocess.py, line 946, in 
_make_inheritable
_subprocess.DUPLICATE_SAME_ACCESS)
WindowsError: [Error 6] The handle is invalid
 

So, the issue is definitely not fixed.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Denver Coneybeare]

Denver Coneybeare denver.coneybe...@gmail.com added the comment:

I just re-tested this in cpython trunk at changeset and the issue does not 
appear to be reproducible.

I first launched IDLE by running python lib\idlelib\idle.py.  Then I entered 
the following:

Python 3.3a0 (default, Apr  2 2011, 21:55:40) [MSC v.1500 32 bit (Intel)] on 
win32
Type copyright, credits or license() for more information.
 import sys
 sys.getwindowsversion()
sys.getwindowsversion(major=5, minor=1, build=2600, platform=2, 
service_pack='Service Pack 3')
 import subprocess
 PIPE = subprocess.PIPE
 p = subprocess.Popen([python, -c, print(32)], stdin=None, 
 stdout=PIPE, stderr=PIPE)
 p
subprocess.Popen object at 0x00F85610
 p.returncode


Popen() did not raise any exceptions.  I also tried in Python 2.7 and the Popen 
called succeeded there as well.  So my inability to reproduce the issue does 
not necessarily mean that the issue is fixed, but rather that it is likely 
dependent on the version of Windows on which Python is running.

That being said, the linked issue issue1124861 logs what appears to be the same 
issue and it was fixed in Python 2.6.  So maybe I'm just not old enough to have 
encountered this issue :)  Anyways, this issue probably can be closed as either 
a duplicate or fixed incidentally.

--
nosy: +denversc

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Weird]

Weird [EMAIL PROTECTED] added the comment:

Georg Brandl thank you, i was having the same problem, but now is fixed

--
nosy: +Weird

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Georg Brandl]

Georg Brandl [EMAIL PROTECTED] added the comment:

Jean-Michel, you'll need to apply the idlelib/run.py patch; replace

sockthread.set_daemon(True)

with

sockthread.daemon = True

around line 75.

--
resolution:  - fixed
status: open - closed
versions: +Python 3.0 -Python 2.5, Python 2.6

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Todd Whiteman]

Todd Whiteman [EMAIL PROTECTED] added the comment:

Uh, the Idle bug reported by Jean-Michel is a completely different bug
(please see the first three messages of this report).

Please re-open this bug, as the subprocess issue I have reported is
still outstanding.

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Georg Brandl]

Changes by Georg Brandl [EMAIL PROTECTED]:


--
resolution: fixed - 
status: closed - open

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Jean-Michel Fauth]

Jean-Michel Fauth [EMAIL PROTECTED] added the comment:

 Georg Brandl

Thanks, patch applied on w2k+sp4 box, swiss french, Python 3.0rc1.
IDLE is working fine.

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Georg Brandl]

Georg Brandl [EMAIL PROTECTED] added the comment:

There are also instances of set_daemon left in socketserver and
multiprocessing/dummy. How is that possible?

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Georg Brandl]

Georg Brandl [EMAIL PROTECTED] added the comment:

Benjamin, I think you're responsible.

--
assignee:  - benjamin.peterson
nosy: +benjamin.peterson, georg.brandl

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Benjamin Peterson]

Benjamin Peterson [EMAIL PROTECTED] added the comment:

The idle problem has already been fixed, and I got the socket server one
in r66520.

--
assignee: benjamin.peterson - 

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Jean-Michel Fauth]

Jean-Michel Fauth [EMAIL PROTECTED] added the comment:

Just for information and from an end user perspective.

I have tried to replace the socketserver.py from the original 3.0rc1
distribution by the socketserver.py as proposed by Benjamin Peterson
(r66520).

Script difference (line 568):

if self.daemon_threads:
t.daemon = True
t.start()

and 

if self.daemon_threads:
t.set_daemon(True)
t.start()

Unfortunately, no luck, I'm still getting exactly the same error
messages, the msg box and the raised AttributeError:

AttributeError: 'Thread' object has no attribute 'set_daemon'

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Kevin Watters]

Changes by Kevin Watters [EMAIL PROTECTED]:


--
nosy: +kevinwatters

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Jean-Michel Fauth]

Jean-Michel Fauth [EMAIL PROTECTED] added the comment:

I do not really know if this is related to this, but I suspect yes.
On my w2k+sp4 box, swiss french, Python 3.0rc1, IDLE does not start or
more precisely it starts by opening the following message box:

Subprocess Startup Error
---
IDLE's subprocess didn't make connection.  Either IDLE can't start a
subprocess or personal firewall software is blocking the connection.

It is certainly neither a firewall issue, nor a tkinter issue (tkinter 
applications are working fine.)

No problem with the 3.0b2 realease, 3.0b3: not tested.

--
nosy: +jmfauth

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:

jmfauth: please try to run idle from a command prompt:

 cd path_to_python3.0rc1
 python Lib/idlelib/idle.py

Do you see interesting output there?

--
nosy: +amaury.forgeotdarc

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Jean-Michel Fauth]

Jean-Michel Fauth [EMAIL PROTECTED] added the comment:

Microsoft Windows 2000 [Version 5.00.2195]
(C) Copyright 1985-2000 Microsoft Corp.

C:\cd python30

C:\Python30python Lib/idlelib/idle.py
Traceback (most recent call last):
  File string, line 1, in module
  File C:\Python30\lib\idlelib\run.py, line 76, in main
sockthread.set_daemon(True)
AttributeError: 'Thread' object has no attribute 'set_daemon'

C:\Python30

Sorry if I can help here, things like socket and subprocess are not my
cup of tea.

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Todd Whiteman]

Todd Whiteman [EMAIL PROTECTED] added the comment:

This seems to be somewhat related to issue1124861:
http://bugs.python.org/issue1124861

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3905] subprocess failing in GUI applications on Windows

[Trent Mick]

Trent Mick [EMAIL PROTECTED] added the comment:

The failure is in the DuplicateHandle call that subprocess makes on, in
this case, the stdin handle (as returned by
`GetStdHandle(STD_INPUT_HANDLE)`) call earlier.

Two cases here:

1. When this is run in a subsystem:windows process (like PythonWin or
IDLE) that is launched from Windows Explorer (e.g. from a Start Menu
shortcut or Desktop shortcut) then `GetStdHandle(STD_INPUT_HANDLE)`
returns None. 

[http://msdn.microsoft.com/en-us/library/ms683231(VS.85).aspx]
 If an application does not have associated standard handles,
 such as a service running on an interactive desktop, and has
 not redirected them, the return value is NULL.

In this case you *don't* get the error that Todd described, because the
code path taken in subprocess.py then use CreatePipe for the `p2cread`
variable on which `DuplicateHandle` is called.

2. However, when the subsystem:windows process is launched from the
cmd.exe shell the `GetStdHandle` call returns a value -- in Todd's and
my testing, the value 3.

The code path in subprocess.py then calls `DuplicateHandle` on this in
`Popen._make_inheritable`. This fails with traceback Todd posted.

My *guess* at what the problem is stems from this comment in the MSDN
docs on console handles:

[http://msdn.microsoft.com/en-us/library/ms682075(VS.85).aspx]
 A process can use the DuplicateHandle function to create a 
 duplicate console handle that has different access or 
 inheritability from the original handle. Note, however,
 that a process can create a duplicate console handle only
 for its own use. This differs from other handle types (such
 as file, pipe, or mutex objects), for which DuplicateHandle
 can create a duplicate that is valid for a different process.

My guess is that the stdin handle (3) is inherited from the shell
(cmd.exe) and attempting to `DuplicateHandle` on it violates the clause
that you can on dupe a console handle of your own. I'm not sure of that
though.

Anyone else have more light to shed on this?

If this is the case I'm not sure what a possible or good solution could
be for subprocess.

___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3905
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com