This link might be relevant
-- https://github.com/pyinstaller/pyinstaller/wiki/Recipe-subprocess
Have you tried those suggestions?
On Thu, Mar 16, 2017 at 10:47 AM, Coyot Linden (Glenn Glazer)
<[email protected] <mailto:[email protected]>> wrote:
We discovered that code which ran perfectly correctly by executing
the scripts in the POSIX environment or via python on the Windows
command prompt would fail when compiled using the -w flag to
PyInstaller. This flag prevents the application from launching a
console window when the application starts, which we clearly did
not want with our client. The problem occurs when a caller
executes Python's subprocess and the caller tries to implicitly or
explictly access stderr or stdin on Windows.
Because this originally occurred in our code with a caller of an
imported module our test code used that mode. We believe it can
be reproduced without the caller.
We also believe that this error is a deep down result in the
pythonw implementation on Windows and PyInstaller just makes it
more obvious. Some trials suggest that pythonw produces the same
failure cases, albeit with somewhat different output.
To the PyInstaller team: we found (and is reproducible with the
code in the appendices below) that even with -w, a console window
flashes for a tiny fraction of a second. Is there any way to
eliminate that completely?
-----
So beginning with code that fails that shows the example, we have:
Sample problematic caller code:
#!/usr/bin/env python
import cgitb
import os.path
import subwrapper
import sys
cwd = os.path.dirname(os.path.realpath(str(sys.executable)))
cgitb.enable(logdir=cwd, format='text')
print subwrapper.getMachineID()
Sample problematic imported module code:
#!/usr/bin/env python
import subprocess
def getMachineID():
return subprocess.check_output(['wmic','csproduct','get','UUID'])
Sample output under python:
>python caller.py
UUID
D8F59000-4F39-0000-0000-000000000000
Sample output when compiled with the -y --clean --onefile flags:
>caller.exe
UUID
D8F59000-4F39-0000-0000-000000000000
Sample output when compiled with the -y -w --clean --onefile flags:
No output. Failure to execute script window appears (the name
depends on the script name):
Note that this error message is specific to PyInstaller, it comes
from
https://github.com/pyinstaller/pyinstaller/blob/a70b20e4de6a6817987d28ca9f3201c8105fd858/bootloader/src/pyi_launch.c#L411
<https://github.com/pyinstaller/pyinstaller/blob/a70b20e4de6a6817987d28ca9f3201c8105fd858/bootloader/src/pyi_launch.c#L411>
.
As noted above, we ran the toy scripts under cgitb tracing to
obtain detailed call stack information which does not normally
appear when that Window appears. Full cgitb output is below, but
the important part is:
Traceback (most recent call last):
File "caller.py", line 11, in <module>
File "subwrapper.py", line 6, in getMachineID
File "subprocess.py", line 566, in check_output
File "subprocess.py", line 702, in __init__
File "subprocess.py", line 823, in _get_handles
WindowsError: [Error 6] The handle is invalid
Researching that error message led us to:
https://github.com/incuna/django-wkhtmltopdf/issues/91#issuecomment-179080434
<https://github.com/incuna/django-wkhtmltopdf/issues/91#issuecomment-179080434>
which points at the problem being related to the passing of
filehandles, but suggests hacking subprocess.py, which we were
extremely reluctant to do for production software.
We also found:
http://stackoverflow.com/questions/337870/python-subprocess-call-fails-when-using-pythonw-exe
<http://stackoverflow.com/questions/337870/python-subprocess-call-fails-when-using-pythonw-exe>
which states
/"sys.stdin and sys.stdout handles are invalid because pythonw
does not provide console support as it runs as a deamon, so
default arguments of subprocess.call() are failing.//
//
//Deamon (sic) programs close stdin/stdout/stderr purposedly and
use logging instead, so that you have to manage this yourself: I
would suggest to use subprocess.PIPE."/
The last comment at the end of that StackOverflow indicates that
people have been running into this problem with pythonw and
PyInstaller since 2014.
Putting these things together, we realized that -w is destroying
the stderr filehandle. We now do subprocess calls this way:
with open(os.path.join(cwd,'output'),'a') as bar:
with open(os.devnull) as nullin:
try:
foo =
subprocess.check_output(['wmic','csproduct','get','UUID'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=nullin)
print >>bar, foo
which is to say that we never, ever use stdin or stderr on Windows
for anything, explicitly OR implicitly through default options.
E.g., passing None to check_output() for a handle or not
specifying a handle causes it to inherit from the parent and there
is nothing to inherit and hence the invalid handle error message.
No print statements, either. All debugging must go to logs.
Best,
coyot
Appendix A - Correct Version of caller.py
#!/usr/bin/env python
import cgitb
import os.path
import subwrapper
import sys
cwd = os.path.dirname(os.path.realpath(str(sys.executable)))
cgitb.enable(logdir=cwd, format='text')
with open(os.path.join(cwd,'caller_output'),'a') as bar:
print >>bar, subwrapper.getMachineID()
Appendix B - Correct Version of subwrapper.py
Note that while this uses subprocess.PIPE for stderr, in practice
we use a file handle to the log file to capture any potentially
diagnostic output.
#!/usr/bin/env python
import cgitb
import os
import os.path
import subprocess
import sys
def getMachineID():
#return subprocess.check_output(['wmic','csproduct','get','UUID'])
foo = ""
#note sys.executable works in the compiled environment. If you
try this with
#pythonw caller.py, it will write its log files to wherever
python is and probably not
#what you want
cwd = os.path.dirname(os.path.realpath(str(sys.executable)))
cgitb.enable(logdir=cwd, format='text')
with open(os.path.join(cwd,'output'),'a') as bar:
with open(os.devnull) as nullin:
try:
foo =
subprocess.check_output(['wmic','csproduct','get','UUID'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=nullin)
except Exception, e:
print >>bar, repr(e)
return foo
if __name__ == "__main__":
cwd = os.path.dirname(os.path.realpath(str(sys.executable)))
with open(os.path.join(cwd,'output'),'a') as bar:
print >>bar, getMachineID()
Appendix C - output from caller_output
>cat caller_output
UUID
D8F59000-4F39-0000-0000-000000000000
Appendix D - cgitb output
N.B., the code lives in a subdirectory of my cygwin homedir, but
everything was executed from the Windows command prompt.
<type 'exceptions.WindowsError'>
Python 2.7.11: c:\cygwin64\home\coyot\hg\stderrtoy\dist\caller.exe
Wed Mar 15 10:36:25 2017
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
c:\cygwin64\home\coyot\hg\stderrtoy\dist\caller.py in <module>()
c:\cygwin64\home\coyot\hg\stderrtoy\dist\subwrapper.py in
getMachineID()
c:\cygwin64\home\coyot\hg\stderrtoy\dist\subprocess.py in
check_output(*popenargs=(['wmic', 'csproduct', 'get', 'UUID'],),
**kwargs={})
c:\cygwin64\home\coyot\hg\stderrtoy\dist\subprocess.py in
__init__(self=<subprocess.Popen object>, args=['wmic',
'csproduct', 'get', 'UUID'], bufsize=0, executable=None,
stdin=None, stdout=-1, stderr=None, preexec_fn=None,
close_fds=False, shell=False, cwd=None, env=None,
universal_newlines=False, startupinfo=None, creationflags=0)
c:\cygwin64\home\coyot\hg\stderrtoy\dist\subprocess.py in
_get_handles(self=<subprocess.Popen object>, stdin=None,
stdout=-1, stderr=None)
<type 'exceptions.WindowsError'>: [Error 6] The handle is invalid
__class__ = <type 'exceptions.WindowsError'>
__delattr__ = <method-wrapper '__delattr__' of
exceptions.WindowsError object>
__dict__ = {}
__doc__ = 'MS-Windows OS system call failed.'
__format__ = <built-in method __format__ of
exceptions.WindowsError object>
__getattribute__ = <method-wrapper '__getattribute__' of
exceptions.WindowsError object>
__getitem__ = <method-wrapper '__getitem__' of
exceptions.WindowsError object>
__getslice__ = <method-wrapper '__getslice__' of
exceptions.WindowsError object>
__hash__ = <method-wrapper '__hash__' of
exceptions.WindowsError object>
__init__ = <method-wrapper '__init__' of
exceptions.WindowsError object>
__new__ = <built-in method __new__ of type object>
__reduce__ = <built-in method __reduce__ of
exceptions.WindowsError object>
__reduce_ex__ = <built-in method __reduce_ex__ of
exceptions.WindowsError object>
__repr__ = <method-wrapper '__repr__' of
exceptions.WindowsError object>
__setattr__ = <method-wrapper '__setattr__' of
exceptions.WindowsError object>
__setstate__ = <built-in method __setstate__ of
exceptions.WindowsError object>
__sizeof__ = <built-in method __sizeof__ of
exceptions.WindowsError object>
__str__ = <method-wrapper '__str__' of exceptions.WindowsError
object>
__subclasshook__ = <built-in method __subclasshook__ of type
object>
__unicode__ = <built-in method __unicode__ of
exceptions.WindowsError object>
args = (6, 'The handle is invalid')
errno = 9
filename = None
message = ''
strerror = 'The handle is invalid'
winerror = 6
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "caller.py", line 11, in <module>
File "subwrapper.py", line 6, in getMachineID
File "subprocess.py", line 566, in check_output
File "subprocess.py", line 702, in __init__
File "subprocess.py", line 823, in _get_handles
WindowsError: [Error 6] The handle is invalid
--
You received this message because you are subscribed to the Google
Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at https://groups.google.com/group/pyinstaller
<https://groups.google.com/group/pyinstaller>.
For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
Bryan A. Jones, Ph.D.
Associate Professor
Department of Electrical and Computer Engineering
231 Simrall / PO Box 9571
Mississippi State University
Mississippi state, MS 39762
http://www.ece.msstate.edu/~bjones <http://www.ece.msstate.edu/%7Ebjones>
bjones AT ece DOT msstate DOT edu
voice 662-325-3149
fax 662-325-2298
