This link might be relevant -- https://github.com/pyinstaller/pyinstaller/wiki/Recipe-subprocess
Have you tried those suggestions? On Thu, Mar 16, 2017 at 10:47 AM, Coyot Linden (Glenn Glazer) < [email protected]> wrote: > We discovered that code which ran perfectly correctly by executing the > scripts in the POSIX environment or via python on the Windows command > prompt would fail when compiled using the -w flag to PyInstaller. This flag > prevents the application from launching a console window when the > application starts, which we clearly did not want with our client. The > problem occurs when a caller executes Python's subprocess and the caller > tries to implicitly or explictly access stderr or stdin on Windows. > > Because this originally occurred in our code with a caller of an imported > module our test code used that mode. We believe it can be reproduced > without the caller. > > We also believe that this error is a deep down result in the pythonw > implementation on Windows and PyInstaller just makes it more obvious. Some > trials suggest that pythonw produces the same failure cases, albeit with > somewhat different output. > > To the PyInstaller team: we found (and is reproducible with the code in > the appendices below) that even with -w, a console window flashes for a > tiny fraction of a second. Is there any way to eliminate that completely? > > ----- > > So beginning with code that fails that shows the example, we have: > > Sample problematic caller code: > > #!/usr/bin/env python > > import cgitb > import os.path > import subwrapper > import sys > > cwd = os.path.dirname(os.path.realpath(str(sys.executable))) > cgitb.enable(logdir=cwd, format='text') > > print subwrapper.getMachineID() > > Sample problematic imported module code: > > #!/usr/bin/env python > > import subprocess > > def getMachineID(): > return subprocess.check_output(['wmic','csproduct','get','UUID']) > > Sample output under python: > > >python caller.py > UUID > D8F59000-4F39-0000-0000-000000000000 > > Sample output when compiled with the -y --clean --onefile flags: > > >caller.exe > UUID > D8F59000-4F39-0000-0000-000000000000 > > Sample output when compiled with the -y -w --clean --onefile flags: > > No output. Failure to execute script window appears (the name depends on > the script name): > > > > Note that this error message is specific to PyInstaller, it comes from > https://github.com/pyinstaller/pyinstaller/blob/ > a70b20e4de6a6817987d28ca9f3201c8105fd858/bootloader/src/pyi_launch.c#L411 > . > > As noted above, we ran the toy scripts under cgitb tracing to obtain > detailed call stack information which does not normally appear when that > Window appears. Full cgitb output is below, but the important part is: > > Traceback (most recent call last): > File "caller.py", line 11, in <module> > File "subwrapper.py", line 6, in getMachineID > File "subprocess.py", line 566, in check_output > File "subprocess.py", line 702, in __init__ > File "subprocess.py", line 823, in _get_handles > WindowsError: [Error 6] The handle is invalid > > Researching that error message led us to: > > https://github.com/incuna/django-wkhtmltopdf/issues/91# > issuecomment-179080434 > > which points at the problem being related to the passing of filehandles, > but suggests hacking subprocess.py, which we were extremely reluctant to do > for production software. > > We also found: > > http://stackoverflow.com/questions/337870/python- > subprocess-call-fails-when-using-pythonw-exe > > which states > > *"sys.stdin and sys.stdout handles are invalid because pythonw does not > provide console support as it runs as a deamon, so default arguments of > subprocess.call() are failing.* > > *Deamon (sic) programs close stdin/stdout/stderr purposedly and use > logging instead, so that you have to manage this yourself: I would suggest > to use subprocess.PIPE."* > > The last comment at the end of that StackOverflow indicates that people > have been running into this problem with pythonw and PyInstaller since 2014. > > Putting these things together, we realized that -w is destroying the > stderr filehandle. We now do subprocess calls this way: > > with open(os.path.join(cwd,'output'),'a') as bar: > with open(os.devnull) as nullin: > try: > foo = subprocess.check_output([' > wmic','csproduct','get','UUID'], stdout=subprocess.PIPE, > stderr=subprocess.PIPE, stdin=nullin) > print >>bar, foo > > which is to say that we never, ever use stdin or stderr on Windows for > anything, explicitly OR implicitly through default options. E.g., passing > None to check_output() for a handle or not specifying a handle causes it to > inherit from the parent and there is nothing to inherit and hence the > invalid handle error message. No print statements, either. All debugging > must go to logs. > > Best, > > coyot > > Appendix A - Correct Version of caller.py > > #!/usr/bin/env python > > import cgitb > import os.path > import subwrapper > import sys > > cwd = os.path.dirname(os.path.realpath(str(sys.executable))) > cgitb.enable(logdir=cwd, format='text') > > with open(os.path.join(cwd,'caller_output'),'a') as bar: > print >>bar, subwrapper.getMachineID() > > Appendix B - Correct Version of subwrapper.py > > Note that while this uses subprocess.PIPE for stderr, in practice we use a > file handle to the log file to capture any potentially diagnostic output. > > #!/usr/bin/env python > > import cgitb > import os > import os.path > import subprocess > import sys > > def getMachineID(): > #return subprocess.check_output(['wmic','csproduct','get','UUID']) > foo = "" > #note sys.executable works in the compiled environment. If you try > this with > #pythonw caller.py, it will write its log files to wherever python is > and probably not > #what you want > cwd = os.path.dirname(os.path.realpath(str(sys.executable))) > cgitb.enable(logdir=cwd, format='text') > with open(os.path.join(cwd,'output'),'a') as bar: > with open(os.devnull) as nullin: > try: > foo = subprocess.check_output([' > wmic','csproduct','get','UUID'], stdout=subprocess.PIPE, > stderr=subprocess.PIPE, stdin=nullin) > except Exception, e: > print >>bar, repr(e) > return foo > > if __name__ == "__main__": > cwd = os.path.dirname(os.path.realpath(str(sys.executable))) > with open(os.path.join(cwd,'output'),'a') as bar: > print >>bar, getMachineID() > > Appendix C - output from caller_output > > >cat caller_output > UUID > D8F59000-4F39-0000-0000-000000000000 > > Appendix D - cgitb output > > N.B., the code lives in a subdirectory of my cygwin homedir, but > everything was executed from the Windows command prompt. > > <type 'exceptions.WindowsError'> > Python 2.7.11: c:\cygwin64\home\coyot\hg\stderrtoy\dist\caller.exe > Wed Mar 15 10:36:25 2017 > > A problem occurred in a Python script. Here is the sequence of > function calls leading up to the error, in the order they occurred. > > c:\cygwin64\home\coyot\hg\stderrtoy\dist\caller.py in <module>() > > > c:\cygwin64\home\coyot\hg\stderrtoy\dist\subwrapper.py in getMachineID() > > > c:\cygwin64\home\coyot\hg\stderrtoy\dist\subprocess.py in > check_output(*popenargs=(['wmic', 'csproduct', 'get', 'UUID'],), > **kwargs={}) > > > c:\cygwin64\home\coyot\hg\stderrtoy\dist\subprocess.py in > __init__(self=<subprocess.Popen object>, args=['wmic', 'csproduct', > 'get', 'UUID'], bufsize=0, executable=None, stdin=None, stdout=-1, > stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, > env=None, universal_newlines=False, startupinfo=None, creationflags=0) > > > c:\cygwin64\home\coyot\hg\stderrtoy\dist\subprocess.py in > _get_handles(self=<subprocess.Popen object>, stdin=None, stdout=-1, > stderr=None) > > <type 'exceptions.WindowsError'>: [Error 6] The handle is invalid > __class__ = <type 'exceptions.WindowsError'> > __delattr__ = <method-wrapper '__delattr__' of exceptions.WindowsError > object> > __dict__ = {} > __doc__ = 'MS-Windows OS system call failed.' > __format__ = <built-in method __format__ of exceptions.WindowsError > object> > __getattribute__ = <method-wrapper '__getattribute__' of > exceptions.WindowsError object> > __getitem__ = <method-wrapper '__getitem__' of exceptions.WindowsError > object> > __getslice__ = <method-wrapper '__getslice__' of > exceptions.WindowsError object> > __hash__ = <method-wrapper '__hash__' of exceptions.WindowsError > object> > __init__ = <method-wrapper '__init__' of exceptions.WindowsError > object> > __new__ = <built-in method __new__ of type object> > __reduce__ = <built-in method __reduce__ of exceptions.WindowsError > object> > __reduce_ex__ = <built-in method __reduce_ex__ of > exceptions.WindowsError object> > __repr__ = <method-wrapper '__repr__' of exceptions.WindowsError > object> > __setattr__ = <method-wrapper '__setattr__' of exceptions.WindowsError > object> > __setstate__ = <built-in method __setstate__ of > exceptions.WindowsError object> > __sizeof__ = <built-in method __sizeof__ of exceptions.WindowsError > object> > __str__ = <method-wrapper '__str__' of exceptions.WindowsError object> > __subclasshook__ = <built-in method __subclasshook__ of type object> > __unicode__ = <built-in method __unicode__ of exceptions.WindowsError > object> > args = (6, 'The handle is invalid') > errno = 9 > filename = None > message = '' > strerror = 'The handle is invalid' > winerror = 6 > > The above is a description of an error in a Python program. Here is > the original traceback: > > Traceback (most recent call last): > File "caller.py", line 11, in <module> > File "subwrapper.py", line 6, in getMachineID > File "subprocess.py", line 566, in check_output > File "subprocess.py", line 702, in __init__ > File "subprocess.py", line 823, in _get_handles > WindowsError: [Error 6] The handle is invalid > > -- > You received this message because you are subscribed to the Google Groups > "PyInstaller" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/pyinstaller. > For more options, visit https://groups.google.com/d/optout. > -- Bryan A. Jones, Ph.D. Associate Professor Department of Electrical and Computer Engineering 231 Simrall / PO Box 9571 Mississippi State University Mississippi state, MS 39762 http://www.ece.msstate.edu/~bjones bjones AT ece DOT msstate DOT edu voice 662-325-3149 fax 662-325-2298 Our Master, Jesus Christ, is on his way. He'll show up right on time, his arrival guaranteed by the Blessed and Undisputed Ruler, High King, High God. - 1 Tim. 6:14b-15 (The Message) -- You received this message because you are subscribed to the Google Groups "PyInstaller" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/pyinstaller. For more options, visit https://groups.google.com/d/optout.
