New submission from Segev Finer:
_WindowsConsoleIO works by taking the handle out of the file descriptors of the
stdio handles (get_osfhandle) and reading/writing to it directly using
ReadConsoleW/WriteConsoleW.
The problem is that some Python code wants to do file descriptor level
redirection by overwriting the standard file descriptors using dup2. The
problem is that this is also going to close the handle that Python itself is
going to use to read/write to the console. Leading to the following:
>>> fd = os.open('stdout.txt', os.O_CREAT | os.O_WRONLY)
>>> os.dup2(fd, 1)
>>> print('spam')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [WinError 87] The parameter is incorrect # "OSError: [WinError 6]
The handle is invalid" after https://bugs.python.org/issue30544
This manifests itself for example in Pytest which does fd redirection to
capture test output. See https://github.com/pytest-dev/py/issues/103 for the
issue. And see https://github.com/pytest-dev/pytest/pull/2462 for an WIP
attempt to workaround this.
This issue also impacts other code that uses console handles itself like
colorama: https://github.com/pytest-dev/pytest/issues/2465. Though that code is
likely going to have to deal with this by itself.
Those file descriptors are an emulation implemented by the Universal CRT and
you can see their implementation in your copy of the Windows SDK. It's quite
possible that this doesn't happen in older CRT versions either since colorama
doesn't seem to break on Python 2.7 for example.
One way I can think working around this is that Python will DuplicateHandle the
console handles so that even if dup2 closes the original ones it will keep on
working. We can also switch to calling _get_osfhandle always instead of caching
the handle, it will break when the fd is redirected to a non-console. But so
does _WindowsConsoleIO in general since it will try to continue writing to the
console despite the redirection, meaning that Python code doing redirection has
to handle sys.std* anyhow. Though I'm not sure about the performance of
constantly calling _get_osfhandle. And their yet might be other ways to solve
this.
Also see comment by eryksun https://bugs.python.org/msg294988
Solving this in CPython will remove the need for hacks like the PR I referenced.
----------
components: IO, Windows
messages: 295041
nosy: Segev Finer, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: _io._WindowsConsoleIO breaks in the face of fd redirection
type: behavior
versions: Python 3.6, Python 3.7
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue30555>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
