[issue46502] Py_CompileString no longer allows to tell "incomplete input" from "invalid input"
Mateusz Loskot added the comment: Possibly, the new partial-input mode of the parser (https://bugs.python.org/issue46521#msg412832) will improve this issue in 3.11. I'm going to play with it soon and I will report back. -- ___ Python tracker <https://bugs.python.org/issue46502> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46502] Py_CompileString no longer allows to tell "incomplete input" from "invalid input"
Mateusz Loskot added the comment: Not quite the answer I'd like to received, but thank you very much for the explanation. I've submitted pull request removing the deprecated FAQ entry https://github.com/python/cpython/pull/30925 -- ___ Python tracker <https://bugs.python.org/issue46502> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46502] Py_CompileString no longer allows to tell "incomplete input" from "invalid input"
Mateusz Loskot added the comment: Irit, I can and I will submit a patch. However, I'm very keen to learn what is an alternative solution then to distinguish hard invalid from incomplete input. IOW, what would be the new way of achieving what's described in the old FAQ? I believe it would be more useful to the community if I updated the sample in the FAQ instead of just removing it. -- ___ Python tracker <https://bugs.python.org/issue46502> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46502] Py_CompileString no longer allows to tell "incomplete input" from "invalid input"
New submission from Mateusz Loskot : Something has changed in Python 3.7 through 3.10 (I'm observing it in 3.10) in behaviour of the Python C API function Py_CompileString such that for an incomplete input it no longer raises SyntaxError: "unexpected EOF while parsing" but IndentationError: expected an indented block after ... The new behaviour makes the sample program from the "How do I tell “incomplete input” from “invalid input”?" at https://docs.python.org/3/faq/extending.html no longer work as described there. For example: ``` for i in []: ``` raises IndentationError: expected an indented block after 'for' statement on line 1 ``` if True: ``` raises IndentationError: expected an indented block after 'if' statement on line 1 instead of SyntaxError: unexpected EOF while parsing This effectively makes it impossible to detect incomplete input using the Py_CompileString in applications where it is not possible to use PyRun_InteractiveLoop. I have failed to identify what could be related changes in the release notes and the documentation does not seem to offer any update on that. So, I'm assuming the new behaviour is not desired or expected. Attached, is the VS 2022 screenshot with debugging session of the sample program from the FAQ presenting the difference in the behaviour between Python 3.6 and 3.10 on Windows. -- components: C API files: Py_CompileString-Py36-vs-Py310.png messages: 411484 nosy: mloskot priority: normal severity: normal status: open title: Py_CompileString no longer allows to tell "incomplete input" from "invalid input" versions: Python 3.10 Added file: https://bugs.python.org/file50582/Py_CompileString-Py36-vs-Py310.png ___ Python tracker <https://bugs.python.org/issue46502> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)
Change by Mateusz Loskot : -- nosy: +mloskot ___ Python tracker <https://bugs.python.org/issue39511> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14597] Cannot unload dll in ctypes until script exits
Changes by Mateusz Loskot <mate...@loskot.net>: -- nosy: +mloskot ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue14597> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: Re msg238016, I confirm python-3.5.0a2-fdvalidation.patch fixes the problem for Python 3.5.0a4 and VS2013. The only issue I encountered was with HAVE_FSTAT which is missing from PC/pyconifg.h, so I edited the patch and removed the check if defined(HAVE_FSTAT). Does PC/pyconifg.h need update? I also confirm vanilla Python 3.5.0a4 with VS2015RC does not require this patch, because the fileno regression/bug has been fixed in VS2015RC (details at https://connect.microsoft.com/VisualStudio/feedback/details/785119/). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: FYI, I've got it confirmed fix for the bug in VS has been released [1] We have fixed this behavior for the next major release, Visual Studio 14. The fix is present in the Visual Studio 14 CTP that was released earlier this month. [1] https://connect.microsoft.com/VisualStudio/feedback/details/785119/fileno-stdin-0-for-non-console-application#tabs -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21452] make_buildinfo.exe with VS2013 fails due ill-formed IntDir path
New submission from Mateusz Loskot: While building Python 3.2 or Python 3.4 with Visual Studio 2013 on Windows 8.1, pythoncore.vcxproj fails to build due to illformed pre-link event command. (FYI, I have upgraded all .vcxproj files to VS2013 locally.) Here is the command and the error: pre 1PreLinkEvent: 1 Description: Generate build information... 1 cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -D_DEBUG -MDd ..\Modules\getbuildinfo.c -FoD:\tmp\Python-3.4.0-vs2013\PCbuild\Win32-temp-Debug\pythoncore \getbuildinfo.o -I..\Include -I..\PC 1 getbuildinfo.c 1..\Modules\getbuildinfo.c(1): fatal error C1083: Cannot open include file: 'Python.h': No such file or directory /pre Notice the superfluous double quote followed by whitespace in the following argument: pre -FoD:\tmp\Python-3.4.0-vs2013\PCbuild\Win32-temp-Debug\pythoncore \getbuildinfo.o /pre The problem is somewhat known as there is related comment in the make_buildinfo.c file about the custom commands arguments parsing: pre /* Hack fix for bad command line: If the command is issued like this: * $(SolutionDir)make_buildinfo.exe Debug $(IntDir) * we will get a trailing quote because IntDir ends with a backslash that then * escapes the final . To simplify the life for developers, catch that problem * here by cutting it off. * The proper command line, btw is: * $(SolutionDir)make_buildinfo.exe Debug $(IntDir)\ * Hooray for command line parsing on windows. */ /pre There are two solutions possible: 1) Correct the PreLinkEvent command in make_buildinfo.vcxproj according to the comment above: pre $(SolutionDir)make_buildinfo.exe Debug $(IntDir)\ /pre 2) Patch make_buildinfo.c (attached). Then, the cl.exe command is correct: pre 1 cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL -D_DEBUG -MDd ..\Modules\getbuildinfo.c -FoF:\V81-VS2013.win32\Librarie s\External\Source\Python\Debug\PCbuild\Win32-temp-Debug\pythoncore\getbuildinfo.o -I..\Include -I..\PC 1 getbuildinfo.c /pre I'm not sure why this problems leaks while building with VS2013. I have been building Python 3.2 with VS2012 for long time w/o any issues. Perhaps, it is related to combination of VS2013 and Windows 8.1. Although, I'm not observing this problem while building with VS2012 on Windows 8.1, it would be helpful if someone else who uses VS2012 or VS2013 could confirm my results. -- components: Build files: Python340-make_buildinfo-vs2013.patch keywords: patch messages: 218097 nosy: mloskot priority: normal severity: normal status: open title: make_buildinfo.exe with VS2013 fails due ill-formed IntDir path type: compile error versions: Python 3.2, Python 3.4 Added file: http://bugs.python.org/file35183/Python340-make_buildinfo-vs2013.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21452 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment:
I have just tested Windows GUI application built against Python 3.2.1 with
is_valid_fd patch according to http://hg.python.org/cpython/rev/f15943505db0/
All built using VS2012.
Again, I can confirm that is_valid_fd does NOT solve the problem.
Here is extract of execution flow in initstdio function:
1. fd = 0, despite it is GUI app, see
https://connect.microsoft.com/VisualStudio/feedback/details/785119/
fd = fileno(stdin);
2. is_valid_fd will return __true__, so it moves to calling create_stdio()
if (!is_valid_fd(fd)) {
...
}
else {
std = create_stdio(iomod, fd, 0, stdin, encoding, errors);
if (std == NULL)
goto error;
}
3. The create_stdio() call fails though, causing error followed by abort
Still, the only solution that solves this problem in Windows GUI applications
buitl using VS2012 is to use check_fd() function to check fd, instead of
is_valid_fd(). The check_fd() is more reliable as it will return -1 due to
errno == EBADF.
My previous attempt to analyse _PyVerify_fd() is not relevant for the problem,
let's forget it.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17797
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: @Antoine (Re msg197480) I'm not sure which minor version of Python 3.2 it was, but in my comment in msg187362, I confirmed that I have tested the later version with added is_valid_fd function. As far as I understand, the changes in http://hg.python.org/cpython/rev/f15943505db0/ introduce that is_valid_fd function, which does not seem to solve the problem. -- status: pending - open ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: On 11 September 2013 12:34, Antoine Pitrou rep...@bugs.python.org wrote: I'm not sure which minor version of Python 3.2 it was, but in my comment in msg187362, I confirmed that I have tested the later version with added is_valid_fd function. As far as I understand, the changes in http://hg.python.org/cpython/rev/f15943505db0/ introduce that is_valid_fd function, which does not seem to solve the problem. Could you try to investigate this a bit further? Does is_valid_fd() return true, and why? I have already checked that (see msg187362) Shortly, is_valid_fd always returns true because fd 0 is always false as my tests with Visual Studio 2012 (Visual C++ 11.0 (1700) confirmed (see also bug report linked in msg188117) assert(fdi == 0); assert(fdo == 1); assert(fde == 2); IOW, fd is always larger than 0. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment:
On 12 September 2013 00:06, Antoine Pitrou rep...@bugs.python.org wrote:
Shortly, is_valid_fd always returns true because fd 0 is always
false as my tests with Visual Studio 2012
Well, that's not all there is, is_valid_fd() does other checks before
returning true.
Given the function:
is_valid_fd(int fd)
{
int dummy_fd;
if (fd 0 || !_PyVerify_fd(fd))
return 0;
dummy_fd = dup(fd);
if (dummy_fd 0)
return 0;
close(dummy_fd);
return 1;
}
for fd values of 0, 1 or 2
1. fd 0 is always false
2. _PyVerify_fd(fd) is always true. Given the current definition:
#define _PyVerify_fd(fd) (_get_osfhandle(fd) = 0)
for those values of fd _get_osfhandle(fd) = 0, always.
3. for those fd values, dup() never returns fd 0
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17797
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: This is still an issue in VS 2012 Version 11.0.60610.01 Update 3 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: I've just got an update on the bug report [1] I submitted to Microsoft. The Visual C++ team confirmed It does appear to be a regression from Visual Studio 2010. So, it's not a bug in Python, but I think it may be important for Python to consider applying some reliable workaround (i.e. use of check_fd() function) [1] http://connect.microsoft.com/VisualStudio/feedback/details/785119/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
New submission from Mateusz Loskot:
In pythonrun.c, there is function initstdio() with the following test:
static int
initstdio(void)
{
...
/* Set sys.stdin */
fd = fileno(stdin);
/* Under some conditions stdin, stdout and stderr may not be connected
* and fileno() may point to an invalid file descriptor. For example
* GUI apps don't have valid standard streams by default.
*/
if (fd 0) {
#ifdef MS_WINDOWS
std = Py_None;
Py_INCREF(std);
#else
goto error;
#endif
}
else {
...
}
This function is fails for non-console applications (i.e. MFC) built using
Visual C++ 11.0 (Visual Studio 2012), becasue **strangely**, fileno(stdin) ==
0, so this test results in false and Python initialisation routines attempt to
setup streams.
Apparently, fileno(stdin) return value has changed between Visual C++ 10.0 (VS
2010)
and 11.0. The VC++ 10.0 reports fileno(stdin) == -2.
--
components: IO, Interpreter Core
messages: 187351
nosy: mloskot
priority: normal
severity: normal
status: open
title: Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
versions: Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17797
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment:
Yes, it does. In file Modulfileio.c, in function fileio_init, there is this
code:
if (fd = 0) {
if (check_fd(fd))
goto error;
self-fd = fd;
self-closefd = closefd;
}
The check_fd tests:
if (!_PyVerify_fd(fd) || (fstat(fd, buf) 0 errno == EBADF)) {
The _PyVerify_fd(fd) == 1, but errno is Bad file descriptor.
This eventually leads to Py_InitializeEx failure at:
if (initstdio() 0)
Py_FatalError(
Py_Initialize: can't initialize sys standard streams);
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17797
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: In file Modulfileio.c, I messed the path and filename above I meant: In file Modules/_io/fileio.c, -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17797] Visual C++ 11.0 reports fileno(stdin) == 0 for non-console program
Mateusz Loskot added the comment: Replacing if the current test in Python 3.2 if (fd 0) with if (check_fd(fd) 0) Seems to be a working solution. I just noticed, that in current Python/pythonrun.c in the repo, there the fd 0 tests have been replaced with new function is_valid_fd(). But, its semantic is equivalent to fd 0, so it does not check anything really. Perhaps is_valid_fd could be redefined as check_fd. Here are Visual C++ 11.0 (1700) vs 10.0 differences of fileno return value for all the standard streams int fdi, fdo, fde; fdi = fileno(stdin); fdo = fileno(stdout); fde = fileno(stderr); #if _MSC_VER 1700 assert(fdi == -2); assert(fdo == -2); assert(fde == -2); #else assert(fdi == 0); assert(fdo == 1); assert(fde == 2); #endif By the way, I assume such sudden change in fileno(std*) behaviour between Visual C++ versions is suspicious, so I also submitted bug report to Visual Studio: https://connect.microsoft.com/VisualStudio/feedback/details/785119/ -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17797 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4442] document immutable type subclassing via __new__
Mateusz Loskot mate...@loskot.net added the comment: Chris Withers' note clarifies it to me, that this should be Python-level rather than C-level documentation. Then the note under __new__() in 3. Data model seems right. Simply, I expected to have some notes in C API too Side note: as mainly Python C API user, I expected to have it documented at C API level too. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4442 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4442] document immutable type subclassing via __new__
Changes by Mateusz Loskot mate...@loskot.net: -- nosy: +mloskot ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4442 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4442] document immutable type subclassing via __new__
Mateusz Loskot mate...@loskot.net added the comment: Is this report about documenting of the concept of immutable types in Python in general or regarding existing built-in types, like datetime.datetime? Generally, the concept of immutable type with relation to tp_new is mentioned (sneaked) here: 1) http://docs.python.org/release/3.2.2/c-api/typeobj.html A good rule of thumb is that for immutable types, all initialization should take place in tp_new, while for mutable types, most initialization should be deferred to tp_init. 2) http://www.python.org/dev/peps/pep-0253/ Note that for immutable object types, the initialization cannot be done by the tp_init() slot: this would provide the Python user with a way to change the initialization. Therefore, immutable objects typically have an empty tp_init() implementation and do all their initialization in their tp_new() slot. IMHO, it deserves a dedicated section/chapter in the docs. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue4442 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15029] Update Defining New Types chapter according to PEP 253
New submission from Mateusz Loskot mate...@loskot.net: The chapter '2. Defining New Types in the Python 3.2 documentation [1] does not cover all important elements, especially in the subsection 2.1.4. Subclassing other types. The accepted PEP 253 [2] provides much more detailed and thorough explanation for Python C API users willing to define and subclass new types, than the official manual. I'd like to suggest update of the manual with information enclosed in the PEP 253. In fact, the PEP's sections like * Preparing a type for subtyping * Creating a subtype of a built-in type in C could be copied with little editing, IMHO. The PEP 253 really seems to be a must-read document for Python C API users. [1] http://docs.python.org/release/3.2.2/extending/newtypes.html [2] http://www.python.org/dev/peps/pep-0253/ -- assignee: docs@python components: Documentation messages: 162480 nosy: docs@python, mloskot priority: normal severity: normal status: open title: Update Defining New Types chapter according to PEP 253 type: enhancement versions: Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15029 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15029] Update Defining New Types chapter according to PEP 253
Mateusz Loskot mate...@loskot.net added the comment: Similar request has been rejected in response to the Issue 621526 [1], but I'm not proposing to include PEP into docs, but to update and improve docs with material discussed in accepted PEPs. [1] http://bugs.python.org/issue621526 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15029 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue621526] docs do not include spec info from PEPs
Mateusz Loskot mate...@loskot.net added the comment: I reported issue 15029 [1] which may be related to this one. [1] http://bugs.python.org/issue15029 -- nosy: +mloskot ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue621526 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
