[issue20115] NUL bytes in commented lines

Guido van Rossum added the comment: Serhiy’s comment from 2014-01-04 gives the answer. It’s different reading from a file than from a string. And only “python x.py” still reads from a file. -- ___ Python tracker

[issue20115] NUL bytes in commented lines

Terry J. Reedy added the comment: What I missed before is that duplicating the effect of the first two interactive entries (no exception) requires escaping the backslash so that the source argument for the explicit compile does not have a null. compile("'\\0'", '', 'exec') at 0x0214431

[issue20115] NUL bytes in commented lines

Guido van Rossum added the comment: Which part puzzles you? I see that you tried >>> #\0 This does not contain a null byte, just three characters: a hash, a backslash, and a digit zero. -- ___ Python tracker

[issue20115] NUL bytes in commented lines

Terry J. Reedy added the comment: The compile() doc currently says ""This function raises SyntaxError if the compiled source is invalid, and ValueError if the source contains null bytes." And indeed, in repository 3.9, 3.10, 3.11, >>> compile('\0','','exec') Traceback (most recent call last

[issue20115] NUL bytes in commented lines

Irit Katriel added the comment: See also issue1105770. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue20115] NUL bytes in commented lines

Guido van Rossum added the comment: Yeah, null bytes should just be rejected. If someone comes up with a fix for this we'll accept it. -- ___ Python tracker ___ _

[issue20115] NUL bytes in commented lines

Terry J. Reedy added the comment: https://docs.python.org/3/reference/toplevel_components.html#file-input says that file input and exec input (should) have the same grammar. This implies that the divergence is a bug. -- nosy: +gvanrossum ___ Python

[issue20115] NUL bytes in commented lines

Irit Katriel added the comment: This is still the same in 3.11. I added another line to the example's file, which shows more clearly what's happening: >>> open('x.py', 'wb').write(b'#\x00\na\nb\n') % ./python.exe x.py Traceback (most recent call last): File "x.py", line 2, in a NameEr

[issue20115] NUL bytes in commented lines

Changes by Jakub Wilk : -- nosy: +jwilk ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org

[issue20115] NUL bytes in commented lines

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue20115] NUL bytes in commented lines

Changes by Alex Gaynor : -- nosy: +alex ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org

[issue20115] NUL bytes in commented lines

ita1024 added the comment: Do not touch that please The null bytes are already rejected when forbidden by the encoding (utf-8 for example). Null byte characters in comments are perfectly valid in ISO8859-1 encoding, and a few scripts depend on them: http://ftp.waf.io/pub/release/waf-1.7.1

[issue20115] NUL bytes in commented lines

Serhiy Storchaka added the comment: See also issue13617. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue20115] NUL bytes in commented lines

Serhiy Storchaka added the comment: I'll try, but I'm not sure this is possible. Some used C functions (e.g. fgets()) returns char* and doesn't work with string containing null bytes. Some public API (e.g. PyParser_SimpleParseString()) work with null-terminated C strings. --

[issue20115] NUL bytes in commented lines

Georg Brandl added the comment: I'm in favor of PyPy's behavior: null bytes anywhere in the source, even in comments, usually mean there's something weird or fishy going on with either the editor or (if downloaded/copied) the source of the code. -- nosy: +georg.brandl

[issue20115] NUL bytes in commented lines

Armin Rigo added the comment: PyPy 2.x accepts null characters in all of import, exec and eval, and complains if they occur in non-comment. PyPy 3.x refuses them in import, which is where this bug report originally comes from (someone complained that CPython 3.x "accepts" them but not PyPy 3.

[issue20115] NUL bytes in commented lines

Terry J. Reedy added the comment: Armin, what is the different behavior of PyPy? We should perhaps get Guido's opinion on this issue. -- ___ Python tracker ___ _

[issue20115] NUL bytes in commented lines

Terry J. Reedy added the comment: Python should have a uniform definition of 'Python source' in both the doc and in practice in all source code processing functions. Currently, "2. Lexical analysis" in the Language Manual just says "Python reads program text as Unicode code points; the encodin

[issue20115] NUL bytes in commented lines

Armin Rigo added the comment: Fwiw, both exec and eval() ban NUL bytes, which means that there is a strange case in which some files can be imported, but not loaded and exec'ed. So I agree with Benjamin. -- ___ Python tracker

[issue20115] NUL bytes in commented lines

Benjamin Peterson added the comment: I guess NULL bytes should just be banned. -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue20115] NUL bytes in commented lines

Serhiy Storchaka added the comment: Indeed. CPython parser reads first line '#\x00\n' and save it in the buffer. But because C strings are used here (result of decode_str()), the line is truncated to '#'. As far as this data is not ended by '\n', it considered incomplete and next line is read

[issue20115] NUL bytes in commented lines

Changes by Antoine Pitrou : -- stage: -> needs patch type: compile error -> behavior versions: +Python 3.3, Python 3.4 ___ Python tracker ___ ___

[issue20115] NUL bytes in commented lines

Changes by STINNER Victor : -- nosy: +benjamin.peterson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:/

[issue20115] NUL bytes in commented lines

New submission from Armin Rigo: This is probably the smallest example of a .py file that behaves differently in CPython vs PyPy, and for once, I'd argue that the CPython behavior is unexpected: # make the file: >>> open('x.py', 'wb').write('#\x00\na') # run it: python x.py Expect