[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[miss-islington]

miss-islington  added the comment:


New changeset a427eb862f11888fa69fee520eb8a20bd396fcdb by Miss Islington (bot) 
in branch '3.10':
bpo-45494: Fix error location in EOF tokenizer errors (GH-29108)
https://github.com/python/cpython/commit/a427eb862f11888fa69fee520eb8a20bd396fcdb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +27911
pull_request: https://github.com/python/cpython/pull/29672

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 79ff0d1687e3f823fb121a19f0297ad052871b1b by Pablo Galindo Salgado 
in branch 'main':
bpo-45494: Fix error location in EOF tokenizer errors (GH-29108)
https://github.com/python/cpython/commit/79ff0d1687e3f823fb121a19f0297ad052871b1b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +27376
pull_request: https://github.com/python/cpython/pull/29108

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Łukasz Langa  added the comment:

Thanks for the fix, Pablo! ✨ 🍰 ✨

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 88f4ec88e282bf861f0af2d237e9fe28fbc8deac by Łukasz Langa in 
branch '3.9':
[3.9] bpo-45494: Fix parser crash when reporting errors involving invalid 
continuation characters (GH-28993) (#29071)
https://github.com/python/cpython/commit/88f4ec88e282bf861f0af2d237e9fe28fbc8deac


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Łukasz Langa  added the comment:

Note: this *does* fail on 3.9, too. Even if it doesn't crash the production 
build, it does fail an assertion in a pydebug build:


test_error_offset_continuation_characters (test.test_exceptions.ExceptionTests) 
... Assertion failed: (!_PyErr_Occurred(tstate)), function _PyObject_Call, file 
Objects/call.c, line 261.
Fatal Python error: Aborted

Current thread 0x0001184d1dc0 (most recent call first):
  File "/private/tmp/cpy/Lib/test/test_exceptions.py", line 187 in check
  File "/private/tmp/cpy/Lib/test/test_exceptions.py", line 198 in 
test_error_offset_continuation_characters

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 5c9cab595e56aeb118bff77ece784dbac30b4338 by Łukasz Langa in 
branch '3.10':
[3.10] bpo-45494: Fix parser crash when reporting errors involving invalid 
continuation characters (GH-28993) (GH-29070)
https://github.com/python/cpython/commit/5c9cab595e56aeb118bff77ece784dbac30b4338


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +27339
pull_request: https://github.com/python/cpython/pull/29071

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +27338
pull_request: https://github.com/python/cpython/pull/29070

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset a106343f632a99c8ebb0136fa140cf189b4a6a57 by Pablo Galindo Salgado 
in branch 'main':
bpo-45494: Fix parser crash when reporting errors involving invalid 
continuation characters (GH-28993)
https://github.com/python/cpython/commit/a106343f632a99c8ebb0136fa140cf189b4a6a57


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Gregory P. Smith]

Change by Gregory P. Smith :


--
priority: normal -> high
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> I confirmed that 3.9 does NOT seem to have the problem:


It does, is just that is not a crash. The point where the error message point 
is totally wrong

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I confirmed that 3.9 does NOT seem to have the problem:

Python 3.9.5 (default, May 19 2021, 11:32:47) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> x = r'''
... "\
... "(1for c in I,\
... \ '''
>>> import ast
>>> ast.literal_eval(x)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.9/ast.py", line 62, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
  File "/usr/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
  File "", line 3
"\
  ^
SyntaxError: Generator expression must be parenthesized

--
nosy: +gregory.p.smith
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Presto!! PR 28993

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +27276
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28993

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45494] [fuzzer] Parser null deref with continuation characters and generator parenthesis error

[Ammar Askar]

New submission from Ammar Askar :

Another parser crash found by the fuzzer:

"\
"(1for c in I,\
\

Recreator:

>>> import ast
>>> ast.literal_eval('"\\\n"(1for c in I,\\\n\\')
[1]17916 segmentation fault  ./python

>>> import ast
>>> ast.literal_eval(r'''
... "\
... "(1for c in I,\
... \ ''')
[1]17935 segmentation fault  ./python


---
Raw ASAN stacktrace
---

==1668==ERROR: AddressSanitizer: SEGV on unknown address 0x0001 
(pc 0x7f4157e5e08c bp 0x7fffbd48b300 sp 0x7fffbd48aab8 T0)
==1668==The signal is caused by a READ memory access.
==1668==Hint: address points to the zero page.
#0 0x7f4157e5e08c in strchr-avx2.S:57 
/build/glibc-eX1tMB/glibc-2.31/sysdeps/x86_64/multiarch/strchr-avx2.S:57
#1 0x4d7a88 in strchr 
/src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:0
#2 0x9fa6f5 in get_error_line cpython3/Parser/pegen.c:406:25
#3 0x9fa6f5 in _PyPegen_raise_error_known_location 
cpython3/Parser/pegen.c:497:26
#4 0xa18a92 in RAISE_ERROR_KNOWN_LOCATION cpython3/Parser/pegen.h:169:5
#5 0xa331d5 in invalid_arguments_rule cpython3/Parser/parser.c:17831:20
#6 0xa21a87 in arguments_rule cpython3/Parser/parser.c:15462:38
#7 0xa2056b in primary_raw cpython3/Parser/parser.c:12867:18
#8 0xa2056b in primary_rule cpython3/Parser/parser.c:12745:22
#9 0xa1f9cd in await_primary_rule cpython3/Parser/parser.c:12700:28
#10 0xa1f119 in power_rule cpython3/Parser/parser.c:12578:18
#11 0xa1eabc in factor_rule cpython3/Parser/parser.c:12530:26
#12 0xa1dc04 in term_raw cpython3/Parser/parser.c:12373:27
#13 0xa1dc04 in term_rule cpython3/Parser/parser.c:12138:22
#14 0xa1c899 in sum_raw cpython3/Parser/parser.c:12093:25
#15 0xa1c899 in sum_rule cpython3/Parser/parser.c:11975:22
#16 0xa1bb99 in shift_expr_raw cpython3/Parser/parser.c:11936:24
#17 0xa1bb99 in shift_expr_rule cpython3/Parser/parser.c:11818:22
#18 0xa1af2c in bitwise_and_raw cpython3/Parser/parser.c:11779:31
#19 0xa1af2c in bitwise_and_rule cpython3/Parser/parser.c:11700:22
#20 0xa1a49c in bitwise_xor_raw cpython3/Parser/parser.c:11661:32
#21 0xa1a49c in bitwise_xor_rule cpython3/Parser/parser.c:11582:22
#22 0xa1917c in bitwise_or_raw cpython3/Parser/parser.c:11543:32
#23 0xa1917c in bitwise_or_rule cpython3/Parser/parser.c:11464:22
#24 0xa2cd39 in comparison_rule cpython3/Parser/parser.c:10727:18
#25 0xa2c912 in inversion_rule cpython3/Parser/parser.c:10680:31
#26 0xa2b951 in conjunction_rule cpython3/Parser/parser.c:10559:18
#27 0xa258e1 in disjunction_rule cpython3/Parser/parser.c:10473:18
#28 0xa17cb1 in invalid_expression_rule cpython3/Parser/parser.c:18253:18
#29 0xa17cb1 in expression_rule cpython3/Parser/parser.c:9754:39
#30 0xa56979 in expressions_rule cpython3/Parser/parser.c:9628:18
#31 0xa0acf5 in eval_rule cpython3/Parser/parser.c:1035:18
#32 0xa0acf5 in _PyPegen_parse cpython3/Parser/parser.c:33076:18
#33 0xa001a5 in _PyPegen_run_parser cpython3/Parser/pegen.c:1350:9
#34 0xa01fa5 in _PyPegen_run_parser_from_string 
cpython3/Parser/pegen.c:1482:14
#35 0xa80fc9 in _PyParser_ASTFromString cpython3/Parser/peg_api.c:14:21
#36 0x8611ca in Py_CompileStringObject cpython3/Python/pythonrun.c:1371:11
#37 0xc04a8f in builtin_compile_impl cpython3/Python/bltinmodule.c:842:14
#38 0xc04a8f in builtin_compile 
cpython3/Python/clinic/bltinmodule.c.h:249:20
#39 0xb78ade in cfunction_vectorcall_FASTCALL_KEYWORDS 
cpython3/Objects/methodobject.c:446:24
#40 0x57c0ec in _PyObject_VectorcallTstate 
cpython3/Include/internal/pycore_call.h:89:11
#41 0x57c0ec in PyObject_Vectorcall cpython3/Objects/call.c:298:12
#42 0x766191 in call_function cpython3/Python/ceval.c:6619:13
#43 0x748137 in _PyEval_EvalFrameDefault cpython3/Python/ceval.c:4734:19
#44 0x741ae4 in _PyEval_EvalFrame 
cpython3/Include/internal/pycore_ceval.h:48:16
#45 0x741ae4 in _PyEval_Vector cpython3/Python/ceval.c:5810:24
#46 0x57cb50 in _PyFunction_Vectorcall cpython3/Objects/call.c:0
#47 0x57c0ec in _PyObject_VectorcallTstate 
cpython3/Include/internal/pycore_call.h:89:11
#48 0x57c0ec in PyObject_Vectorcall cpython3/Objects/call.c:298:12
#49 0x766191 in call_function cpython3/Python/ceval.c:6619:13
#50 0x748137 in _PyEval_EvalFrameDefault cpython3/Python/ceval.c:4734:19
#51 0x741ae4 in _PyEval_EvalFrame 
cpython3/Include/internal/pycore_ceval.h:48:16
#52 0x741ae4 in _PyEval_Vector cpython3/Python/ceval.c:5810:24
#53 0x57cb50 in _PyFunction_Vectorcall cpython3/Objects/call.c:0
#54 0x57c920 in _PyObject_VectorcallTstate 
cpython3/Include/internal/pycore_call.h:89:11
#55 0x57c920 in PyObject_CallOneArg cpython3/Objects/call.c:375:12
#56 0x579d18 in fuzz_ast_literal_eval 
cpython3/Modules/_xxtestfuzz/fuzzer.c:425:25
#57 0x579d18 in _run_fuzz cpython3/Modules/_xxtestfuzz/fuzzer.c:443:14
#58 0x579d1