[issue41697] Heap buffer overflow in the parser

Change by Guido van Rossum : -- stage: resolved -> ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://m

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: That's true. Is a bit sad because a considerable amount of problems we experienced with the new parser were due to invalid casts from these structures :( -- ___ Python tracker

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: Hm, I don't believe that will work -- each node type (e.g. `expr_ty`, `mod_ty`) has its own enum for `kind` (e.g. `_expr_kind`, `_mod_kind`) and some don't have a `kind` field at all (e.g. `keyword_ty`). So you'd have to add an extra field to each node type a

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: Given that using asdl_seq currently means casting from void*, we could maybe have a set of macros like asdl_seq_GET that are type specialized (there aren't many of them) and in debug mode they can check that the ->kind attribute is consistent with the

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: FWIW the bug was a classic type error -- in a decent language the asdl_seq type would have been generic and this would have been caught without an ASAN builder... -- ___ Python tracker

[issue41697] Heap buffer overflow in the parser

Brad Larsen added the comment: Nice work with the quick fix! I'm also happy to see the addition of the Linux ASAN builder -- that should help detect memory errors earlier on in the future. -- ___ Python tracker

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: Thanks for the report Brad, and thanks for the quick fix Pablo! -- ___ Python tracker ___ ___ P

[issue41697] Heap buffer overflow in the parser

Change by Pablo Galindo Salgado : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: New changeset be17295280c89771c80f317da072f6c0d016cc60 by Pablo Galindo in branch '3.9': [3.9] bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (GH-22077) (GH-22079) https://github.com/python/cpython/commit/be17295280c

[issue41697] Heap buffer overflow in the parser

Change by Pablo Galindo Salgado : -- pull_requests: +21166 pull_request: https://github.com/python/cpython/pull/22079 ___ Python tracker ___ ___

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: New changeset 315a61f7a9418d904e0eea14b1f054fac3a90e9f by Pablo Galindo in branch 'master': bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (GH-22077) https://github.com/python/cpython/commit/315a61f7a9418d904e0eea14b

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: > return _Py_Call(_PyPegen_dummy_name(p), args, keywords, EXTRA_EXPR(first, > last->element)); Actually, this is not enough because last->element may be a keyword_ty. I have updated PR 22077 to receive the EXTRA macro in the call because that simplif

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: I have added a new builbot builder to detect these problems in the future more immediately: https://buildbot.python.org/all/#/builders/582 For example, building the current master: https://buildbot.python.org/all/#/builders/582/builds/1 And with PR

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: The reason this does not manifest itself without the address sanitizer is because that information is thrown away later, and the line and col numbers for the Call node end being correct. -- ___ Python track

[issue41697] Heap buffer overflow in the parser

Change by Pablo Galindo Salgado : -- keywords: +patch pull_requests: +21164 stage: -> patch review pull_request: https://github.com/python/cpython/pull/22077 ___ Python tracker __

[issue41697] Heap buffer overflow in the parser

Pablo Galindo Salgado added the comment: I think I have the problem. The problem is that the asdl_seq that is b is filled with KeywordOrStarred* not with expr_ty. We need to do: KeywordOrStarred* last = asdl_seq_GET(b, asdl_seq_LEN(b)-1); return _Py_Call(_PyPegen_dummy_name(p), args, keywor

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: Smaller repro: ./python.exe -m configparser Bisection shows that this is the call that causes the crash (line 1301): return _impl(self._name, option, raw=raw, vars=vars, fallback=fallback, **kwargs) But just that line doesn't

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: Adding 3.9 since the offending commit was backported there. -- versions: +Python 3.9 ___ Python tracker ___ ___

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: Looking at it in lldb, 'last' seems to be bogus. kind = 6961392 ... lineno = -33686019 col_offset = -33686019 end_lineno = 0 end_col_offset = 0 Here I'm stuck. -- ___ Python tracker

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: Hm, it's not in sysconfig.py, it's in the next step: ./python.exe -E ./setup.py build -- ___ Python tracker ___ _

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: I do have a repro of the crash (with clang 11 no less). But it could be anywhere in sysconfig.py or anything it imports... -- ___ Python tracker _

[issue41697] Heap buffer overflow in the parser

Guido van Rossum added the comment: So that's the last line of the new helper function (in fact the end of the file). Maybe args can still be empty at this point? -- assignee: -> pablogsal priority: normal -> release blocker ___ Python tracker

[issue41697] Heap buffer overflow in the parser

New submission from Brad Larsen : It looks like commit 4a97b1517a6b5ff22e2984b677a680b07ff0ce11 introduced a heap buffer overflow: commit 4a97b1517a6b5ff22e2984b677a680b07ff0ce11 (HEAD -> master, origin/master, origin/HEAD) Author: Pablo Galindo Date: Wed Sep 2 17:44:19 2020 +0