[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[alessandro mantovani]

alessandro mantovani  added the comment:

Fuzzing experimental techniques, but then I observed the same behavior was 
happening with vanilla afl++. As a starting queue I used the *.py files that I 
found in the repo under ‘test’ or so 

Best 

Alessandro Mantovani 

Inviato da iPhone

> Il giorno 12.06.2021, alle ore 19:57, Pablo Galindo Salgado 
>  ha scritto:
> 
> 
> Pablo Galindo Salgado  added the comment:
> 
> alessandro mantovani, one question, how did you generate the crash scripts?
> 
> --
> 
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset d03f342a8389f1ea9100efb0d1a205601e607254 by Miss Islington (bot) 
in branch '3.10':
bpo-44396: Update multi-line-start location when reallocating tokenizer buffers 
(GH-26676) (GH-26695)
https://github.com/python/cpython/commit/d03f342a8389f1ea9100efb0d1a205601e607254


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

alessandro mantovani, one question, how did you generate the crash scripts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[miss-islington]

miss-islington  added the comment:


New changeset a342cc5891dbd8a08d40e9444f2e2c9e93258721 by Pablo Galindo in 
branch 'main':
bpo-44396: Update multi-line-start location when reallocating tokenizer buffers 
(GH-26676)
https://github.com/python/cpython/commit/a342cc5891dbd8a08d40e9444f2e2c9e93258721


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[miss-islington]

Change by miss-islington :


--
pull_requests: +25280
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26695

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Ok, found the problem, we are not resetting the multi-line-start pointer when 
we are reallocating the tokenizer buffers.

--
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +25262
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26676

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This affects 3.10 as well

--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I think this should fix the issue, but someone should validate this:

diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c
index 6002f3e05a..1c28737183 100644
--- a/Parser/tokenizer.c
+++ b/Parser/tokenizer.c
@@ -1084,17 +1084,16 @@ syntaxerror(struct tok_state *tok, const char *format, 
...)
 goto error;
 }

-errtext = PyUnicode_DecodeUTF8(tok->line_start, tok->cur - tok->line_start,
+errtext = PyUnicode_DecodeUTF8(tok->buf, tok->inp - tok->buf,
"replace");
 if (!errtext) {
 goto error;
 }
 int offset = (int)PyUnicode_GET_LENGTH(errtext);
-Py_ssize_t line_len = strcspn(tok->line_start, "\n");
-if (line_len != tok->cur - tok->line_start) {
+Py_ssize_t line_len = strcspn(tok->buf, "\n");
+if (line_len != tok->buf - tok->inp) {
 Py_DECREF(errtext);
-errtext = PyUnicode_DecodeUTF8(tok->line_start, line_len,
-   "replace");
+errtext = PyUnicode_DecodeUTF8(tok->buf, line_len, "replace");
 }
 if (!errtext) {
 goto error;

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Here is a smaller reproducer:

x = "ijosdfsd\
def blech():
pass

This seems to be an error with:

commit a698d52c3975c80b45b139b2f08402ec514dce75
Author: Batuhan Taskaya 
Date:   Thu Jan 21 00:38:47 2021 +0300

bpo-40176: Improve error messages for unclosed string literals (GH-19346)



Automerge-Triggered-By: GH:isidentical

Batuhan, can you take a look?

--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[Guido van Rossum]

Guido van Rossum  added the comment:

Lysandros and Pablo, this *only* occurs when the lexer is reading directly from 
a file, not when it's reading the same source code from a (bytes) string. All 
examples are syntax errors (some raise ValueError in the parser).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44396] pegen _PyParser_ASTFromFile(): Use-After-Free in syntaxerror()

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +gvanrossum, lys.nikolaou, pablogsal
title: Use-After-Free -> pegen _PyParser_ASTFromFile(): Use-After-Free in 
syntaxerror()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com