[issue40335] Regression in multiline SyntaxError offsets

[Anthony Sottile]

Anthony Sottile  added the comment:

cool, reported there as well!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> pyflakes's testsuite has many failures under the new parser

Can you report this also on the PEP 617 issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Anthony Sottile]

Anthony Sottile  added the comment:

pyflakes's testsuite has many failures under the new parser -- is the 
expectation that those will be fixed by 3.9 final?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

And the regression happens because we are ignoring that test currently due to 
the new parser not currently reporting the same offsets:

https://github.com/python/cpython/blob/master/Lib/test/test_exceptions.py#L181

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

With the old parser it works:

~/github/python/master master*
❯ ./python -X oldparser t.py
3.9.0a5+ (heads/master:503de7149d, Apr 24 2020, 22:02:28)
[GCC 9.3.0]

SyntaxError:
- line: 8
- offset: 8
- text: "'''\n\ndef bar():\npass\n\ndef baz():\n'''quux'''\n"


~/github/python/master master*
❯ ./python  t.py
3.9.0a5+ (heads/master:503de7149d, Apr 24 2020, 22:02:28)
[GCC 9.3.0]

SyntaxError:
- line: 8
- offset: 52
- text: "'''"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Yeah, I can confirm after bisecting that the commit that introduced the 
regression is the new parser:

c5fc15685202cda73f7c3f5c6f299b0945f58508 is the first bad commit
commit c5fc15685202cda73f7c3f5c6f299b0945f58508
Author: Pablo Galindo 
Date:   Wed Apr 22 23:29:27 2020 +0100

bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)

Co-authored-by: Guido van Rossum 
Co-authored-by: Lysandros Nikolaou 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg367225

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Hu.actually, no. This is something else...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This is likely due to the new parser (see PEP 617). Do you get the same problem 
running with -X oldparser ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Anthony Sottile]

Anthony Sottile  added the comment:

This seems to have regressed again

$ ./python --version --version
Python 3.9.0a5+ (heads/master:503de7149d, Apr 24 2020, 13:34:49) 
[GCC 7.5.0]
$ ./python t.py
  File "/home/asottile/workspace/cpython/t.py", line 8

^
SyntaxError: invalid string prefix

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset 11a7f158ef51b0edcde3c3d9215172354e385877 by Pablo Galindo in 
branch 'master':
bpo-40335: Correctly handle multi-line strings in tokenize error scenarios 
(GH-19619)
https://github.com/python/cpython/commit/11a7f158ef51b0edcde3c3d9215172354e385877


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +18949
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/19619

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The regression happened in:

commit 41d5b94af44e34ac05d4cd57460ed104ccf96628
Author: Lysandros Nikolaou 
Date:   Sun Apr 12 21:21:00 2020 +0300

bpo-40246: Report a better error message for invalid string prefixes 
(GH-19476)

Will try to get a look at this shortly

--
nosy: +lys.nikolaou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[gaborbernat]

Change by gaborbernat :


--
nosy: +gaborbernat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +BTaskaya

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40335] Regression in multiline SyntaxError offsets

[Anthony Sottile]

New submission from Anthony Sottile :

this was noticed in pyflakes's testsuite: 
https://github.com/PyCQA/pyflakes/pull/532#pullrequestreview-396059622

I've created a small script to reproduce the problem

```
import sys

SRC = b"""\
def foo():
'''

def bar():
pass

def baz():
'''quux'''
"""

try:
exec(SRC)
except SyntaxError as e:
print(
f'{sys.version}\n\n'
f'{type(e).__name__}:\n'
f'- line: {e.lineno}\n'
f'- offset: {e.offset}\n'
f'- text: {e.text!r}\n'
)
```

This was "fixed" in python3.8, but has regressed:

```
$ python3.6 t2.py
3.6.9 (default, Nov  7 2019, 10:44:02) 
[GCC 8.3.0]

SyntaxError:
- line: 8
- offset: 52
- text: "'''\n\ndef bar():\npass\n\ndef baz():\n'''quux'''\n"

$ python3.8 t2.py
3.8.2 (default, Feb 26 2020, 02:56:10) 
[GCC 7.4.0]

SyntaxError:
- line: 8
- offset: 8
- text: "'''\n\ndef bar():\npass\n\ndef baz():\n'''quux'''\n"

$ ./python t2.py
3.9.0a5+ (heads/master:3955da8568, Apr 19 2020, 14:29:48) 
[GCC 7.5.0]

SyntaxError:
- line: 8
- offset: 52
- text: "'''\n\ndef bar():\npass\n\ndef baz():\n'''quux'''\n"

```

--
components: Interpreter Core
messages: 366805
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: Regression in multiline SyntaxError offsets
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com