[issue43950] Include column offsets for bytecode instructions

Nikita Sobolev added the comment: I've opened a new issue about unused `dis.Positions` here: https://bugs.python.org/issue46422 -- nosy: +sobolevn ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +26588 pull_request: https://github.com/python/cpython/pull/28150 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Batuhan Taskaya added the comment: New changeset 85ea2d6165dec0cffa6302eb6dc40406eae1edf5 by Batuhan Taskaya in branch 'main': bpo-43950: support positions for dis.Instructions created through dis.Bytecode (GH-28142)

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +26581 pull_request: https://github.com/python/cpython/pull/28142 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

miss-islington added the comment: New changeset 3e235e0447e373d81f195f4292959c7be9c013dc by Batuhan Taskaya in branch 'main': bpo-43950: support some multi-line expressions for PEP 657 (GH-27339) https://github.com/python/cpython/commit/3e235e0447e373d81f195f4292959c7be9c013dc --

[issue43950] Include column offsets for bytecode instructions

Batuhan Taskaya added the comment: There is a subtle difference between traceback.py and Python/traceback.c which makes the latter (C version) output the line without trimming the trailing whitespace. The Python version simply uses `.strip()` which strips both the left (starting) and right

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25882 pull_request: https://github.com/python/cpython/pull/27339 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Batuhan Taskaya added the comment: New changeset 4f5980a4f57dab68b9137304f58bd08891d43d5a by Batuhan Taskaya in branch 'main': bpo-43950: support long lines in traceback.py (GH-27336) https://github.com/python/cpython/commit/4f5980a4f57dab68b9137304f58bd08891d43d5a --

[issue43950] Include column offsets for bytecode instructions

Batuhan Taskaya added the comment: New changeset ef8b8535cb3cd705392af9b927d6ff336d98427d by Batuhan Taskaya in branch 'main': bpo-43950: check against the raw string, not the pyobject (GH-27337) https://github.com/python/cpython/commit/ef8b8535cb3cd705392af9b927d6ff336d98427d --

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25880 pull_request: https://github.com/python/cpython/pull/27337 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25879 pull_request: https://github.com/python/cpython/pull/27336 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset c8362314cce53a5b59da7523fbdfa00f122aa319 by Batuhan Taskaya in branch 'main': bpo-43950: ensure source_line is present when specialising the traceback (GH-27313)

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25858 pull_request: https://github.com/python/cpython/pull/27313 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Dennis Sweeney added the comment: bpo-44719 managed to make the `assert(source_line);` fail -- nosy: +Dennis Sweeney ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: Even if this doesn't cover all cases it seems that everyone else is going with this solution for what I can see, and currently this looks suboptimal on python because we get this wrong on the vast majority of cases so my suggestion is to try to fix

[issue43950] Include column offsets for bytecode instructions

Ammar Askar added the comment: Indeed, and the unicode_width package seems to implement the wcwidth algorithm: * https://github.com/unicode-rs/unicode-width/blob/master/src/tables.rs#L39-L48 * https://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c https://bugs.python.org/issue12568 is an issue from

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: They seem to be checking the unicode width as we are proposing here. Check the calls to unicode_width::UnicodeWidthChar::width: https://github.com/rust-lang/rust/blob/master/compiler/rustc_errors/src/emitter.rs#L729-L733 --

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: In bpo this looks bad, but test this small program yourself and check the error in your terminal: fn test(number: String) -> () { println!("Hello, world: {}", number); } fn main() { test("hellooo: 拾     some emojis"); } --

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: I don't know, seems that rust can deal with this just fine: ❯ cargo run Compiling rtest v0.1.0 (/home/pablogsal/rtest) error[E0308]: mismatched types --> src/main.rs:7:10 | 7 | test("hellooo: 拾     some emojis"); |

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset fbc349ff790c21f1a59af939d42033470790c530 by Batuhan Taskaya in branch 'main': bpo-43950: Distinguish errors happening on character offset decoding (GH-27217)

[issue43950] Include column offsets for bytecode instructions

Ammar Askar added the comment: I think this is the previous issue you are talking about Terry. https://bugs.python.org/issue24665 The correct algorithm is a little more complex than just using east_asian_widths. Ideally we would like to use the wcwidth function

[issue43950] Include column offsets for bytecode instructions

Terry J. Reedy added the comment: The effort to match caret lines to general unicode is similar to a previous issue that was closed as futile. (But I could not find it.) It has a downside that should be considered. The fundamental problem is that there is no fixed pitch font for unicode.

[issue43950] Include column offsets for bytecode instructions

Ammar Askar added the comment: Had some time to look into this. Just to summarize this problem, it deals with unicode points that are single characters but take up more than the width of a single character, even with a monospace font [1]. In the examples from above, the Chinese character

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25758 pull_request: https://github.com/python/cpython/pull/27217 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: We can probably leverage the unicodedata_UCD_east_asian_width_impl function in the unicodedata extension module to get the width of every character as emojis normally take 2 characters in the terminal. We could expose the necessary functions from

[issue43950] Include column offsets for bytecode instructions

Ammar Askar added the comment: Aah, I won't have time to investigate until Monday. I'll take a look then unless someone gets to it first :) -- ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: Ah, this is our friend _PyPegen_byte_offset_to_character_offset. I am glad we refactor it to be used by the Parser and the traceback mechanism. Ammar, would you like to take a look? -- ___ Python tracker

[issue43950] Include column offsets for bytecode instructions

Ammar Askar added the comment: As reported by Will McGugan on twitter: Printing may be wrong for some unicode characters like: Traceback (most recent call last): File "test.py", line 1, in x = ("该" / 0) + (1 / 2) ~~^ TypeError: unsupported operand type(s) for /: 'str' and

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset 919ad537510fdc2c750109e0bc4eceea234324b2 by Batuhan Taskaya in branch 'main': bpo-43950: make BinOp specializations more reliable (GH-27126) https://github.com/python/cpython/commit/919ad537510fdc2c750109e0bc4eceea234324b2 --

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25669 pull_request: https://github.com/python/cpython/pull/27126 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25664 pull_request: https://github.com/python/cpython/pull/27117 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset 9c3eaf88dc5d5bed80cc45936de06b7b3162bc6d by Ammar Askar in branch 'main': bpo-43950: Add documentation for PEP-657 (GH-27047) https://github.com/python/cpython/commit/9c3eaf88dc5d5bed80cc45936de06b7b3162bc6d --

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset 1890dd235f618d60c938f6904d2e1a8a56f99c1c by Batuhan Taskaya in branch 'main': bpo-43950: Specialize tracebacks for subscripts/binary ops (GH-27037) https://github.com/python/cpython/commit/1890dd235f618d60c938f6904d2e1a8a56f99c1c

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset 4823d9a51281ebbc8e8d82a0dd3edc7d13ea8ac7 by Ammar Askar in branch 'main': bpo-43950: Add option to opt-out of PEP-657 (GH-27023) https://github.com/python/cpython/commit/4823d9a51281ebbc8e8d82a0dd3edc7d13ea8ac7 --

[issue43950] Include column offsets for bytecode instructions

Change by Ammar Askar : -- pull_requests: +25603 pull_request: https://github.com/python/cpython/pull/27047 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25597 pull_request: https://github.com/python/cpython/pull/27037 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset 5644c7b3ffd49bed58dc095be6e6148e0bb4431e by Ammar Askar in branch 'main': bpo-43950: Print columns in tracebacks (PEP 657) (GH-26958) https://github.com/python/cpython/commit/5644c7b3ffd49bed58dc095be6e6148e0bb4431e --

[issue43950] Include column offsets for bytecode instructions

Change by Ammar Askar : -- pull_requests: +25582, 25583 pull_request: https://github.com/python/cpython/pull/27023 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Ammar Askar : -- pull_requests: +25582, 25583, 25584 pull_request: https://github.com/python/cpython/pull/27023 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Ammar Askar : -- pull_requests: +25582 pull_request: https://github.com/python/cpython/pull/27023 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

miss-islington added the comment: New changeset 693cec0e2dcafa393ed5cdaa606f8dc8e3876adf by Batuhan Taskaya in branch 'main': bpo-43950: include position in dis.Instruction (GH-27015) https://github.com/python/cpython/commit/693cec0e2dcafa393ed5cdaa606f8dc8e3876adf --

[issue43950] Include column offsets for bytecode instructions

miss-islington added the comment: New changeset 44f91fc802a2b71312e9abd9e8e9dcbf02e8216d by Batuhan Taskaya in branch 'main': bpo-43950: use 0-indexed column offsets for bytecode positions (GH-27011) https://github.com/python/cpython/commit/44f91fc802a2b71312e9abd9e8e9dcbf02e8216d

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25574 pull_request: https://github.com/python/cpython/pull/27015 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25571 pull_request: https://github.com/python/cpython/pull/27011 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

miss-islington added the comment: New changeset ec8759b060eff83ff466f42c5a96d83a685016ce by Batuhan Taskaya in branch 'main': bpo-43950: optimize column table assembling with pre-sizing object (GH-26997) https://github.com/python/cpython/commit/ec8759b060eff83ff466f42c5a96d83a685016ce

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- pull_requests: +25557 pull_request: https://github.com/python/cpython/pull/26997 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: New changeset 98eee94421dcb42c15f2d7fc4cd21357722fbe2a by Pablo Galindo in branch 'main': bpo-43950: Add code.co_positions (PEP 657) (GH-26955) https://github.com/python/cpython/commit/98eee94421dcb42c15f2d7fc4cd21357722fbe2a --

[issue43950] Include column offsets for bytecode instructions

Change by Ammar Askar : -- nosy: +ammar2 nosy_count: 8.0 -> 9.0 pull_requests: +25524 pull_request: https://github.com/python/cpython/pull/26958 ___ Python tracker ___

[issue43950] Include column offsets for bytecode instructions

Change by Andre Roberge : -- nosy: +aroberge ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43950] Include column offsets for bytecode instructions

Change by Pablo Galindo Salgado : -- keywords: +patch pull_requests: +25521 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26955 ___ Python tracker

[issue43950] Include column offsets for bytecode instructions

Change by Batuhan Taskaya : -- nosy: +BTaskaya ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: > So would marking the 0 denominator when there is more than one candidate: "e > = a/b + c/d". No, it will mark the offset of the bytecode that was getting executed when the exception was raised. Is just a way to mark what is raising the particular

[issue43950] Include column offsets for bytecode instructions

Terry J. Reedy added the comment: Marking what expression evaluated to None would be extremely helpful. So would marking the 0 denominator when there is more than one candidate: "e = a/b + c/d". It should be easy to revise IDLE Shell's print_exception to tag the span. In some cases, the

[issue43950] Include column offsets for bytecode instructions

Change by Brandt Bucher : -- nosy: +brandtbucher ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: Basically, to highlight in all exceptions the range in the displayed line where the error ocurred. For instance: >>> foo(a, b/z+2, c, 132432 /x, d /y) ^ Traceback (most recent call last): File "", line 1, in

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: > Specific examples of current messages and proposed improvements would help > focus discussion. Yeah, I am proposing going from: >>> x['aa']['bb']['cc']['dd'].sddfsdf.sdfsdf Traceback (most recent call last): File "", line 1, in

[issue43950] Include column offsets for bytecode instructions

Terry J. Reedy added the comment: Specific examples of current messages and proposed improvements would help focus discussion. If you are willing to only handle code lines up to 256 chars, only 2 bytes should be needed. (0,0) or (255,255) could mean 'somewhere beyond the 256th char'.

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: Yup, but I still think is worth the cost, giving that debugging improvements are usually extremely popular among users. -- ___ Python tracker

[issue43950] Include column offsets for bytecode instructions

Mark Shannon added the comment: The additional cost will not only be the line number table, but we need to store the line for exceptions that are reraised after cleanup. Adding a column will mean more stack consumption. -- ___ Python tracker

[issue43950] Include column offsets for bytecode instructions

Pablo Galindo Salgado added the comment: I'm going to prepare a PEP since the discussion regarding if the two integers per bytecode are worth enough is going to be eternal. -- ___ Python tracker

[issue43950] Include column offsets for bytecode instructions

Change by Pablo Galindo Salgado : -- nosy: +nedbat, serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue43950] Include column offsets for bytecode instructions

New submission from Pablo Galindo Salgado : If we could include column offsets from the AST nodes for every bytecode instructions we could leverage these to offer much better tracebacks and a lot more information to debuggers and similar tools. For instance, in an expression such as: