[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-20 Thread Pablo Galindo Salgado
> Hopefully if we add more pseudokeywords in the future it won't break the ability to parse traceback spans. It won't because soft keywords are now handled natively with the peg parser (as "match" and "case" now) instead of hacked into the tokenizer :) On Fri, 21 May 2021 at 01:55, Nathaniel

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-20 Thread Nathaniel Smith
On Wed, May 19, 2021 at 7:28 PM Pablo Galindo Salgado wrote: >> >> Excellent point! Do you know how reliable this is in practice, i.e. >> what proportion of bytecode source spans are something you can >> successfully pass to ast.parse? If it works it's obviously nicer, but >> I can't tell how

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-19 Thread Pablo Galindo Salgado
> > Excellent point! Do you know how reliable this is in practice, i.e. > what proportion of bytecode source spans are something you can > successfully pass to ast.parse? If it works it's obviously nicer, but > I can't tell how often it works. E.g. anything including >

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-19 Thread Nathaniel Smith
On Tue, May 18, 2021 at 2:49 PM Pablo Galindo Salgado wrote: > * It actually doesn't have more advantages. The current solution in the PEP > can do exactly the same as this solution if you allow reparsing when > displaying tracebacks. This is because with the start line, end line, start >

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
Ok, we have implemented a rough prototype and we have decided not to go with this for the following reasons: * Is almost the same storage cost as the solution we already have. Since storing the node id cost 4 bytes (an integer) per instruction and our solution needs 2+ bytes per instruction for

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Larry Hastings
On 5/18/21 5:25 AM, Pablo Galindo Salgado wrote: Yet another problem that I found: One integer is actually not enough to assign IDs. One unsigned integer can cover 4,294,967,295 AST nodes, but is technically possibleto have more than that in a single file. Surely you could use a 64-bit int

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
> One integer is actually not enough to assign IDs. Actually, disregard this particular problem. I think that we could perfectly stop assigning IDs if we reach the overflow limit and call it a day since you need to have a truly horrendous file to reach 4,294,967,295 AST nodes (I did some tests to

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
Yet another problem that I found: One integer is actually not enough to assign IDs. One unsigned integer can cover 4,294,967,295 AST nodes, but is technically possible to have more than that in a single file. While in PEP 657 we are tracking offsets that are normally very low < 100 typically or

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
Hu, actually another problem of this approach: Nodes are created and modified after the optimization pass, so the AST produced by the parser is not enough to reconstruct the actual information, we need to also run the optimization passes, but unfortunately, this is (by design) not don't in

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-18 Thread Pablo Galindo Salgado
Hi Nathaniel, Thanks a lot for your suggestion! I like the idea although I still think is more complex than our current proposal, but on the other hand it allows for a much richer results so I'm quite excited to try it out. We are going to give it a go to explore it with a prototype and if we are

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Nathaniel Smith
On Mon, May 17, 2021 at 6:18 AM Mark Shannon wrote: > 2. Repeated binary operations on the same line. > > A single location can also be clearer when all the code is on one line. > > i1 + i2 + s1 > > PEP 657: > > i1 + i2 + s1 > > > Using a single location: > > i1 + i2 + s1 >

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Brandt Bucher
Ethan Furman wrote: > On 5/17/2021 6:13 AM, Mark Shannon wrote: > > Where i1, i2 are integers and s1 is a string. > > > i1 + i2 + s1 > > > Wouldn't the carets just be under the i2 + s1 portion? I don't think so, since this is executed as `((i1 + i2) + s1)`. Mark's carets look

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Ethan Furman
On 5/17/2021 6:13 AM, Mark Shannon wrote: > Where i1, i2 are integers and s1 is a string. > i1 + i2 + s1 > Wouldn't the carets just be under the i2 + s1 portion? -- ~Ethan~ ___ Python-Dev mailing list -- python-dev@python.org To

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Christopher Barker
> > The cost I'm concerned about is the runtime cost of worse code, because > > the compiler can't perform some optimizations due the constraints of > > providing the extended debug information. Python does have an Optimized mode (-O). Granted, it’s not used very often, but this would be a good

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Ammar Askar
> The cost I'm concerned about is the runtime cost of worse code, because > the compiler can't perform some optimizations due the constraints of > providing the extended debug information. Aah thanks for clarifying, I see what you mean now. In cases like this where the compiler is making

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Mark Shannon
Hi, On 17/05/2021 5:22 pm, Ammar Askar wrote: >> While nicer locations for errors is great, it won't be popular if it has >> a negative impact on performance. >> Locations need to tracked through the compiler. > > In performance sensitive contexts won't most code be pre-compiled into > pyc files

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Ammar Askar
> While nicer locations for errors is great, it won't be popular if it has > a negative impact on performance. > Locations need to tracked through the compiler. In performance sensitive contexts won't most code be pre-compiled into pyc files anyway? I feel like the performance cost of accurate

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Pablo Galindo Salgado
P.S. We will add "using a single caret" to the "rejected ideas section" with some rationale. On Mon, 17 May 2021, 14:28 Pablo Galindo Salgado, wrote: > Hi Mark, > > Thanks for your useful feedback. Some comments: > > > 1. Errors spanning multiple lines. > > That is addressed. Check the latest

[Python-Dev] Re: Critique of PEP 657 -- Include Fine Grained Error Locations in Tracebacks

2021-05-17 Thread Pablo Galindo Salgado
Hi Mark, Thanks for your useful feedback. Some comments: > 1. Errors spanning multiple lines. That is addressed. Check the latest version of the PEP: we are propising storing the end lines as well: https://www.python.org/dev/peps/pep-0657/#specification > 2. Repeated binary operations on