[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Richard Damon
On 5/8/21 10:16 PM, Jim J. Jewett wrote: > Antoine Pitrou wrote: >> On Sat, 8 May 2021 02:58:40 + >> Neil Schemenauer nas-pyt...@arctrix.com wrote: >>> It would be cool if we could mmap the pyc files and have the VM run >>> code without an unmarshal step. >>> What happens if another process

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Jim J. Jewett
Antoine Pitrou wrote: > On Sat, 8 May 2021 02:58:40 + > Neil Schemenauer nas-pyt...@arctrix.com wrote: > > It would be cool if we could mmap the pyc files and have the VM run > > code without an unmarshal step. > > What happens if another process mutates or truncates the file while the >

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Steven D'Aprano
Hi Chris, On Fri, May 07, 2021 at 07:13:16PM -0700, Chris Jerdonek wrote: > I'm not sure why you're sounding so negative. Pablo asked for ideas in his > first message to the list: I know that Pablo asked for ideas, but that doesn't mean that we are obliged to agree with every idea. This is a

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Brett Cannon
On Sat, May 8, 2021 at 2:59 PM Gregory P. Smith wrote: > > > On Sat, May 8, 2021 at 2:09 PM Pablo Galindo Salgado > wrote: > >> > Why not put in it -O instead? Then -O means lose asserts and lose >> fine-grained tracebacks, while -OO continues to also >> strip out doc strings. >> >> What if

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
Thanks Greg for the great, detailed response I think I understand now better your proposal and I think is a good idea and I would like to explore that. I have some questions: * One problem I see is that that will make the constructor of the code object depend on global options in the

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 2:40 PM Jonathan Goble wrote: > On Sat, May 8, 2021 at 5:08 PM Pablo Galindo Salgado > wrote: > >> > Why not put in it -O instead? Then -O means lose asserts and lose >> fine-grained tracebacks, while -OO continues to also >> strip out doc strings. >> >> What if someone

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 2:09 PM Pablo Galindo Salgado wrote: > > Why not put in it -O instead? Then -O means lose asserts and lose > fine-grained tracebacks, while -OO continues to also > strip out doc strings. > > What if someone wants to keep asserts but do not want the extra data? > exactly

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Jonathan Goble
On Sat, May 8, 2021 at 5:08 PM Pablo Galindo Salgado wrote: > > Why not put in it -O instead? Then -O means lose asserts and lose > fine-grained tracebacks, while -OO continues to also > strip out doc strings. > > What if someone wants to keep asserts but do not want the extra data? > What if

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
> I don't think the optional existence of column number information needs a different kind of pyc file. Just a flag in a pyc file's header at most. It isn't a new type of file. Greg, what do you think if instead of not writing it to the pyc file with -OO or adding a header entry to decide to

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
> Why not put in it -O instead? Then -O means lose asserts and lose fine-grained tracebacks, while -OO continues to also strip out doc strings. What if someone wants to keep asserts but do not want the extra data? On Sat, 8 May 2021 at 22:05, Ethan Furman wrote: > On 5/8/21 1:31 PM, Pablo

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Ethan Furman
On 5/8/21 1:31 PM, Pablo Galindo Salgado wrote: >> We can't piggy back on -OO as the only way to disable this, it needs to >> have an option of its own. -OO is unusable as code that relies on "doc" >> strings as application data such as http://www.dabeaz.com/ply/ply.html >> exists. > > -OO is

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
> That could work, but in my personal opinion, I would prefer not to do that as it complicates things and I think is overkill. Let me expand on this: I recognize the problem that -OO can be quite unusable if some of your dependencies depend on docstrings and that It would be good to separate

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
> I don't think the optional existence of column number information needs a different kind of pyc file. Just a flag in a pyc file's header at most. It isn't a new type of file. That could work, but in my personal opinion, I would prefer not to do that as it complicates things and I think is

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 1:32 PM Pablo Galindo Salgado wrote: > > We can't piggy back on -OO as the only way to disable this, it needs to > have an option of its own. -OO is unusable as code that relies on > "doc"strings as application data such as > http://www.dabeaz.com/ply/ply.html exists. > >

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
> We can't piggy back on -OO as the only way to disable this, it needs to have an option of its own. -OO is unusable as code that relies on "doc"strings as application data such as http://www.dabeaz.com/ply/ply.html exists. -OO is the only sensible way to disable the data. There are two things

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 11:58 AM Pablo Galindo Salgado wrote: > Hi Brett, > > Just to be clear, .pyo files have not existed for a while: >> https://www.python.org/dev/peps/pep-0488/. > > > Whoops, my bad, I wanted to refer to the pyc files that are generated > with -OO, which have the "opt-2"

[Python-Dev] Re: a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Guido van Rossum
That’s a discussion for another day. On Sat, May 8, 2021 at 09:17 Thomas Grainger wrote: > Would it be possible to drop the requirement that multiple exception types > are parenthesized? Is it only ambiguous with the old Python2 syntax? > > On Sat, 8 May 2021, 20:15 Guido van Rossum, wrote: >

[Python-Dev] Re: a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Thomas Grainger
Would it be possible to drop the requirement that multiple exception types are parenthesized? Is it only ambiguous with the old Python2 syntax? On Sat, 8 May 2021, 20:15 Guido van Rossum, wrote: > I propose “exception tuple”, since syntactically and semantically it must > be a tuple. (Same as

[Python-Dev] Re: a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Guido van Rossum
I propose “exception tuple”, since syntactically and semantically it must be a tuple. (Same as for isinstance() and issubclass().) On Sat, May 8, 2021 at 05:52 Thomas Grainger wrote: > That's this bit: > > ``` > except (A, B): >^^ > ``` > > bpo-43149 currently calls it an

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Pablo Galindo Salgado
Hi Brett, Just to be clear, .pyo files have not existed for a while: > https://www.python.org/dev/peps/pep-0488/. Whoops, my bad, I wanted to refer to the pyc files that are generated with -OO, which have the "opt-2" prefix. This only kicks in at the -OO level. I will correct the PEP so it

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Brett Cannon
On Fri, May 7, 2021 at 7:31 PM Pablo Galindo Salgado wrote: > Although we were originally not sympathetic with it, we may need to offer > an opt-out mechanism for those users that care about the impact of the > overhead of the new data in pyc files > and in in-memory code objectsas was suggested

[Python-Dev] Re: a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Thomas Grainger
There's a PR to use "SyntaxError: multiple exception types must be parenthesized" https://github.com/python/cpython/pull/25996 On Sat, 8 May 2021, 19:20 Gregory P. Smith, wrote: > > > On Sat, May 8, 2021 at 8:54 AM Thomas Grainger wrote: > >> That's this bit: >> >> ``` >> except (A, B):

[Python-Dev] Re: a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 8:54 AM Thomas Grainger wrote: > That's this bit: > > ``` > except (A, B): >^^ > ``` > > bpo-43149 currently calls it an "exception group", but that conflicts with > PEP 654 -- Exception Groups and except* > > ``` > >>>> try: >... pass >

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Jelle Zijlstra
El sáb, 8 may 2021 a las 10:00, Devin Jeanpierre () escribió: > > What are people thoughts on the feature? > > I'm +1, this level of detail in the bytecode is very useful. My main > interest is actually from the AST though. :) In order to be in the > bytecode, one assumes it must first be in the

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Devin Jeanpierre
> What are people thoughts on the feature? I'm +1, this level of detail in the bytecode is very useful. My main interest is actually from the AST though. :) In order to be in the bytecode, one assumes it must first be in the AST. That information is incredibly useful for refactoring tools like

[Python-Dev] a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Thomas Grainger
That's this bit: ``` except (A, B): ^^ ``` bpo-43149 currently calls it an "exception group", but that conflicts with PEP 654 -- Exception Groups and except* ``` >>> try: ... pass ... except A, B: ... pass Traceback (most recent call last):

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Ammar Askar
I really like this idea Nathaniel. We already have a section considering lazy-loading column information in the draft PEP but obviously it has a pretty high implementation complexity since it necessitates a change in the pyc format and the importlib machinery. Long term this might be the way to

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Antoine Pitrou
You can certainly get fancy and apply delta encoding + entropy compression, such as done in Parquet, a high-performance data storage format: https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-encoding-delta_binary_packed--5 (the linked paper from Lemire and Boytsov gives a

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Antoine Pitrou
On Fri, 7 May 2021 23:20:55 +0100 Irit Katriel via Python-Dev wrote: > On Fri, May 7, 2021 at 10:52 PM Pablo Galindo Salgado > wrote: > > > > > The cost of this is having the start column number and end column number > > information for every bytecode instruction > > > > > Is it really

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Antoine Pitrou
On Sat, 8 May 2021 02:58:40 + Neil Schemenauer wrote: > On 2021-05-07, Pablo Galindo Salgado wrote: > > Technically the main concern may be the size of the unmarshalled > > pyc files in memory, more than the storage size of disk. > > It would be cool if we could mmap the pyc files and have