[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Victor Stinner
Hi Skip, I modified co_lnotab format when I worked on the FAT Python optimization project which modified the AST to emit more efficient bytecode. My optimizer moved instructions, the line number could become non-monotonic (..., line 2, line 3, line 2, line 4, ...), and so lnotab (..., +1, -1, +2,

[Python-Dev] aiter/anext review request

2021-03-17 Thread Joshua Bronson
Dear python-dev, New here (but not to Python).  Brett Cannon recommended I start a thread here (thanks, Brett!). In December, two colleagues and I submitted https://github.com/python/cpython/pull/23847, "Add aiter and anext to builtins", which would fix https://bugs.python.org/issue31861.

[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Skip Montanaro
> co_lnotab has had negative deltas since 3.6. Thanks. I'm probably misreading Objects/lnotab_notes.txt. Skip ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org

[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Ned Batchelder
On 3/17/21 6:41 PM, MRAB wrote: On 2021-03-17 22:10, Skip Montanaro wrote: I stumbled on this while trying to generate a line number table in my side project register VM. As I understand it, the line number delta in the output table is supposed to always be >= 0. In my code I'm using

[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread MRAB
On 2021-03-17 22:10, Skip Montanaro wrote: Consider this little session from the tip of the spear: >>> sys.version '3.10.0a6+ (heads/master:0ab152c6b5, Mar 15 2021, 17:24:38) [GCC 10.2.0]' >>> def while2(a): ...     while a >= 0: ...         a -= 1 ...     return a ... >>> dis.dis(while2)  

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Ivan Pozdeev via Python-Dev
On 17.03.2021 23:04, Steve Dower wrote: On 3/17/2021 7:34 PM, Ivan Pozdeev via Python-Dev wrote: On 17.03.2021 20:30, Steve Dower wrote: On 3/17/2021 8:00 AM, Michał Górny wrote: How about writing paths as bytestrings in the long term?  I think this should eliminate the necessity of knowing

[Python-Dev] Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Skip Montanaro
Consider this little session from the tip of the spear: >>> sys.version '3.10.0a6+ (heads/master:0ab152c6b5, Mar 15 2021, 17:24:38) [GCC 10.2.0]' >>> def while2(a): ... while a >= 0: ... a -= 1 ... return a ... >>> dis.dis(while2) 2 0 LOAD_FAST0 (a)

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Steve Dower
On 3/17/2021 7:34 PM, Ivan Pozdeev via Python-Dev wrote: On 17.03.2021 20:30, Steve Dower wrote: On 3/17/2021 8:00 AM, Michał Górny wrote: How about writing paths as bytestrings in the long term?  I think this should eliminate the necessity of knowing the correct encoding for the filesystem.

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Ivan Pozdeev via Python-Dev
On 17.03.2021 20:30, Steve Dower wrote: On 3/17/2021 8:00 AM, Michał Górny wrote: How about writing paths as bytestrings in the long term?  I think this should eliminate the necessity of knowing the correct encoding for the filesystem. That's what we're trying to do, the problem is that they

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Steve Dower
On 3/17/2021 6:08 PM, Stefan Ring wrote: A somewhat radical idea carrying this to the extreme would be to use UTF-16 (LE) on Windows. After all, this _is_ the native file system encoding, and Notepad will happily read and write it. I'm not opposed to detecting a BOM by default (when no other

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Stefan Ring
On Wed, Mar 17, 2021 at 6:37 PM Steve Dower wrote: > > On 3/17/2021 8:00 AM, Michał Górny wrote: > > How about writing paths as bytestrings in the long term? I think this > > should eliminate the necessity of knowing the correct encoding for > > the filesystem. > > That's what we're trying to

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Steve Dower
On 3/17/2021 8:00 AM, Michał Górny wrote: How about writing paths as bytestrings in the long term? I think this should eliminate the necessity of knowing the correct encoding for the filesystem. That's what we're trying to do, the problem is that they start as strings, and so we need to

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Paul Moore
On Wed, 17 Mar 2021 at 09:26, Paul Moore wrote: > The problem is with the transition - we need to find a way to deal > with existing `.pth` files, and with people using older version of > tools (like setuptools and pipx) that write `.pth` files (so we can't > assume, for example, that Python 3.12

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Paul Moore
On Wed, 17 Mar 2021 at 08:52, Inada Naoki wrote: > On Windows, it must be UTF-8. For example, we use `chcp 65001` in > `activate.bat` to support unicode path. > On Unix, raw path is bytestring. So paths can be written as-is. Python > decode it with fsencoding. Remember that .pth files contain

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Antoine Pitrou
On Tue, 16 Mar 2021 11:44:13 +0900 Inada Naoki wrote: > Hi, all. > > I found .pth file is decoded by the default (i.e. locale-specific) encoding. > https://github.com/python/cpython/blob/0269ce87c9347542c54a653dd78b9f60bb9fa822/Lib/site.py#L173 > > pth files contain: > > * import statements >

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Inada Naoki
On Wed, Mar 17, 2021 at 5:33 PM Paul Moore wrote: > > On Wed, 17 Mar 2021 at 08:13, Michał Górny wrote: > > > > On Wed, 2021-03-17 at 13:55 +0900, Inada Naoki wrote: > > > OK. setuptools doesn't specify encoding at all. So locale-specific > > > encoding is used. > > > We can not fix it in short

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Paul Moore
On Wed, 17 Mar 2021 at 08:13, Michał Górny wrote: > > On Wed, 2021-03-17 at 13:55 +0900, Inada Naoki wrote: > > OK. setuptools doesn't specify encoding at all. So locale-specific > > encoding is used. > > We can not fix it in short term. > > How about writing paths as bytestrings in the long

[Python-Dev] Re: pth file encoding

2021-03-17 Thread Michał Górny
On Wed, 2021-03-17 at 13:55 +0900, Inada Naoki wrote: > OK. setuptools doesn't specify encoding at all. So locale-specific > encoding is used. > We can not fix it in short term. How about writing paths as bytestrings in the long term? I think this should eliminate the necessity of knowing the