[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Inada Naoki schrieb am 08.02.22 um 06:15: On Tue, Feb 8, 2022 at 1:47 PM Guido van Rossum wrote: Thanks for trying it! I'm curious why it would be slower (perhaps less locality? perhaps the ...Id... APIs have some other trick up their sleeve?) but since it's also messier and less backwards

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Am 03.02.22 um 22:41 schrieb Gregory P. Smith: datapoint: an internal code search turns up blender, multidict, and typed_ast as open source users of _Py_IDENTIFIER .  Easy to clean up as PRs.  There are a couple of internal uses as well, all of which are similarly easy to address and are only

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Tue, Feb 8, 2022 at 1:47 PM Guido van Rossum wrote: > > Thanks for trying it! I'm curious why it would be slower (perhaps less > locality? perhaps the ...Id... APIs have some other trick up their sleeve?) > but since it's also messier and less backwards compatible than just leaving >

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Thanks for trying it! I'm curious why it would be slower (perhaps less locality? perhaps the ...Id... APIs have some other trick up their sleeve?) but since it's also messier and less backwards compatible than just leaving _Py_IDENTIFIER alone and just not using it, I'd say let's not spend more

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Fri, Feb 4, 2022 at 8:25 PM Eric Snow wrote: > On Fri, Feb 4, 2022, 16:03 Guido van Rossum wrote: >> I wonder if a better solution than that PR wouldn't be to somehow change the >> implementation of _Py_IDENTIFIER() to do that, > > Yeah, I had the same realization today. I'm going to try it

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Yes, please build it; I can't quite follow what you're getting at (probably since I don't know the Windows APIs). On Mon, Feb 7, 2022 at 4:39 AM Steve Dower wrote: > On 2/5/2022 4:09 PM, Guido van Rossum wrote: > > On Sat, Feb 5, 2022 at 5:18 AM Steve Dower > >

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On 2/5/2022 4:09 PM, Guido van Rossum wrote: On Sat, Feb 5, 2022 at 5:18 AM Steve Dower > wrote: The gap this has with what I'd like to see is that it will only work for compile-time strings. If instead it could work for an arbitrary uint8_t

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Sat, Feb 5, 2022 at 5:18 AM Steve Dower wrote: > The gap this has with what I'd like to see is that it will only work for > compile-time strings. If instead it could work for an arbitrary uint8_t > pointer, rather than an embedded array, that allows even runtime strings > to be very cheaply

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On 04Feb2022 2303, Guido van Rossum wrote: You *can* allocate unicode objects statically. We do it in deepfreeze, and Eric's PR under discussion here (https://github.com/python/cpython/pull/30928 ) does it. I wonder if a better solution than that

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Fri, Feb 4, 2022, 16:03 Guido van Rossum wrote: > I wonder if a better solution than that PR wouldn't be to somehow change > the implementation of _Py_IDENTIFIER() to do that, > Yeah, I had the same realization today. I'm going to try it out. -eric >

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

You *can* allocate unicode objects statically. We do it in deepfreeze, and Eric's PR under discussion here ( https://github.com/python/cpython/pull/30928) does it. I wonder if a better solution than that PR wouldn't be to somehow change the implementation of _Py_IDENTIFIER() to do that, and make

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On 2/4/2022 5:37 PM, Eric Snow wrote: On Thu, Feb 3, 2022 at 3:49 PM Eric Snow wrote: I suppose I'd like to know what the value of _Py_IDENTIFIER() is for 3rd party modules. Between Guido, Victor, Stefan, and Sebastian, I'm getting the sense that a public replacement for _Py_IDENTIFER()

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Eric Snow schrieb am 04.02.22 um 17:35: On Fri, Feb 4, 2022 at 8:21 AM Stefan Behnel wrote: Correct. We (intentionally) have our own way to intern strings and do not depend on CPython's identifier framework. You're talking about __Pyx_StringTabEntry (and __Pyx_InitString())? Yes, that's

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 3:49 PM Eric Snow wrote: > I suppose I'd like to know what the value of _Py_IDENTIFIER() is for > 3rd party modules. Between Guido, Victor, Stefan, and Sebastian, I'm getting the sense that a public replacement for _Py_IDENTIFER() would be worth pursuing. Considering that

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Fri, Feb 4, 2022 at 8:21 AM Stefan Behnel wrote: > Correct. We (intentionally) have our own way to intern strings and do not > depend on CPython's identifier framework. You're talking about __Pyx_StringTabEntry (and __Pyx_InitString())? -eric ___

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, 2022-02-03 at 15:32 +0100, Victor Stinner wrote: > By the way, Argument Clinic now produces way faster calling > conventions than hand-written METH_VARARGS with PyArg_ParseTuple(). > It > would be make this tool available to 3rd party projects. > > Either extract it and put it on PyPI,

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Ronald Oussoren via Python-Dev schrieb am 03.02.22 um 14:46: On 2 Feb 2022, at 23:41, Eric Snow wrote: * a little less convenient: adding a global string requires modifying a separate file from the one where you actually want to use the string * strings can get "orphaned" (I'm planning on

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Victor Stinner schrieb am 03.02.22 um 22:46: Oh right, Cython seems to be a false positive. A code search found 3 references to __Pyx_PyObject_LookupSpecial(): PYPI-2022-01-26-TOP-5000/Cython-0.29.26.tar.gz: Cython-0.29.26/Cython/Compiler/ExprNodes.py: lookup_func_name =

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 11:49 PM Eric Snow wrote: > I suppose I'd like to know what the value of _Py_IDENTIFIER() is for > 3rd party modules. _Py_IDENTIFIER() API is simple and short, Python creates the Python string object automatically, *and* clears this object at exit. It's convenient. On the

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 4:01 PM Guido van Rossum wrote: > Why not read through some of that code and see what they are doing with it? Yep, I'm planning on it. > I imagine one advantage is that _Py_IDENTIFIER() can be used entirely local > to a function. Yeah, they'd have to put something like

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 7:26 AM Victor Stinner wrote: > In bpo-39465, I made the _PyUnicode_FromId() compatible with running > sub-interpreters in parallel (one GIL per interpreter). > > A "static" PyUnicodeObject would have to share the reference count > between sub-interpreters, whereas

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 2:50 PM Eric Snow wrote: > On Wed, Feb 2, 2022 at 11:50 PM Inada Naoki > wrote: > > It would be nice to provide something similar to _PY_IDENTIFIER, but > > designed (and documented) for 3rd party modules like this. > > I suppose I'd like to know what the value of

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 7:17 AM Victor Stinner wrote: > In the top 5000 PyPI projects, I found 11 projects using them: > [snip[ > They use the these 17 functions: Thanks! That is super helpful. > If the _Py_IDENTIFIER() API is removed, it would be *nice* to provide > a migrate path (tool?) to

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 6:46 AM Ronald Oussoren wrote: > Although my gut feeling is that adding a the CI check you mention is good > enough and adding the tooling for generating code isn’t worth the additional > complexity. Yeah, I came to the same conclusion. :) -eric

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Wed, Feb 2, 2022 at 11:50 PM Inada Naoki wrote: > It would be nice to provide something similar to _PY_IDENTIFIER, but > designed (and documented) for 3rd party modules like this. I suppose I'd like to know what the value of _Py_IDENTIFIER() is for 3rd party modules. They can already use

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Oh right, Cython seems to be a false positive. A code search found 3 references to __Pyx_PyObject_LookupSpecial(): PYPI-2022-01-26-TOP-5000/Cython-0.29.26.tar.gz: Cython-0.29.26/Cython/Compiler/ExprNodes.py: lookup_func_name = '__Pyx_PyObject_LookupSpecial'

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Wed, Feb 2, 2022 at 2:48 PM Eric Snow wrote: > I'm planning on moving us to a simpler, more efficient alternative to > _Py_IDENTIFIER(), but want to see if there are any objections first > before moving ahead. Also see https://bugs.python.org/issue46541. > > _Py_IDENTIFIER() was added in

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 6:31 AM Victor Stinner wrote: > On Wed, Feb 2, 2022 at 11:43 PM Eric Snow > wrote: > > My plan is to replace our use of _Py_IDENTIFIER() with statically > > initialized string objects (as fields under _PyRuntimeState). That > > involves the following: > > > > * add a

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Thu, Feb 3, 2022 at 6:37 AM Victor Stinner wrote: > By the way, Argument Clinic now produces way faster calling > conventions than hand-written METH_VARARGS with PyArg_ParseTuple(). It > would be make this tool available to 3rd party projects. > > Either extract it and put it on PyPI, but it

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

Victor Stinner wrote: > On Wed, Feb 2, 2022 at 11:49 PM Eric Snow ericsnowcurren...@gmail.com wrote: > In the top 5000 PyPI projects, I found 11 projects using them: > * Cython-0.29.26 (and so indirect most projects using Cython) I believe Cython is (for once) a false alarm here. I don't think

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

By the way, Argument Clinic now produces way faster calling conventions than hand-written METH_VARARGS with PyArg_ParseTuple(). It would be make this tool available to 3rd party projects. Either extract it and put it on PyPI, but it means that Python will need to Python and pip install something

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Wed, Feb 2, 2022 at 11:43 PM Eric Snow wrote: > My plan is to replace our use of _Py_IDENTIFIER() with statically > initialized string objects (as fields under _PyRuntimeState). That > involves the following: > > * add a PyUnicodeObject field (not a pointer) to _PyRuntimeState for > each

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Wed, Feb 2, 2022 at 11:49 PM Eric Snow wrote: > FTR, here is the (private/internal) C-API affected by getting rid of > _Py_IDENTIFIER(): > > * 21 C-API functions with `_Py_Identifer` parameters - would be dropped >+ _PyUnicode_FromId() >+ _PyUnicode_EqualToASCIIId() >+

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

> On 2 Feb 2022, at 23:41, Eric Snow wrote: > > I […] > > Cons: > > * a little less convenient: adding a global string requires modifying > a separate file from the one where you actually want to use the string > * strings can get "orphaned" (I'm planning on checking in CI) > * some strings

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

+1 for overall. On Thu, Feb 3, 2022 at 7:45 AM Eric Snow wrote: > > > I'd also like to actually get rid of _Py_IDENTIFIER(), along with > other related API including ~14 (private) C-API functions. Dropping > all that helps reduce maintenance costs. However, at least one PyPI > project

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

On Wed, Feb 2, 2022 at 3:41 PM Eric Snow wrote: > I'd also like to actually get rid of _Py_IDENTIFIER(), along with > other related API including ~14 (private) C-API functions. FTR, here is the (private/internal) C-API affected by getting rid of _Py_IDENTIFIER(): * 21 C-API functions with