[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:


New changeset 21412d037b07c08266e96dfd0c0e44a1b7693bc1 by Eric Snow in branch 
'main':
bpo-46541: Add a Comment About When to Use _Py_DECLARE_STR(). (gh-32063)
https://github.com/python/cpython/commit/21412d037b07c08266e96dfd0c0e44a1b7693bc1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +30154
pull_request: https://github.com/python/cpython/pull/32063

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset d168c728f7114959e8fc147538ea1d24f2f5af79 by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from lzma module (GH-31683)
https://github.com/python/cpython/commit/d168c728f7114959e8fc147538ea1d24f2f5af79


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29804
pull_request: https://github.com/python/cpython/pull/31683

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset e91b0a7139d4a4cbd2351ccb5cd021a100cf42d2 by Kumar Aditya in 
branch 'main':
bpo-46541: remove usage of _Py_IDENTIFIER from _ssl module (GH-31599)
https://github.com/python/cpython/commit/e91b0a7139d4a4cbd2351ccb5cd021a100cf42d2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 0cc63641859b2f60ea65bb7c0b6d1cfcec1e2f1a by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from multibytecodec (GH-31475)
https://github.com/python/cpython/commit/0cc63641859b2f60ea65bb7c0b6d1cfcec1e2f1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset c32aef48533769161e1247927a5b418322e0860c by Erlend Egeberg 
Aasland in branch 'main':
bpo-46541: Remove unneeded visits from sqlite3 (GH-31609)
https://github.com/python/cpython/commit/c32aef48533769161e1247927a5b418322e0860c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
pull_requests: +29732
pull_request: https://github.com/python/cpython/pull/31609

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Christian Heimes]

Christian Heimes  added the comment:


New changeset 088dd76dba68c2538776d9920607f81e54544cbd by Dong-hee Na in branch 
'main':
bpo-46541: Remove unnecessary Py_VISIT (GH-31608)
https://github.com/python/cpython/commit/088dd76dba68c2538776d9920607f81e54544cbd


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29731
pull_request: https://github.com/python/cpython/pull/31608

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29722
pull_request: https://github.com/python/cpython/pull/31599

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29611
pull_request: https://github.com/python/cpython/pull/31475

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 2b86616456629e11de33629da1bb732f033c436e by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from pyexpat (GH-31468)
https://github.com/python/cpython/commit/2b86616456629e11de33629da1bb732f033c436e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29598
pull_request: https://github.com/python/cpython/pull/31468

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Christoph Reiter]

Change by Christoph Reiter :


--
nosy:  -lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 8cb5f707a841832aeac4b01c8b6a0e72dced52ae by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from array module (GH-31376)
https://github.com/python/cpython/commit/8cb5f707a841832aeac4b01c8b6a0e72dced52ae


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Jason Wilkes]

Change by Jason Wilkes :


--
nosy: +notarealdeveloper
nosy_count: 7.0 -> 8.0
pull_requests: +29528
pull_request: https://github.com/python/cpython/pull/30310

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:

> I think it's fine to use the one issue.

Great! and thank you for all your efforts :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

(from https://github.com/python/cpython/pull/31376#issuecomment-1041836106)

[corona10]
> Should we create the separate bpo issue if module changes are too noisy?

I think it's fine to use the one issue.  There are only 26 modules with 
`NEEDS_PY_IDENTIFIER` and none have much risk with the change.  So it's 
unlikely that we'll get any discussion about any specific module.  If we do, we 
can easily create an issue then for the module in question.  If one of the 
modules had many uses of `_Py_IDENTIFIER()` then it might make sense to split 
it out, but IIRC none of them have very many.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29525
pull_request: https://github.com/python/cpython/pull/31376

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset e8a19b092fc0551581e10d6f310dd5feabac7609 by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from mmap module (GH-31375)
https://github.com/python/cpython/commit/e8a19b092fc0551581e10d6f310dd5feabac7609


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29524
pull_request: https://github.com/python/cpython/pull/31375

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset b2077117d125925210148294eefee28797b7ff4c by Erlend Egeberg 
Aasland in branch 'main':
bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351)
https://github.com/python/cpython/commit/b2077117d125925210148294eefee28797b7ff4c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset d64f3caebe8f8e31ecd193e0bd25105400153ece by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from csv module (GH-31372)
https://github.com/python/cpython/commit/d64f3caebe8f8e31ecd193e0bd25105400153ece


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29522
pull_request: https://github.com/python/cpython/pull/31372

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset e59309b9d0969d5c8f493cb8df28afa6f38d6273 by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from dbms modules (GH-31358)
https://github.com/python/cpython/commit/e59309b9d0969d5c8f493cb8df28afa6f38d6273


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:


New changeset 4d8a515d193a4c9f3844704f974ddb870d7ee383 by Eric Snow in branch 
'main':
bpo-46541: Scan Fewer Files in generate_global_objects.py (gh-31364)
https://github.com/python/cpython/commit/4d8a515d193a4c9f3844704f974ddb870d7ee383


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:


New changeset 6c8958948666403f2370ca7b4c0a52b2010ec16d by Eric Snow in branch 
'main':
bpo-46541: Drop the check for orphaned global strings. (gh-31363)
https://github.com/python/cpython/commit/6c8958948666403f2370ca7b4c0a52b2010ec16d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +29514
pull_request: https://github.com/python/cpython/pull/31364

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +29513
pull_request: https://github.com/python/cpython/pull/31363

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
pull_requests: +29508
pull_request: https://github.com/python/cpython/pull/31358

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
nosy: +erlendaasland
nosy_count: 5.0 -> 6.0
pull_requests: +29500
pull_request: https://github.com/python/cpython/pull/31351

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:


New changeset 12360aa159c42c7798fd14225d271e6fd84db7eb by Eric Snow in branch 
'main':
bpo-46541: Discover the global strings. (gh-31346)
https://github.com/python/cpython/commit/12360aa159c42c7798fd14225d271e6fd84db7eb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +29496
pull_request: https://github.com/python/cpython/pull/31346

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +29494
pull_request: https://github.com/python/cpython/pull/31344

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

With core code sorted out, stdlib and 3rd party extension modules are left to 
sort out.

I see the following possibilities:

1. leave `_Py_IDENTIFIER()` alone (it is already disallowed in core code)
2. change `_Py_IDENTIFIER()` to create static string objects (then get rid of 
global state)
3. get rid of `_Py_IDENTIFIER()`
   a. provide a public alternative (needs a PEP)
   b. first work with 3rd party projects to stop using it

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
Removed message: https://bugs.python.org/msg413241

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

With core code sorted out, stdlib and 3rd party extension modules are left to 
sort out.

I see the following possibilities:

* leave `_Py_IDENTIFIER()` alone (it is already disallowed in core code)
* change `_Py_IDENTIFIER()` to create static string objects (then get rid of 
global state)
* get rid of `_Py_IDENTIFIER()`
   a. provide a public alternative (needs a PEP)
   b. first work with 3rd party projects to stop using it

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

On Fri, Feb 11, 2022 at 1:36 AM Christoph Reiter  wrote:
> Sorry if off topic, but I noticed that CPython doesn't deprecate macros in 
> code, while with gcc/clang it's possible to show compiler warnings for them 
> using some pragma magic:
> [snip]
> Maybe that makes getting rid of them easier in the long run?

That's a good question.  We do have Py_DEPRECATED() (in
Include/pyport.h), which is used for symbols.  I'm not sure anyone has
given much thought to deprecating macros, but it's probably worth
considering.  I recommend that you post something about this to
python-...@python.org.

FWIW, here are other explanations of how to deprecate macros:

* 
https://stackoverflow.com/questions/57478368/what-is-the-best-way-to-mark-macro-as-deprecated/57479189#57479189
* 
https://stackoverflow.com/questions/2681259/how-to-deprecate-a-c-pre-processor-macro-in-gcc/29297970#29297970

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Christoph Reiter]

Christoph Reiter  added the comment:

Sorry if off topic, but I noticed that CPython doesn't deprecate macros in 
code, while with gcc/clang it's possible to show compiler warnings for them 
using some pragma magic:

$ gcc a.c
a.c: In function 'main':
a.c:29:13: warning: Deprecated pre-processor symbol
   29 | PySomethingDeprecated (0);
  | ^~
a.c:30:13: warning: Deprecated pre-processor symbol: replace with 
"SomethingCompletelyDifferent"
   30 | PySomethingDeprecated2 (42);
  | ^~~~

Here is how glib implements this for example: 
https://gist.github.com/lazka/4749c74249a3918a059d944040aca4a3

Maybe that makes getting rid of them easier in the long run?

--
nosy: +lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Kumar Aditya]

Change by Kumar Aditya :


--
pull_requests: +29428
pull_request: https://github.com/python/cpython/pull/31261

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:


New changeset 81c72044a181dbbfbf689d7a977d0d99090f26a8 by Eric Snow in branch 
'main':
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized 
global objects. (gh-30928)
https://github.com/python/cpython/commit/81c72044a181dbbfbf689d7a977d0d99090f26a8


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

(thanks Victor: 
https://mail.python.org/archives/list/python-...@python.org/message/7RMLIJHUWVBZFV747TFEHOE6LNBVQSMM/)

3rd party use of _Py_IDENTIFIER():

* blender
   + 
https://github.com/blender/blender/blob/master/source/blender/python/intern/bpy_traceback.c#L53
  - copied from core code
  - "msg", "filename", "lineno", "offset", "text", ""
  - uses _PyObject_GetAttrId()
* datatable
   + 
https://github.com/h2oai/datatable/blob/45a87337bc68576c7fb6900f524925d4fb77d6a6/src/core/python/obj.cc#L76
  - in C++ wrapper getting sys.stdout, etc. and writing to sys.stdout
  - has to hack around C++14 support
  - has a fallback under limited API
  - "stdin", "stdout", "stderr", "write"
  - uses _PySys_GetObjectId(), _PyObject_GetAttrId()
* multidict (in aiohttp)
   + 
https://github.com/aio-libs/multidict/blob/6dedb623cca8e8fe64f502dfa479826efc321385/multidict/_multilib/defs.h#L8
   + 
https://github.com/aio-libs/multidict/blob/6dedb623cca8e8fe64f502dfa479826efc321385/multidict/_multilib/istr.h#L46
   + 
https://github.com/aio-libs/multidict/blob/6dedb623cca8e8fe64f502dfa479826efc321385/multidict/_multilib/pair_list.h#L114
  - calling str.lower()
  - "lower"
  - uses _PyObject_CallMethodId()
* mypy (exclusively in mypyc, including in generated code!)
   + 
https://github.com/python/mypy/blob/3c935bdd1332672f5daeae7f3f9a858a45d4/mypyc/lib-rt/dict_ops.c#L76
   + 
https://github.com/python/mypy/blob/3c935bdd1332672f5daeae7f3f9a858a45d4/mypyc/lib-rt/dict_ops.c#L131
  - "setdefault", "update"
  - uses _PyObject_CallMethodIdObjArgs(), _PyObject_CallMethodIdOneArg()
   + 
https://github.com/python/mypy/blob/2b7e2df923f7e4a3a199915b3c8563f45bc69dfa/mypyc/lib-rt/pythonsupport.h#L26
   + 
https://github.com/python/mypy/blob/2b7e2df923f7e4a3a199915b3c8563f45bc69dfa/mypyc/lib-rt/pythonsupport.h#L109
  - "__mro_entries__", "__init_subclass__"
  - uses _PyObject_LookupAttrId(), _PyObject_GetAttrId()
   + 
https://github.com/python/mypy/blob/2b7e2df923f7e4a3a199915b3c8563f45bc69dfa/mypyc/lib-rt/misc_ops.c#L27
   + 
https://github.com/python/mypy/blob/2b7e2df923f7e4a3a199915b3c8563f45bc69dfa/mypyc/lib-rt/misc_ops.c#L47
  - "send", "throw", "close"
  - uses _PyObject_CallMethodIdOneArg(), _PyObject_GetAttrId()
   + 
https://github.com/python/mypy/blob/8c5c915a89ec0f35b3e07332c7090e62f143043e/mypyc/lib-rt/bytes_ops.c#L104
  - "join"
  - uses _PyObject_CallMethodIdOneArg()
   + 
https://github.com/python/mypy/blob/3c935bdd1332672f5daeae7f3f9a858a45d4/mypyc/codegen/emitwrapper.py#L326
   + 
https://github.com/python/mypy/blob/2b7e2df923f7e4a3a199915b3c8563f45bc69dfa/mypyc/lib-rt/misc_ops.c#L694
  - uses _PyObject_GetAttrId()
* pickle5
   + 
https://github.com/pitrou/pickle5-backport/blob/e6117502435aba2901585cc6c692fb9582545f08/pickle5/_pickle.c#L224
   + 
https://github.com/pitrou/pickle5-backport/blob/e6117502435aba2901585cc6c692fb9582545f08/pickle5/compat.h
  - "getattr"
  - uses _PyUnicode_FromId()
* pysqlite3
   + 
https://github.com/coleifer/pysqlite3/blob/f302859dc9ddb47a1089324dbca3873740b74af9/src/microprotocols.c#L103
   + 
https://github.com/coleifer/pysqlite3/blob/f302859dc9ddb47a1089324dbca3873740b74af9/src/microprotocols.c#L119
  - "__adapt__", "__conform__"
  - uses _PyObject_CallMethodId()
   + 
https://github.com/coleifer/pysqlite3/blob/093b88d1a58b141db8cf971c35ea1f6b674d0d02/src/connection.c#L51
   + 
https://github.com/coleifer/pysqlite3/blob/093b88d1a58b141db8cf971c35ea1f6b674d0d02/src/connection.c#L772
  - "finalize", "value", "upper", "cursor"
  - uses _PyObject_CallMethodId(), _PyObject_CallMethodIdObjArgs()
   + 
https://github.com/coleifer/pysqlite3/blob/49ce9c7a89a3c9f47ab8d32b6c4e2f7d629c1688/src/module.c#L195
  - "upper"
  - uses _PyObject_CallMethodId()
   + 
https://github.com/coleifer/pysqlite3/blob/91b2664f525b19feedfca3f0913302c6f1e8be8a/src/cursor.c#L103
  - "upper"
  - uses _PyObject_CallMethodId()
* typed_ast
   + a fork of CPython's ast code
* zodbpickle
   + a fork of CPython's pickle

All of them should be trivial to drop _Py_IDENTIFIER() without any real 
performance impact or mess.

Also, the following implies that PyPy has some sort of _Py_IDENTIFIER() 
support: 
https://github.com/benhoyt/scandir/blob/3396aa4155ffde8600a0e9ca50d5872569169b5d/_scandir.c#L51.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

FYI, I've posted to python-dev for feedback before proceeding:

https://mail.python.org/archives/list/python-...@python.org/thread/DNMZAMB4M6RVR76RDZMUK2WRLI6KAAYS/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

If necessary, we can keep _Py_IDENTIFIER() (and the functions).  Regardless, we 
can stop using it internally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[STINNER Victor]

STINNER Victor  added the comment:

> `_Py_Identifier` has been useful but at this point there is a faster and 
> simpler approach we could take as a replacement: statically initialize the 
> objects as fields on `_PyRuntimeState` and reference them directly through a 
> macro.

This change is going to break projects in the wild. Yes, people use the 
_Py_IDENTIFIER(), _PyUnicode_FromId() and other "Id" variant of many functions 
in 3rd party projects.

Is it possible to keep runtime initialization if this API is used by 3rd party 
code?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:


New changeset 247480a21cb165efdacc346a2d589dfc27e18283 by Eric Snow in branch 
'main':
bpo-46541: Generate the global objects initializer. (gh-30941)
https://github.com/python/cpython/commit/247480a21cb165efdacc346a2d589dfc27e18283


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Kumar Aditya]

Change by Kumar Aditya :


--
nosy: +kumaraditya303

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Change by Eric Snow :


--
keywords: +patch
pull_requests: +29120
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/30941

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

Eric Snow  added the comment:

## Background ##

`_Py_Identifier` (and `_Py_IDENTIFIER()`, etc.) was added in 2011 [1][2] for 
several reasons:

* provide a consistent approach for a common optimization: caching C-string 
based string objects
* facilitate freeing those objects during runtime finalization

The solution involved using a static variable defined, using `_Py_IDENTIFIER()` 
near the code that needed the string.  The variable (a `_Py_Identifier`) would 
hold the desired C string (statically initialized) and the corresponding 
(lazily created) `PyUnicodeObject`.  The code where the `_Py_Identifier` was 
defined would then pass it to specialized versions of various C-API that would 
normally consume a C string or `PyUnicodeObject`.  Then that code would use 
either the C-string or the object (creating and caching it first if not done 
already).  This approach decentralized the caching but also provided a single 
tracking mechanism that made it easier to clean up the objects.

Over the last decade a number of changes were made, including recent changes to 
make the identifiers per-interpreter and to use a centralized cache.


[1] 
https://github.com/python/cpython/commit/afe55bba33a20f87a58f940186359237064b428f
[2] 
https://mail.python.org/archives/list/python-...@python.org/message/FRUTTE47JO2XN3LXV2J4VB5A5VILILLA/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Eric Snow]

New submission from Eric Snow :

`_Py_Identifier` has been useful but at this point there is a faster and 
simpler approach we could take as a replacement: statically initialize the 
objects as fields on `_PyRuntimeState` and reference them directly through a 
macro.

This would involve the following:

* add a `PyUnicodeObject field (not a pointer) to `_PyRuntimeState` for each 
string that currently uses `_Py_IDENTIFIER()`
* initialize each object as part of the static initializer for `_PyRuntimeState`
* make each "immortal" (e.g. start with a really high refcount)
* add a macro to look up a given string
* update each location that currently uses `_Py_IDENTIFIER()` to use the new 
macro instead

As part of this, we would also do the following:

* get rid of all C-API functions with `_Py_Identifer` parameters
* get rid of the old runtime state related to identifiers
* get rid of `_Py_Identifier`, `_Py_IDENTIFIER()`, etc.

(Note that there are several hundred uses of `_Py_IDENTIFIER()`, including a 
number of duplicates.)


Pros:

* reduces indirection (and extra calls) for C-API using the strings (making the 
code easier to understand and speeding it up)
* the objects are referenced from a fixed address in the static data section 
(speeding things up and allowing the C compiler to optimize better)
* there is no lazy allocation (or lookup, etc.) so there are fewer possible 
failures when the objects get used (thus less error return checking)
* simplifies the runtime state
* saves memory (at little, at least)
* the approach for per-interpreter is simpler (if needed)
* reduces the number of static variables in any given C module
* reduces the number of functions in the ("private") C-API
* "deep frozen" modules can use these strings
* other commonly-used strings could be pre-allocated by adding 
`_PyRuntimeState` fields for them

Cons:

* churn
* adding a string to the list requires modifying a separate file from the one 
where you actually want to use the string
* strings can get "orphaned" (we could prevent this with a check in `make 
check`)
* some PyPI packages may rely on `_Py_IDENTIFIER()` (even though it is 
"private" C-API)
* some strings may never get used for any given ./python invocation


Note that with a basic partial implementation (GH-30928) I'm seeing a 1% 
improvement in performance (see 
https://github.com/faster-cpython/ideas/issues/230).

--
assignee: eric.snow
components: Interpreter Core
messages: 411799
nosy: eric.snow, serhiy.storchaka, vstinner
priority: normal
pull_requests: 29107
severity: normal
stage: needs patch
status: open
title: Replace _Py_IDENTIFIER() with statically initialized objects.
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com