[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: @Andrei: Ping? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Andrei Dorian Duma added the comment: Removed _Py_IDENTIFIERs duplicated in the same file, except: _Py_IDENTIFIER(replace): Modules/zipimport.c:563 which was enclosed in some #ifdefs. -- keywords: +patch Added file: http://bugs.python.org/file32587/remove_duplicated_pyidentifiers.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Changes by STINNER Victor victor.stin...@gmail.com: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Roundup Robot added the comment: New changeset 518e3b174120 by Victor Stinner in branch 'default': Issue #19515: Remove identifiers duplicated in the same file. http://hg.python.org/cpython/rev/518e3b174120 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Roundup Robot added the comment: New changeset d2209b9f8971 by Victor Stinner in branch 'default': Issue #19515: Remove duplicated identifiers in zipimport.c http://hg.python.org/cpython/rev/d2209b9f8971 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Martin v. Löwis added the comment: I'd be +0 on extracting common identifiers. I see a slight increase of readability, and expect a slight reduction of memory usage, and a tiny reduction in runtime. It's not really worth the effort, but I fail to see that it causes harm. I see no point in reverting cases where this approach is already taken. I don't quite understand Victor's interest in this, either, as there are hundreds of open real bugs that could use his attention. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: 2013/11/8 Martin v. Löwis rep...@bugs.python.org: I'd be +0 on extracting common identifiers. I (...) expect a slight reduction of memory usage, and a tiny reduction in runtime. Only duplicated Py_IDENTIFIER structures would be removed in memory, but these structures are very small (3 pointers or something like that). The identifier strings are already interned, so not duplicated in memory. It's not really worth the effort, but I fail to see that it causes harm. Initializing an identifier has to decode the literal byte string from UTF-8, but Python UTF-8 decoder is really fast. I'm not sure that it's possible to see a difference on the startup time. I see no point in reverting cases where this approach is already taken. I only reverted shared identifiers added a few days ago in issue #19512. I agree to leave the old code unchanged. I don't quite understand Victor's interest in this, either, as there are hundreds of open real bugs that could use his attention. I tried to explain my motivation on using more identifiers in the issue #19512. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +andrei.duma, loewis ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: If most identifiers are stored in the same place, it would become possible to have a cleanup function to clear all identifiers. Such function could be called at Python shutdown to release as much memory as possible. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Antoine Pitrou added the comment: What are you trying to achieve exactly? I don't think sharing identifier structs will gain anything significant. Please don't make the source code less readable in search for some mythical efficiency. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Martin v. Löwis added the comment: Victor: There already *is* a cleanup function that clears all allocated identifier memory at interpreter shutdown. Please read the source. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: Victor: There already *is* a cleanup function that clears all allocated identifier memory at interpreter shutdown. Please read the source. Oh, great! I never noticed _PyUnicode_ClearStaticStrings(). Call trace: Py_Finalize() - _PyUnicode_ClearStaticStrings() - _PyUnicode_Fini(). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Martin v. Löwis added the comment: Well, that was one of the motivations of introducing this Py_IDENTIFIER machinery: to be able to cleanup at the end (unlike the static variables that were used before, which couldn't be cleaned up). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: Antoine, Martin: So, what do you think? Is it worth to move most common identifiers to a single place to not duplicate them? If identifiers are already cleared at exit, the advantage would be to initialize duplicated identifiers more quickly, and don't initialize duplicated identifiers multiple times (once per copy). If we choose to not share common identifiers, _PyId_xxx identifiers from pythonrun.c must be removed. There are also some identifiers duplicated in the same file which can be moved at the top to remove at least duplicates in a single file, as it was done for other identifiers in issue #19514. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Antoine Pitrou added the comment: Antoine, Martin: So, what do you think? Is it worth to move most common identifiers to a single place to not duplicate them? Well, worth what? :) If you don't tell us what it brings (numbers?), I'm against it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: If you don't tell us what it brings (numbers?), I'm against it. For performances, it's probably very close to zero speed up. For the memory, it's a few bytes per duplicated identifier. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Antoine Pitrou added the comment: If you don't tell us what it brings (numbers?), I'm against it. For performances, it's probably very close to zero speed up. For the memory, it's a few bytes per duplicated identifier. Well, then IMHO it's not worth it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: Well, then IMHO it's not worth it. Ok, you are probably right :-) @Andrei: Are you interested to work on a patch to remove identifiers duplicated in the same file? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Andrei Dorian Duma added the comment: @Andrei: Are you interested to work on a patch to remove identifiers duplicated in the same file? Yes, I will provide a patch in a day or two. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: New changeset 01c4a0af73cf by Victor Stinner in branch 'default': Issue #19512, #19515: remove shared identifiers, move identifiers where they http://hg.python.org/cpython/rev/01c4a0af73cf This changeset removes some identifiers duplicated in the same file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
Roundup Robot added the comment: New changeset 01c4a0af73cf by Victor Stinner in branch 'default': Issue #19512, #19515: remove shared identifiers, move identifiers where they http://hg.python.org/cpython/rev/01c4a0af73cf -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
New submission from STINNER Victor: I started to share some common identifiers in Python/pythonrun.c, extract: /* Common identifiers */ _Py_Identifier _PyId_argv = _Py_static_string_init(argv); _Py_Identifier _PyId_builtins = _Py_static_string_init(builtins); ... Do you think it would be interesting to continue to share such identifier somewhere? Maybe in a new file? We might do the same for some common strings like empty string, single character (like \n), etc. See also issue #19514. Duplicated identifiers in the io module: _Py_IDENTIFIER(_dealloc_warn): Modules/_io/bufferedio.c:17 _Py_IDENTIFIER(_dealloc_warn): Modules/_io/textio.c:15 _Py_IDENTIFIER(__IOBase_closed): Modules/_io/iobase.c:183 _Py_IDENTIFIER(__IOBase_closed): Modules/_io/iobase.c:62 _Py_IDENTIFIER(read1): Modules/_io/bufferedio.c:24 _Py_IDENTIFIER(read1): Modules/_io/textio.c:25 _Py_IDENTIFIER(readable): Modules/_io/bufferedio.c:25 _Py_IDENTIFIER(readable): Modules/_io/textio.c:26 _Py_IDENTIFIER(readall): Modules/_io/fileio.c:590 _Py_IDENTIFIER(readall): Modules/_io/iobase.c:802 _Py_IDENTIFIER(seek): Modules/_io/iobase.c:101 _Py_IDENTIFIER(seek): Modules/_io/textio.c:29 _Py_IDENTIFIER(writable): Modules/_io/bufferedio.c:27 _Py_IDENTIFIER(writable): Modules/_io/textio.c:33 Duplicated identifiers in other files: _Py_IDENTIFIER(append): Modules/_elementtree.c:2373 _Py_IDENTIFIER(append): Modules/_pickle.c:5056 _Py_IDENTIFIER(__bases__): Objects/abstract.c:2417 _Py_IDENTIFIER(__bases__): Objects/typeobject.c:2813 _Py_IDENTIFIER(builtins): pythonrun.c _Py_IDENTIFIER(builtins): Python/sysmodule.c:169 _Py_IDENTIFIER(__bytes__): Objects/bytesobject.c:2458 _Py_IDENTIFIER(__bytes__): Objects/object.c:563 _Py_IDENTIFIER(__class__): Objects/abstract.c:2492 _Py_IDENTIFIER(__class__): Objects/typeobject.c:42 _Py_IDENTIFIER(__class__): Python/codecs.c:470 _Py_IDENTIFIER(__class__): Python/compile.c:553 _Py_IDENTIFIER(close): Modules/_io/bufferedio.c:16 _Py_IDENTIFIER(close): Modules/_io/fileio.c:129 _Py_IDENTIFIER(close): Modules/_io/textio.c:14 _Py_IDENTIFIER(close): Modules/mmapmodule.c:707 _Py_IDENTIFIER(close): Modules/ossaudiodev.c:540 _Py_IDENTIFIER(close): Modules/selectmodule.c:1513 _Py_IDENTIFIER(close): Objects/genobject.c:173 _Py_IDENTIFIER(close): Python/traceback.c:235 _Py_IDENTIFIER(__delitem__): Objects/typeobject.c:5132 _Py_IDENTIFIER(__delitem__): Objects/typeobject.c:5183 _Py_IDENTIFIER(__dict__): Modules/arraymodule.c:2040 _Py_IDENTIFIER(__dict__): Modules/_collectionsmodule.c:894 _Py_IDENTIFIER(__dict__): Modules/_pickle.c:5204 _Py_IDENTIFIER(__dict__): Objects/bytearrayobject.c:2704 _Py_IDENTIFIER(__dict__): Objects/moduleobject.c:479 _Py_IDENTIFIER(__dict__): Objects/setobject.c:1946 _Py_IDENTIFIER(__dict__): Objects/typeobject.c:43 _Py_IDENTIFIER(__dict__): Parser/asdl_c.py:702 _Py_IDENTIFIER(__dict__): Python/bltinmodule.c:1942 _Py_IDENTIFIER(__dict__): Python/ceval.c:4660 _Py_IDENTIFIER(__dict__): Python/Python-ast.c:544 _Py_IDENTIFIER(__doc__): Objects/descrobject.c:1443 _Py_IDENTIFIER(__doc__): Objects/typeobject.c:44 _Py_IDENTIFIER(enable): Modules/faulthandler.c:1050 _Py_IDENTIFIER(enable): Modules/_posixsubprocess.c:50 _Py_IDENTIFIER(encoding): Python/bltinmodule.c:1716 _Py_IDENTIFIER(encoding): Python/pythonrun.c:1359 _Py_IDENTIFIER(encoding): Python/sysmodule.c:107 _Py_IDENTIFIER(filename): Python/errors.c:932 _Py_IDENTIFIER(filename): Python/pythonrun.c:1634 _Py_IDENTIFIER(fileno): Modules/faulthandler.c:133 _Py_IDENTIFIER(fileno): Modules/_io/_iomodule.c:238 _Py_IDENTIFIER(fileno): Modules/_io/textio.c:17 _Py_IDENTIFIER(fileno): Objects/fileobject.c:200 _Py_IDENTIFIER(fileno): Python/bltinmodule.c:35 _Py_IDENTIFIER(flush): Modules/faulthandler.c:134 _Py_IDENTIFIER(flush): Modules/_io/bufferedio.c:18 _Py_IDENTIFIER(flush): Modules/_io/textio.c:18 _Py_IDENTIFIER(flush): Python/bltinmodule.c:1550 _Py_IDENTIFIER(flush): Python/bltinmodule.c:36 _Py_IDENTIFIER(flush): Python/pythonrun.c:2120 _Py_IDENTIFIER(flush): Python/pythonrun.c:519 _Py_IDENTIFIER(__getinitargs__): Modules/_datetimemodule.c:3075 _Py_IDENTIFIER(__getinitargs__): Modules/_pickle.c:4501 _Py_IDENTIFIER(TextIOWrapper): Python/pythonrun.c:1005 _Py_IDENTIFIER(TextIOWrapper): Python/traceback.c:237 _Py_IDENTIFIER(__getstate__): Modules/_datetimemodule.c:3076 _Py_IDENTIFIER(__getstate__): Objects/typeobject.c:3448 _Py_IDENTIFIER(__import__): Python/ceval.c:2428 _Py_IDENTIFIER(__import__): Python/import.c:1230 _Py_IDENTIFIER(__module__): Objects/typeobject.c:48 _Py_IDENTIFIER(__module__): Python/errors.c:840 _Py_IDENTIFIER(__module__): Python/pythonrun.c:1916 _Py_IDENTIFIER(__name__): Objects/classobject.c:17 _Py_IDENTIFIER(__name__): Objects/typeobject.c:49 _Py_IDENTIFIER(__name__): Objects/weakrefobject.c:159 _Py_IDENTIFIER(__name__): Python/_warnings.c:260 _Py_IDENTIFIER(__name__): Python/codecs.c:471 _Py_IDENTIFIER(__name__): Python/import.c:1234 _Py_IDENTIFIER(__new__): Modules/_ctypes/callproc.c:1643
[issue19515] Share duplicated _Py_IDENTIFIER identifiers in C code
STINNER Victor added the comment: _Py_IDENTIFIER(__delitem__): Objects/typeobject.c:5132 _Py_IDENTIFIER(__delitem__): Objects/typeobject.c:5183 I moved this one to #19514. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19515 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
