Le ven. 10 avr. 2020 à 22:00, Antoine Pitrou <solip...@pitrou.net> a écrit :
> > Examples of issues to make structures opaque:
> >
> > * ``PyGC_Head``: https://bugs.python.org/issue40241
> > * ``PyObject``: https://bugs.python.org/issue39573
> > * ``PyTypeObject``: https://bugs.python.org/issue40170
>
> How do you keep fast type checking such as PyTuple_Check() if extension
> code doesn't have access e.g. to tp_flags?
Hum. I should clarify that we have the choice to not having any impact
on performance for the regular runtime: only use opaque function for
the "new" runtime. It's exactly what is already done with the
Py_LIMITED_API. Concrete example:
static inline int
PyType_HasFeature(PyTypeObject *type, unsigned long feature) {
#ifdef Py_LIMITED_API
return ((PyType_GetFlags(type) & feature) != 0);
#else
return ((type->tp_flags & feature) != 0);
#endif
}
The Py_LIMITED_API goes through PyType_GetFlags() function call,
otherwise PyTypeObject.tp_flags field is accessed directly.
I recently modified this function to:
static inline int
PyType_HasFeature(PyTypeObject *type, unsigned long feature) {
return ((PyType_GetFlags(type) & feature) != 0);
}
I consider that checking a type is not performance critical and so I
chose to have the same implementation for everyone. If someone sees
that it's major performance overhead, we can visit this choice and
reintroduce an #ifdef.
It's more a practical issue about the maintenance of two flavors of
Python in the same code base. Do you want to have two implementations
of each function? Or is it possible to have a single implementation
for some functions?
I suggest to reduce the code duplication and accept a performance
overhead when it's small enough.
> > O(1) bytearray to bytes conversion
> > ..................................
> >
> > Convert bytearray to bytes without memory copy.
> > (...)
>
> If that's desirable (I'm not sure it is), (...)
Hum, maybe I should clarify the whole "New optimized CPython runtime"
section. The list of optimizations are not optimizations that must be
implemented. There are more examples of optimizations which becomes
possible to implement, or at least easier to implement, once the C API
will be fixed.
I'm not sure that "bytearray to bytes conversion" is performance
bottleneck. It's just that such optimization is easier to explain that
other more complex optimizations ;-)
The intent of this PEP is not to design a faster CPython, but to show
that reworking the C API allows to implement such faster CPython.
> > Fork and "Copy-on-Read" problem
> > ...............................
> >
> > Solve the "Copy on read" problem with fork: store reference counter
> > outside ``PyObject``.
>
> Nowadays it is strongly recommended to use multiprocessing with the
> "forkserver" start method:
> https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
I understood that the Instagram workload is to load heavy data only
once, and fork later. I 'm not sure that forkserver fits such
workload.
> > One solution for that would be to store reference counters outside
> > ``PyObject``. For example, in a separated hash table (pointer to
> > reference counter). Changing ``PyObject`` structures requires that C
> > extensions don't access them directly.
>
> You're planning to introduce a large overhead for each reference
> count lookup just to satisfy a rather niche use case? CPython
> probably does millions of reference counts per second.
Sorry, again, I'm not proposing to move ob_refcnt outside PyObject for
everyone. The intent is to show that it becomes possible to do if you
have a very specific use case where it would be more efficient.
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/JPDCY4BVKZ2USN5MWFWUQ2JVBQWEAYGM/
Code of Conduct: http://python.org/psf/codeofconduct/
