https://github.com/python/cpython/commit/040583387bd659be02b867c31e19cb72978e0ff6
commit: 040583387bd659be02b867c31e19cb72978e0ff6
branch: 3.12
author: Miss Islington (bot) <31488909+miss-isling...@users.noreply.github.com>
committer: encukou <encu...@gmail.com>
date: 2025-03-17T16:18:14+01:00
summary:
[3.12] gh-129675: Update documentation for tp_basicsize & tp_itemsize
(GH-129850) (GH-131078)
- Add alignment requirement
- Mention that ob_size is unreliable if you don't control it
- Add some links for context
- basicsize should include the base type in generaly not just PyObject
- suggest Py_SIZE & Py_SET_SIZE
This adds a “by-the-way” link to `PyObject_New`, which shouldn't be
used for GC types. In order to be comfortable linking to it, I also
add a link to `PyObject_GC_New` from its docs. And the same for
`*Var` variants, while I'm here.
(cherry picked from commit ad0f618ab3eb1f26f8830a863aaf7bb70835c00d)
Co-authored-by: Petr Viktorin <encu...@gmail.com>
files:
M Doc/c-api/allocation.rst
M Doc/c-api/type.rst
M Doc/c-api/typeobj.rst
diff --git a/Doc/c-api/allocation.rst b/Doc/c-api/allocation.rst
index e6ff40ab46e7c2..0deb632f0df661 100644
--- a/Doc/c-api/allocation.rst
+++ b/Doc/c-api/allocation.rst
@@ -35,6 +35,10 @@ Allocating Objects on the Heap
The size of the memory allocation is determined from the
:c:member:`~PyTypeObject.tp_basicsize` field of the type object.
+ Note that this function is unsuitable if *typeobj* has
+ :c:macro:`Py_TPFLAGS_HAVE_GC` set. For such objects,
+ use :c:func:`PyObject_GC_New` instead.
+
.. c:macro:: PyObject_NewVar(TYPE, typeobj, size)
@@ -49,6 +53,10 @@ Allocating Objects on the Heap
fields into the same allocation decreases the number of allocations,
improving the memory management efficiency.
+ Note that this function is unsuitable if *typeobj* has
+ :c:macro:`Py_TPFLAGS_HAVE_GC` set. For such objects,
+ use :c:func:`PyObject_GC_NewVar` instead.
+
.. c:function:: void PyObject_Del(void *op)
diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst
index 1564fa94efa9bf..241450a7d8d7d8 100644
--- a/Doc/c-api/type.rst
+++ b/Doc/c-api/type.rst
@@ -397,6 +397,9 @@ The following functions and structs are used to create
class need *in addition* to the superclass.
Use :c:func:`PyObject_GetTypeData` to get a pointer to subclass-specific
memory reserved this way.
+ For negative :c:member:`!basicsize`, Python will insert padding when
+ needed to meet :c:member:`~PyTypeObject.tp_basicsize`'s alignment
+ requirements.
.. versionchanged:: 3.12
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index a7ec665c340b3f..ddcbe91d487e63 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -559,6 +559,9 @@ PyVarObject Slots
initialized to zero. For :ref:`dynamically allocated type objects
<heap-types>`, this field has a special internal meaning.
+ This field should be accessed using the :c:func:`Py_SIZE()` and
+ :c:func:`Py_SET_SIZE()` macros.
+
**Inheritance:**
This field is not inherited by subtypes.
@@ -609,47 +612,86 @@ and :c:data:`PyType_Type` effectively act as defaults.)
.. c:member:: Py_ssize_t PyTypeObject.tp_basicsize
- Py_ssize_t PyTypeObject.tp_itemsize
+ Py_ssize_t PyTypeObject.tp_itemsize
These fields allow calculating the size in bytes of instances of the type.
There are two kinds of types: types with fixed-length instances have a zero
- :c:member:`~PyTypeObject.tp_itemsize` field, types with variable-length
instances have a non-zero
- :c:member:`~PyTypeObject.tp_itemsize` field. For a type with fixed-length
instances, all
- instances have the same size, given in
:c:member:`~PyTypeObject.tp_basicsize`.
+ :c:member:`!tp_itemsize` field, types with variable-length instances have a
non-zero
+ :c:member:`!tp_itemsize` field. For a type with fixed-length instances, all
+ instances have the same size, given in :c:member:`!tp_basicsize`.
+ (Exceptions to this rule can be made using
+ :c:func:`PyUnstable_Object_GC_NewWithExtraData`.)
For a type with variable-length instances, the instances must have an
- :c:member:`~PyVarObject.ob_size` field, and the instance size is
:c:member:`~PyTypeObject.tp_basicsize` plus N
- times :c:member:`~PyTypeObject.tp_itemsize`, where N is the "length" of the
object. The value of
- N is typically stored in the instance's :c:member:`~PyVarObject.ob_size`
field. There are
- exceptions: for example, ints use a negative
:c:member:`~PyVarObject.ob_size` to indicate a
- negative number, and N is ``abs(ob_size)`` there. Also, the presence of an
- :c:member:`~PyVarObject.ob_size` field in the instance layout doesn't mean
that the instance
- structure is variable-length (for example, the structure for the list type
has
- fixed-length instances, yet those instances have a meaningful
:c:member:`~PyVarObject.ob_size`
- field).
-
- The basic size includes the fields in the instance declared by the macro
- :c:macro:`PyObject_HEAD` or :c:macro:`PyObject_VAR_HEAD` (whichever is used
to
- declare the instance struct) and this in turn includes the
:c:member:`~PyObject._ob_prev` and
- :c:member:`~PyObject._ob_next` fields if they are present. This means that
the only correct
- way to get an initializer for the :c:member:`~PyTypeObject.tp_basicsize` is
to use the
- ``sizeof`` operator on the struct used to declare the instance layout.
- The basic size does not include the GC header size.
+ :c:member:`~PyVarObject.ob_size` field, and the instance size is
+ :c:member:`!tp_basicsize` plus N times :c:member:`!tp_itemsize`,
+ where N is the "length" of the object.
+
+ Functions like :c:func:`PyObject_NewVar` will take the value of N as an
+ argument, and store in the instance's :c:member:`~PyVarObject.ob_size`
field.
+ Note that the :c:member:`~PyVarObject.ob_size` field may later be used for
+ other purposes. For example, :py:type:`int` instances use the bits of
+ :c:member:`~PyVarObject.ob_size` in an implementation-defined
+ way; the underlying storage and its size should be acessed using
+ :c:func:`PyLong_Export`.
+
+ .. note::
- A note about alignment: if the variable items require a particular
alignment,
- this should be taken care of by the value of
:c:member:`~PyTypeObject.tp_basicsize`. Example:
- suppose a type implements an array of ``double``.
:c:member:`~PyTypeObject.tp_itemsize` is
- ``sizeof(double)``. It is the programmer's responsibility that
- :c:member:`~PyTypeObject.tp_basicsize` is a multiple of ``sizeof(double)``
(assuming this is the
- alignment requirement for ``double``).
+ The :c:member:`~PyVarObject.ob_size` field should be accessed using
+ the :c:func:`Py_SIZE()` and :c:func:`Py_SET_SIZE()` macros.
- For any type with variable-length instances, this field must not be
``NULL``.
+ Also, the presence of an :c:member:`~PyVarObject.ob_size` field in the
+ instance layout doesn't mean that the instance structure is variable-length.
+ For example, the :py:type:`list` type has fixed-length instances, yet those
+ instances have a :c:member:`~PyVarObject.ob_size` field.
+ (As with :py:type:`int`, avoid reading lists' :c:member:`!ob_size` directly.
+ Call :c:func:`PyList_Size` instead.)
+
+ The :c:member:`!tp_basicsize` includes size needed for data of the type's
+ :c:member:`~PyTypeObject.tp_base`, plus any extra data needed
+ by each instance.
+
+ The correct way to set :c:member:`!tp_basicsize` is to use the
+ ``sizeof`` operator on the struct used to declare the instance layout.
+ This struct must include the struct used to declare the base type.
+ In other words, :c:member:`!tp_basicsize` must be greater than or equal
+ to the base's :c:member:`!tp_basicsize`.
+
+ Since every type is a subtype of :py:type:`object`, this struct must
+ include :c:type:`PyObject` or :c:type:`PyVarObject` (depending on
+ whether :c:member:`~PyVarObject.ob_size` should be included). These are
+ usually defined by the macro :c:macro:`PyObject_HEAD` or
+ :c:macro:`PyObject_VAR_HEAD`, respectively.
+
+ The basic size does not include the GC header size, as that header is not
+ part of :c:macro:`PyObject_HEAD`.
+
+ For cases where struct used to declare the base type is unknown,
+ see :c:member:`PyType_Spec.basicsize` and :c:func:`PyType_FromMetaclass`.
+
+ Notes about alignment:
+
+ - :c:member:`!tp_basicsize` must be a multiple of ``_Alignof(PyObject)``.
+ When using ``sizeof`` on a ``struct`` that includes
+ :c:macro:`PyObject_HEAD`, as recommended, the compiler ensures this.
+ When not using a C ``struct``, or when using compiler
+ extensions like ``__attribute__((packed))``, it is up to you.
+ - If the variable items require a particular alignment,
+ :c:member:`!tp_basicsize` and :c:member:`!tp_itemsize` must each be a
+ multiple of that alignment.
+ For example, if a type's variable part stores a ``double``, it is
+ your responsibility that both fields are a multiple of
+ ``_Alignof(double)``.
**Inheritance:**
- These fields are inherited separately by subtypes. If the base type has a
- non-zero :c:member:`~PyTypeObject.tp_itemsize`, it is generally not safe to
set
+ These fields are inherited separately by subtypes.
+ (That is, if the field is set to zero, :c:func:`PyType_Ready` will copy
+ the value from the base type, indicating that the instances do not
+ need additional storage.)
+
+ If the base type has a non-zero :c:member:`~PyTypeObject.tp_itemsize`, it
is generally not safe to set
:c:member:`~PyTypeObject.tp_itemsize` to a different non-zero value in a
subtype (though this
depends on the implementation of the base type).
_______________________________________________
Python-checkins mailing list -- python-checkins@python.org
To unsubscribe send an email to python-checkins-le...@python.org
https://mail.python.org/mailman3/lists/python-checkins.python.org/
Member address: arch...@mail-archive.com
