1 .. highlight:: c
2 
3 .. _common-structs:
4 
5 Common Object Structures
6 ========================
7 
8 There are a large number of structures which are used in the definition of
9 object types for Python.  This section describes these structures and how they
10 are used.
11 
12 
13 Base object types and macros
14 ----------------------------
15 
16 All Python objects ultimately share a small number of fields at the beginning
17 of the object's representation in memory.  These are represented by the
18 :c:type:`PyObject` and :c:type:`PyVarObject` types, which are defined, in turn,
19 by the expansions of some macros also used, whether directly or indirectly, in
20 the definition of all other Python objects.
21 
22 
23 .. c:type:: PyObject
24 
25    All object types are extensions of this type.  This is a type which
26    contains the information Python needs to treat a pointer to an object as an
27    object.  In a normal "release" build, it contains only the object's
28    reference count and a pointer to the corresponding type object.
29    Nothing is actually declared to be a :c:type:`PyObject`, but every pointer
30    to a Python object can be cast to a :c:expr:`PyObject*`.  Access to the
31    members must be done by using the macros :c:macro:`Py_REFCNT` and
32    :c:macro:`Py_TYPE`.
33 
34 
35 .. c:type:: PyVarObject
36 
37    This is an extension of :c:type:`PyObject` that adds the :attr:`ob_size`
38    field.  This is only used for objects that have some notion of *length*.
39    This type does not often appear in the Python/C API.
40    Access to the members must be done by using the macros
41    :c:macro:`Py_REFCNT`, :c:macro:`Py_TYPE`, and :c:macro:`Py_SIZE`.
42 
43 
44 .. c:macro:: PyObject_HEAD
45 
46    This is a macro used when declaring new types which represent objects
47    without a varying length.  The PyObject_HEAD macro expands to::
48 
49       PyObject ob_base;
50 
51    See documentation of :c:type:`PyObject` above.
52 
53 
54 .. c:macro:: PyObject_VAR_HEAD
55 
56    This is a macro used when declaring new types which represent objects
57    with a length that varies from instance to instance.
58    The PyObject_VAR_HEAD macro expands to::
59 
60       PyVarObject ob_base;
61 
62    See documentation of :c:type:`PyVarObject` above.
63 
64 
65 .. c:function:: int Py_Is(PyObject *x, PyObject *y)
66 
67    Test if the *x* object is the *y* object, the same as ``x is y`` in Python.
68 
69    .. versionadded:: 3.10
70 
71 
72 .. c:function:: int Py_IsNone(PyObject *x)
73 
74    Test if an object is the ``None`` singleton,
75    the same as ``x is None`` in Python.
76 
77    .. versionadded:: 3.10
78 
79 
80 .. c:function:: int Py_IsTrue(PyObject *x)
81 
82    Test if an object is the ``True`` singleton,
83    the same as ``x is True`` in Python.
84 
85    .. versionadded:: 3.10
86 
87 
88 .. c:function:: int Py_IsFalse(PyObject *x)
89 
90    Test if an object is the ``False`` singleton,
91    the same as ``x is False`` in Python.
92 
93    .. versionadded:: 3.10
94 
95 
96 .. c:function:: PyTypeObject* Py_TYPE(PyObject *o)
97 
98    Get the type of the Python object *o*.
99 
100    Return a :term:`borrowed reference`.
101 
102    Use the :c:func:`Py_SET_TYPE` function to set an object type.
103 
104    .. versionchanged:: 3.11
105       :c:func:`Py_TYPE()` is changed to an inline static function.
106       The parameter type is no longer :c:expr:`const PyObject*`.
107 
108 
109 .. c:function:: int Py_IS_TYPE(PyObject *o, PyTypeObject *type)
110 
111    Return non-zero if the object *o* type is *type*. Return zero otherwise.
112    Equivalent to: ``Py_TYPE(o) == type``.
113 
114    .. versionadded:: 3.9
115 
116 
117 .. c:function:: void Py_SET_TYPE(PyObject *o, PyTypeObject *type)
118 
119    Set the object *o* type to *type*.
120 
121    .. versionadded:: 3.9
122 
123 
124 .. c:function:: Py_ssize_t Py_REFCNT(PyObject *o)
125 
126    Get the reference count of the Python object *o*.
127 
128    Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count.
129 
130    .. versionchanged:: 3.11
131       The parameter type is no longer :c:expr:`const PyObject*`.
132 
133    .. versionchanged:: 3.10
134       :c:func:`Py_REFCNT()` is changed to the inline static function.
135 
136 
137 .. c:function:: void Py_SET_REFCNT(PyObject *o, Py_ssize_t refcnt)
138 
139    Set the object *o* reference counter to *refcnt*.
140 
141    .. versionadded:: 3.9
142 
143 
144 .. c:function:: Py_ssize_t Py_SIZE(PyVarObject *o)
145 
146    Get the size of the Python object *o*.
147 
148    Use the :c:func:`Py_SET_SIZE` function to set an object size.
149 
150    .. versionchanged:: 3.11
151       :c:func:`Py_SIZE()` is changed to an inline static function.
152       The parameter type is no longer :c:expr:`const PyVarObject*`.
153 
154 
155 .. c:function:: void Py_SET_SIZE(PyVarObject *o, Py_ssize_t size)
156 
157    Set the object *o* size to *size*.
158 
159    .. versionadded:: 3.9
160 
161 
162 .. c:macro:: PyObject_HEAD_INIT(type)
163 
164    This is a macro which expands to initialization values for a new
165    :c:type:`PyObject` type.  This macro expands to::
166 
167       _PyObject_EXTRA_INIT
168       1, type,
169 
170 
171 .. c:macro:: PyVarObject_HEAD_INIT(type, size)
172 
173    This is a macro which expands to initialization values for a new
174    :c:type:`PyVarObject` type, including the :attr:`ob_size` field.
175    This macro expands to::
176 
177       _PyObject_EXTRA_INIT
178       1, type, size,
179 
180 
181 Implementing functions and methods
182 ----------------------------------
183 
184 .. c:type:: PyCFunction
185 
186    Type of the functions used to implement most Python callables in C.
187    Functions of this type take two :c:expr:`PyObject*` parameters and return
188    one such value.  If the return value is ``NULL``, an exception shall have
189    been set.  If not ``NULL``, the return value is interpreted as the return
190    value of the function as exposed in Python.  The function must return a new
191    reference.
192 
193    The function signature is::
194 
195       PyObject *PyCFunction(PyObject *self,
196                             PyObject *args);
197 
198 .. c:type:: PyCFunctionWithKeywords
199 
200    Type of the functions used to implement Python callables in C
201    with signature :const:`METH_VARARGS | METH_KEYWORDS`.
202    The function signature is::
203 
204       PyObject *PyCFunctionWithKeywords(PyObject *self,
205                                         PyObject *args,
206                                         PyObject *kwargs);
207 
208 
209 .. c:type:: _PyCFunctionFast
210 
211    Type of the functions used to implement Python callables in C
212    with signature :const:`METH_FASTCALL`.
213    The function signature is::
214 
215       PyObject *_PyCFunctionFast(PyObject *self,
216                                  PyObject *const *args,
217                                  Py_ssize_t nargs);
218 
219 .. c:type:: _PyCFunctionFastWithKeywords
220 
221    Type of the functions used to implement Python callables in C
222    with signature :const:`METH_FASTCALL | METH_KEYWORDS`.
223    The function signature is::
224 
225       PyObject *_PyCFunctionFastWithKeywords(PyObject *self,
226                                              PyObject *const *args,
227                                              Py_ssize_t nargs,
228                                              PyObject *kwnames);
229 
230 .. c:type:: PyCMethod
231 
232    Type of the functions used to implement Python callables in C
233    with signature :const:`METH_METHOD | METH_FASTCALL | METH_KEYWORDS`.
234    The function signature is::
235 
236       PyObject *PyCMethod(PyObject *self,
237                           PyTypeObject *defining_class,
238                           PyObject *const *args,
239                           Py_ssize_t nargs,
240                           PyObject *kwnames)
241 
242    .. versionadded:: 3.9
243 
244 
245 .. c:type:: PyMethodDef
246 
247    Structure used to describe a method of an extension type.  This structure has
248    four fields:
249 
250    .. c:member:: const char* ml_name
251 
252       name of the method
253 
254    .. c:member:: PyCFunction ml_meth
255 
256       pointer to the C implementation
257 
258    .. c:member:: int ml_flags
259 
260       flags bits indicating how the call should be constructed
261 
262    .. c:member:: const char* ml_doc
263 
264       points to the contents of the docstring
265 
266 The :c:member:`ml_meth` is a C function pointer.  The functions may be of different
267 types, but they always return :c:expr:`PyObject*`.  If the function is not of
268 the :c:type:`PyCFunction`, the compiler will require a cast in the method table.
269 Even though :c:type:`PyCFunction` defines the first parameter as
270 :c:expr:`PyObject*`, it is common that the method implementation uses the
271 specific C type of the *self* object.
272 
273 The :c:member:`ml_flags` field is a bitfield which can include the following flags.
274 The individual flags indicate either a calling convention or a binding
275 convention.
276 
277 There are these calling conventions:
278 
279 .. data:: METH_VARARGS
280 
281    This is the typical calling convention, where the methods have the type
282    :c:type:`PyCFunction`. The function expects two :c:expr:`PyObject*` values.
283    The first one is the *self* object for methods; for module functions, it is
284    the module object.  The second parameter (often called *args*) is a tuple
285    object representing all arguments. This parameter is typically processed
286    using :c:func:`PyArg_ParseTuple` or :c:func:`PyArg_UnpackTuple`.
287 
288 
289 .. data:: METH_VARARGS | METH_KEYWORDS
290 
291    Methods with these flags must be of type :c:type:`PyCFunctionWithKeywords`.
292    The function expects three parameters: *self*, *args*, *kwargs* where
293    *kwargs* is a dictionary of all the keyword arguments or possibly ``NULL``
294    if there are no keyword arguments.  The parameters are typically processed
295    using :c:func:`PyArg_ParseTupleAndKeywords`.
296 
297 
298 .. data:: METH_FASTCALL
299 
300    Fast calling convention supporting only positional arguments.
301    The methods have the type :c:type:`_PyCFunctionFast`.
302    The first parameter is *self*, the second parameter is a C array
303    of :c:expr:`PyObject*` values indicating the arguments and the third
304    parameter is the number of arguments (the length of the array).
305 
306    .. versionadded:: 3.7
307 
308    .. versionchanged:: 3.10
309 
310       ``METH_FASTCALL`` is now part of the stable ABI.
311 
312 
313 .. data:: METH_FASTCALL | METH_KEYWORDS
314 
315    Extension of :const:`METH_FASTCALL` supporting also keyword arguments,
316    with methods of type :c:type:`_PyCFunctionFastWithKeywords`.
317    Keyword arguments are passed the same way as in the
318    :ref:`vectorcall protocol <vectorcall>`:
319    there is an additional fourth :c:expr:`PyObject*` parameter
320    which is a tuple representing the names of the keyword arguments
321    (which are guaranteed to be strings)
322    or possibly ``NULL`` if there are no keywords.  The values of the keyword
323    arguments are stored in the *args* array, after the positional arguments.
324 
325    .. versionadded:: 3.7
326 
327 
328 .. data:: METH_METHOD | METH_FASTCALL | METH_KEYWORDS
329 
330    Extension of :const:`METH_FASTCALL | METH_KEYWORDS` supporting the *defining
331    class*, that is, the class that contains the method in question.
332    The defining class might be a superclass of ``Py_TYPE(self)``.
333 
334    The method needs to be of type :c:type:`PyCMethod`, the same as for
335    ``METH_FASTCALL | METH_KEYWORDS`` with ``defining_class`` argument added after
336    ``self``.
337 
338    .. versionadded:: 3.9
339 
340 
341 .. data:: METH_NOARGS
342 
343    Methods without parameters don't need to check whether arguments are given if
344    they are listed with the :const:`METH_NOARGS` flag.  They need to be of type
345    :c:type:`PyCFunction`.  The first parameter is typically named *self* and will
346    hold a reference to the module or object instance.  In all cases the second
347    parameter will be ``NULL``.
348 
349    The function must have 2 parameters. Since the second parameter is unused,
350    :c:macro:`Py_UNUSED` can be used to prevent a compiler warning.
351 
352 
353 .. data:: METH_O
354 
355    Methods with a single object argument can be listed with the :const:`METH_O`
356    flag, instead of invoking :c:func:`PyArg_ParseTuple` with a ``"O"`` argument.
357    They have the type :c:type:`PyCFunction`, with the *self* parameter, and a
358    :c:expr:`PyObject*` parameter representing the single argument.
359 
360 
361 These two constants are not used to indicate the calling convention but the
362 binding when use with methods of classes.  These may not be used for functions
363 defined for modules.  At most one of these flags may be set for any given
364 method.
365 
366 
367 .. data:: METH_CLASS
368 
369    .. index:: pair: built-in function; classmethod
370 
371    The method will be passed the type object as the first parameter rather
372    than an instance of the type.  This is used to create *class methods*,
373    similar to what is created when using the :func:`classmethod` built-in
374    function.
375 
376 
377 .. data:: METH_STATIC
378 
379    .. index:: pair: built-in function; staticmethod
380 
381    The method will be passed ``NULL`` as the first parameter rather than an
382    instance of the type.  This is used to create *static methods*, similar to
383    what is created when using the :func:`staticmethod` built-in function.
384 
385 One other constant controls whether a method is loaded in place of another
386 definition with the same method name.
387 
388 
389 .. data:: METH_COEXIST
390 
391    The method will be loaded in place of existing definitions.  Without
392    *METH_COEXIST*, the default is to skip repeated definitions.  Since slot
393    wrappers are loaded before the method table, the existence of a
394    *sq_contains* slot, for example, would generate a wrapped method named
395    :meth:`__contains__` and preclude the loading of a corresponding
396    PyCFunction with the same name.  With the flag defined, the PyCFunction
397    will be loaded in place of the wrapper object and will co-exist with the
398    slot.  This is helpful because calls to PyCFunctions are optimized more
399    than wrapper object calls.
400 
401 
402 Accessing attributes of extension types
403 ---------------------------------------
404 
405 .. c:type:: PyMemberDef
406 
407    Structure which describes an attribute of a type which corresponds to a C
408    struct member.  Its fields are:
409 
410    +------------------+---------------+-------------------------------+
411    | Field            | C Type        | Meaning                       |
412    +==================+===============+===============================+
413    | :attr:`name`     | const char \* | name of the member            |
414    +------------------+---------------+-------------------------------+
415    | :attr:`!type`    | int           | the type of the member in the |
416    |                  |               | C struct                      |
417    +------------------+---------------+-------------------------------+
418    | :attr:`offset`   | Py_ssize_t    | the offset in bytes that the  |
419    |                  |               | member is located on the      |
420    |                  |               | type's object struct          |
421    +------------------+---------------+-------------------------------+
422    | :attr:`flags`    | int           | flag bits indicating if the   |
423    |                  |               | field should be read-only or  |
424    |                  |               | writable                      |
425    +------------------+---------------+-------------------------------+
426    | :attr:`doc`      | const char \* | points to the contents of the |
427    |                  |               | docstring                     |
428    +------------------+---------------+-------------------------------+
429 
430    :attr:`!type` can be one of many ``T_`` macros corresponding to various C
431    types.  When the member is accessed in Python, it will be converted to the
432    equivalent Python type.
433 
434    =============== ==================
435    Macro name      C type
436    =============== ==================
437    T_SHORT         short
438    T_INT           int
439    T_LONG          long
440    T_FLOAT         float
441    T_DOUBLE        double
442    T_STRING        const char \*
443    T_OBJECT        PyObject \*
444    T_OBJECT_EX     PyObject \*
445    T_CHAR          char
446    T_BYTE          char
447    T_UBYTE         unsigned char
448    T_UINT          unsigned int
449    T_USHORT        unsigned short
450    T_ULONG         unsigned long
451    T_BOOL          char
452    T_LONGLONG      long long
453    T_ULONGLONG     unsigned long long
454    T_PYSSIZET      Py_ssize_t
455    =============== ==================
456 
457    :c:macro:`T_OBJECT` and :c:macro:`T_OBJECT_EX` differ in that
458    :c:macro:`T_OBJECT` returns ``None`` if the member is ``NULL`` and
459    :c:macro:`T_OBJECT_EX` raises an :exc:`AttributeError`.  Try to use
460    :c:macro:`T_OBJECT_EX` over :c:macro:`T_OBJECT` because :c:macro:`T_OBJECT_EX`
461    handles use of the :keyword:`del` statement on that attribute more correctly
462    than :c:macro:`T_OBJECT`.
463 
464    :attr:`flags` can be ``0`` for write and read access or :c:macro:`READONLY` for
465    read-only access.  Using :c:macro:`T_STRING` for :attr:`type` implies
466    :c:macro:`READONLY`.  :c:macro:`T_STRING` data is interpreted as UTF-8.
467    Only :c:macro:`T_OBJECT` and :c:macro:`T_OBJECT_EX`
468    members can be deleted.  (They are set to ``NULL``).
469 
470    .. _pymemberdef-offsets:
471 
472    Heap allocated types (created using :c:func:`PyType_FromSpec` or similar),
473    ``PyMemberDef`` may contain definitions for the special members
474    ``__dictoffset__``, ``__weaklistoffset__`` and ``__vectorcalloffset__``,
475    corresponding to
476    :c:member:`~PyTypeObject.tp_dictoffset`,
477    :c:member:`~PyTypeObject.tp_weaklistoffset` and
478    :c:member:`~PyTypeObject.tp_vectorcall_offset` in type objects.
479    These must be defined with ``T_PYSSIZET`` and ``READONLY``, for example::
480 
481       static PyMemberDef spam_type_members[] = {
482           {"__dictoffset__", T_PYSSIZET, offsetof(Spam_object, dict), READONLY},
483           {NULL}  /* Sentinel */
484       };
485 
486 
487 .. c:function:: PyObject* PyMember_GetOne(const char *obj_addr, struct PyMemberDef *m)
488 
489    Get an attribute belonging to the object at address *obj_addr*.  The
490    attribute is described by ``PyMemberDef`` *m*.  Returns ``NULL``
491    on error.
492 
493 
494 .. c:function:: int PyMember_SetOne(char *obj_addr, struct PyMemberDef *m, PyObject *o)
495 
496    Set an attribute belonging to the object at address *obj_addr* to object *o*.
497    The attribute to set is described by ``PyMemberDef`` *m*.  Returns ``0``
498    if successful and a negative value on failure.
499 
500 
501 .. c:type:: PyGetSetDef
502 
503    Structure to define property-like access for a type. See also description of
504    the :c:member:`PyTypeObject.tp_getset` slot.
505 
506    +-------------+------------------+-----------------------------------+
507    | Field       | C Type           | Meaning                           |
508    +=============+==================+===================================+
509    | name        | const char \*    | attribute name                    |
510    +-------------+------------------+-----------------------------------+
511    | get         | getter           | C function to get the attribute   |
512    +-------------+------------------+-----------------------------------+
513    | set         | setter           | optional C function to set or     |
514    |             |                  | delete the attribute, if omitted  |
515    |             |                  | the attribute is readonly         |
516    +-------------+------------------+-----------------------------------+
517    | doc         | const char \*    | optional docstring                |
518    +-------------+------------------+-----------------------------------+
519    | closure     | void \*          | optional function pointer,        |
520    |             |                  | providing additional data for     |
521    |             |                  | getter and setter                 |
522    +-------------+------------------+-----------------------------------+
523 
524    The ``get`` function takes one :c:expr:`PyObject*` parameter (the
525    instance) and a function pointer (the associated ``closure``)::
526 
527       typedef PyObject *(*getter)(PyObject *, void *);
528 
529    It should return a new reference on success or ``NULL`` with a set exception
530    on failure.
531 
532    ``set`` functions take two :c:expr:`PyObject*` parameters (the instance and
533    the value to be set) and a function pointer (the associated ``closure``)::
534 
535       typedef int (*setter)(PyObject *, PyObject *, void *);
536 
537    In case the attribute should be deleted the second parameter is ``NULL``.
538    Should return ``0`` on success or ``-1`` with a set exception on failure.
539