The GitHub Actions job "CI" on tvm-ffi.git/pyobject has failed. Run started by GitHub user cyx-6 (triggered by cyx-6).
Head commit for run: 3bcc044b001bd074204e5e18032052d46ba37043 / Yaxing Cai <[email protected]> [FEAT][Python] Tie Python wrapper lifetime to underlying C++ FFI object Make `a.x is a.x`, `id(a.x)` stable across drop+refetch, and `f(x) is x` for FFI returns by attaching a 16-byte custom-allocator header to every Object allocation. Implements the PyObjectTying design. Design: - Two-layer custom-allocator hook in core libtvm_ffi.so: `TVMFFIObjectAllocHeader { delete_space }`, `TVMFFICustomAllocator { allocate, context }`, plus `TVMFFIGetCustomAllocator` / `TVMFFISetCustomAllocator`. libtvm_ffi installs a builtin default at registry init, so every Object always carries the 8-byte base header and `TVMFFIGetCustomAllocator` never returns NULL. The Python Cython module overrides the global default at module load with `TVMFFIPyAllocate`, which prepends a 16-byte `PyCustomAllocHeader` encoding the Python wrapper binding. `make_object` / `make_inplace_array_object` / `PyClassDeleter` and the Rust `ObjectArc::new[_with_extra_items]` paths all funnel through the registry, so Python-defined types and Rust-allocated objects share the layout and lifetime semantics. - State machine concentrated in `tvm_ffi_python_helpers.h`. `PyCustomAllocHeader` is `{ PyObject* cached_pyobj; base; }` (16 B). The Active vs Inactive distinction lives on the wrapper's own `chandle` field, not on a tag bit, so the header layout stays pointer-equality clean: Detached: cached_pyobj == NULL (no wrapper) Active: cached_pyobj == self AND self.chandle == chandle (wrapper owns +1) Inactive: cached_pyobj == self AND self.chandle == NULL (preserved bytes) Four helpers expose the transitions: `TVMFFIPyTryGetAttachedPyObject`, `TVMFFIPyAttachPyObject`, `TVMFFIPyDetachPyObject`, `TVMFFIPyTPFinalize`. - Universal cache-on for `make_ret`. Every FFI return funnels through `make_ret_object`, which returns the canonical wrapper for a chandle when one exists. Combined with Inactive -> Active revival on the cached path, this delivers `a.x is a.x`, stable `id(a.x)` across drop+refetch, and `f(x) is x` when a function returns the same chandle it received. - Inactive state via Cython `def __del__` on `CObject`, mapping to `tp_finalize` (PEP 442). When other C++ holders keep the chandle alive past the wrapper's Python refcount hitting 0, `TVMFFIPyTPFinalize` nulls `self.chandle` BEFORE DECREFing the chandle, then `Py_IncRef(self)` to resurrect the PyObject. `cached_pyobj` is unchanged so a future re-fetch can rebind. Cython auto-generates `tp_dealloc`, which now enters with the `self.chandle == NULL` invariant established by `tp_finalize`. Dropped the Py_LIMITED_API / SABI shim and the `CYTHON_USE_TP_FINALIZE` define. - Shutdown guard. `TVMFFIPyMarkPythonFinalizing` is wired to atexit from Cython module init, flipping a `std::atomic<bool>` flag read by `TVMFFIPyDeleteSpace` (via `TVMFFIPyIsPythonAlive`) before `PyGILState_Ensure` to avoid GIL acquire after Python finalization has begun. Inactive wrapper bytes on chandles still alive at that point are intentionally leaked -- process is exiting. - Frontend-allocation detection by `delete_space` pointer comparison (`TVMFFIPyIsCanonical`) avoids a flag bit on `TVMFFIObject`. Pre-Python-init chandles (statically-initialized global functions in libtvm_ffi.so) carry only the base header; the Python side detects this and skips the binding install. `_move()` semantics: callback args alias the caller's wrapper under universal cache-on (one wrapper, one chandle ref) and FFI function returns of the same chandle alias the caller's wrapper. `_move()` is kept as an API: the rvalue setter on either caller or callback side eager-detaches the canonical-wrapper binding before the C++ AnyViewToOwnedAny transfer nulls the source chandle. `test_function.py::test_rvalue_ref` is refactored for the new aliasing-aware use_count expectations. Tests: full Python suite passes (2332 passed, 19 skipped, 2 xfailed); Rust suite passes. New `tests/python/test_pyobject_tying.py` covers Active/Inactive transitions, cache-on aliasing, `_move()` under cache-on, pickle stress, threading stress, GC integration with the Inactive state, multi-chandle isolation, and the weakref limitation. TODO carried in `function.pxi::_get_global_func`: name-keyed dict cache for static-init Function id-stability. Most registry-resident Functions are allocated at C++ static init (before the Python allocator is registered) so their chandle has only the base header -- make_ret_object's cache can't reach them. Deferred to keep this change small. Report URL: https://github.com/apache/tvm-ffi/actions/runs/26593163586 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
