The GitHub Actions job "CI" on tvm-ffi.git/pyobject has failed. Run started by GitHub user cyx-6 (triggered by cyx-6).
Head commit for run: 3374857de9f572abd3645987a057e7559c15a5ef / Yaxing Cai <[email protected]> [FEAT][Python] Tie Python wrapper lifetime to underlying C++ FFI object Make `a.x is a.x`, `id(a.x)` stable across drop+refetch, and `f(x) is x` for FFI returns by attaching a 16-byte tagged-pointer header to every Object allocation. Implements the PyObjectTying design. Design: - Two-layer custom-allocator hook in core libtvm_ffi.so: `TVMFFIObjectAllocHeader { delete_space }`, `TVMFFICustomAllocator { allocate, context }`, plus `TVMFFIGetCustomAllocator` / `TVMFFISetCustomAllocator`. libtvm_ffi installs a builtin default at registry construction so every Object always carries the 8-byte base header. The Python Cython module overrides the global default at module load with `TVMFFIPyAllocate`, which prepends a 16-byte `PyCustomAllocHeader` encoding the Python wrapper binding. `make_object` / `make_inplace_array_object` / `PyClassDeleter` and the Rust `ObjectArc::new[_with_extra_items]` paths all funnel through the registry, so Python-defined types and Rust-allocated objects share the layout and lifetime semantics. - Tagged-pointer state machine concentrated in `tvm_ffi_python_helpers.h`. `PyCustomAllocHeader` is a single `uintptr_t tagged_wrapper` field plus the base header (16 B total). Low bit of `tagged_wrapper` is the "live" tag, encoding three states: Init tagged_wrapper == 0 : never wrapped Active tagged_wrapper == (uintptr_t)w | 1 : live wrapper at w Inactive tagged_wrapper == (uintptr_t)w : preserved bytes at w CPython PyObjects are >= 8-byte aligned (PyObject_HEAD = 16 B), so bit 0 is always free for the tag. Four helpers expose the transitions: `TVMFFIPyTryGetAttachedPyObject`, `TVMFFIPyAttachPyObject`, `TVMFFIPyDetachPyObject`, `TVMFFIPyTPFinalize`. - Universal cache-on for `make_ret`. Every FFI return funnels through `make_ret_object`, which returns the canonical wrapper for a chandle when one exists. Combined with Inactive -> Active revival on the cached path, this delivers `a.x is a.x`, stable `id(a.x)` across drop+refetch, and `f(x) is x` when a function returns the same chandle it received. - Inactive state via Cython `def __del__` on `CObject`, mapping to `tp_finalize` (PEP 442). When other C++ holders keep the chandle alive past the wrapper's Python refcount hitting 0, clear the live bit (keep the pointer), null `self.chandle` BEFORE DECREFing the chandle, then `Py_INCREF(self)` to resurrect. Wrapper memory survives at the same address until revive or chandle death. Dropped the Py_LIMITED_API / SABI shim and the `CYTHON_USE_TP_FINALIZE` define. - Shutdown guard. `g_tvm_ffi_python_alive` flag is set to 0 by an atexit handler registered at Cython module init; read by `TVMFFIPyDeleteSpace` before `PyGILState_Ensure` to avoid GIL acquire after Python finalization has begun. Inactive wrapper bytes on chandles still alive at that point are intentionally leaked -- process is exiting. - Frontend-allocation detection by `delete_space` pointer comparison (`TVMFFIPyIsCanonical`) avoids a flag bit on `TVMFFIObject`. Pre-Python-init chandles (statically-initialized global functions in libtvm_ffi.so) carry only the base header; the Python side detects this and skips the binding install. `_move()` semantics: callback args alias the caller's wrapper under universal cache-on (one wrapper, one chandle ref) and FFI function returns of the same chandle alias the caller's wrapper. `_move()` is kept as an API: the rvalue setter on either caller or callback side eager-detaches the canonical-wrapper binding before the C++ AnyViewToOwnedAny transfer nulls the source chandle. `test_function.py::test_rvalue_ref` is refactored for the new aliasing-aware use_count expectations. Tests: full Python suite passes (2332 passed, 19 skipped, 2 xfailed); Rust suite passes (101 tests). New `tests/python/test_pyobject_tying.py` covers Active/Inactive transitions, cache-on aliasing, `_move()` under cache-on, pickle stress, threading stress, GC integration with the Inactive state, multi-chandle isolation, and the weakref limitation. TODO carried in `function.pxi::_get_global_func`: name-keyed dict cache for static-init Function id-stability. Most registry-resident Functions are allocated at C++ static init (before the Python allocator is registered) so their chandle has only the base header -- make_ret_object's cache can't reach them. Deferred to keep this change small. Report URL: https://github.com/apache/tvm-ffi/actions/runs/26518142423 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
