tugot17 commented on PR #644:
URL: https://github.com/apache/tvm-ffi/pull/644#issuecomment-4809631965
Yeah, fair on cxx11 — for official wheels it's basically always true now, so
that part's really just a guard for custom builds. Happy to drop it and key
purely on `torch.__version__` if you'd rather keep this minimal.
The thing that actually bit us wasn't the abi flag though, it was the
version getting dropped from the name. The key is only
`torch{major}{minor}-{device}`, so everything past major.minor disappears — the
patch version and the `+cu124` / `+rocm7.2` build tag. We run sglang in
containers on ROCm with the host `~/.cache/tvm-ffi` mounted in, and when we
moved between two images (different rocm/torch builds, both torch 2.9) they
both resolved to the same `libtorch_c_dlpack_addon_torch29-rocm.so`. So the
addon compiled against the old image got loaded into the new container, the ABI
didn't match, and instead of erroring it just produced garbage tokens (plus the
odd HSA fault) at decode. Took us way too long to trace it back to the cache.
Sticking `torch.__version__` in the name makes the stale entry miss and
recompile, which fixes it.
On backward compat: the only effect is that one rebuild of the addon the
first time after an upgrade, which is kind of the point here since the whole
problem is a stale build being silently reused. Nothing breaks at runtime, the
old files just sit there orphaned. Happy to add a small cleanup of the old
`…torch{maj}{min}-{device}.so` name too if you want to avoid the clutter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
