pisarev opened a new issue, #645:
URL: https://github.com/apache/tvm-ffi/issues/645
## Summary
The functors `tvm::ffi::StructuralHash` and `tvm::ffi::StructuralEqual`
disagree about tensor (NDArray) content:
- `StructuralHash::operator()` hashes the tensor content
(`skip_tensor_content = false`).
- `StructuralEqual::operator()` ignores the tensor content
(`skip_tensor_content = true`).
They are used together as the hash and key-equal of the `std::unordered_map`
that de-duplicates constants during Relax VM codegen (`const_dedup_map_` in
`src/relax/backend/vm/exec_builder.cc`, in the apache/tvm repo). Because hash
and equal disagree, the map's invariant `equal(a, b) => hash(a) == hash(b)`
does not hold, and two different constants with the same shape and dtype can be
merged into one. A downstream op (reshape/conv) then gets the wrong constant,
and inference returns wrong numbers with no error or warning.
The effect is platform dependent (I hit it on Windows/MSVC, while
Linux/libstdc++ stays correct for the same model), but the defect itself is
present on every platform.
## Root cause
`include/tvm/ffi/extra/structural_equal.h`:
```cpp
TVM_FFI_INLINE bool operator()(const Any& lhs, const Any& rhs) const {
return Equal(lhs, rhs, false, true); // skip_tensor_content = true
}
```
`include/tvm/ffi/extra/structural_hash.h`:
```cpp
TVM_FFI_INLINE uint64_t operator()(const Any& value) const { return
Hash(value); }
// Hash(value, map_free_vars = false, skip_tensor_content = false) ->
content is hashed
```
For two tensors `a`, `b` with equal shape/dtype but different data:
- `StructuralHash{}(a) != StructuralHash{}(b)` (content is hashed)
- `StructuralEqual{}(a, b) == true` (content is ignored)
In `unordered_map<Any, Index, StructuralHash, StructuralEqual>`, when
`hash(a) % bucket_count == hash(b) % bucket_count` (a bucket collision, which
becomes likely once the map holds a few dozen entries), `find(b)` scans that
bucket, calls `StructuralEqual(a, b)`, gets `true`, and treats `b` as a
duplicate of `a`. The emitted bytecode then references `a`'s constant index
where it should reference `b`.
## Why it shows up on one platform and not another
The hash values are the same on both platforms. The bucket count is not:
MSVC's `std::unordered_map` and libstdc++'s grow their bucket counts on
different schedules, so `hash % bucket_count` differs. A pair of constants that
collides into the same bucket under one STL lands in separate buckets under the
other. For a given model this is chance, so the same source produces wrong
output under MSVC and correct output under libstdc++. A different model, more
constants, or a different STL can surface it anywhere.
## Reproduction
Functor level, with any two same-shape, different-data CPU tensors `a`
(zeros) and `b` (ones), shape `(4,)` float32:
```cpp
#include <tvm/ffi/extra/structural_equal.h>
#include <tvm/ffi/extra/structural_hash.h>
tvm::ffi::StructuralEqual eq;
tvm::ffi::StructuralHash h;
CHECK(h(a) != h(b)); // holds: hash includes content
CHECK(eq(a, b) == false); // fails: equal returns true (content ignored)
```
End to end: compile any Relax module for target `llvm` that has a few dozen
distinct weight constants of the same shape (for example a YOLO11n detector
imported with `relax.frontend.onnx.from_onnx`). Under MSVC the compiled module
returns garbage (wrong class, large diff against onnxruntime); under libstdc++
it is correct. Diffing `ex.as_text()` of the two builds shows a single
differing line: a `vm.builtin.reshape` reads constant `c[14]` (MSVC, the
falsely merged one) instead of `c[59]` (libstdc++), and the MSVC executable has
one fewer constant in its pool.
## Fix
Make the `StructuralEqual` functor agree with the `StructuralHash` functor
and compare content:
```cpp
// structural_equal.h
TVM_FFI_INLINE bool operator()(const Any& lhs, const Any& rhs) const {
return Equal(lhs, rhs, false, false); // was: false, true
}
```
With this change, four models that I convert through the ONNX frontend
(YOLO11n det, YOLO11s cls, PP-OCRv5 det, PP-OCRv5 rec) match onnxruntime to
within floating point on a Windows build (MSVC 19.44, LLVM 18.1.8); before the
change, det and the PP-OCR models were off by 6 to 61 percent relative error.
Constants that are genuinely equal are still de-duplicated (their content
hashes equal and compares equal).
If skipping tensor content during equality is wanted for a specific
structural comparison, it can be requested through the static
`StructuralEqual::Equal(..., /*skip_tensor_content=*/true)`. The functor used
inside hash maps has to stay consistent with the `StructuralHash` functor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
