siyiweigeHEW opened a new issue, #20182: URL: https://github.com/apache/tvm/issues/20182
### Expected behavior A valid ONNX `Scatter` model (opset 9/10) whose `indices`/`updates` are **broadcastable** to `data` (i.e. contain size-1 dims) should be imported by `tvm.relax.frontend.onnx.from_onnx` and produce the output defined by the ONNX spec: for each entry `idx` in `indices` (iterated over `indices`' own shape), `output[idx[:axis] + (indices[idx],) + idx[axis+1:]] = updates[idx]` (see [ONNX Scatter spec](https://github.com/onnx/onnx/blob/main/docs/Operators.md#Scatter)). Such models pass `onnx.checker` and run correctly in onnxruntime and the official ONNX numpy reference implementation. ### Actual behavior `from_onnx` imports the model **successfully** but produces **silently wrong numeric output** — no error is raised, the wrong values are simply computed. The frontend's `Scatter._impl_v9` (`python/tvm/relax/frontend/onnx/onnx_frontend.py:834`) passes `indices`/`updates` **as-is** to `relax.op.scatter_elements(data, indices, updates, axis)` without first broadcasting them to `data`'s shape. `relax.op.scatter_elements` (torch-`scatter_` semantics, expects shapes broadcastable to `data`) mishandles a smaller `indices` tensor and silently emits wrong results. Minimal case: `data: (2, 3, 4)`, `indices: (1, 3, 1)`, `updates: (1, 3, 1)`, `axis: 0`, opset 9. The 3 entries of `indices` (values `[1, 0, 0]`, iterated over `(0,0,0),(0,1,0),(0,2,0)`) must write `updates [100, 200, 300]` to `data[1,0,0]`, `data[0,1,0]`, `data[0,2,0]` respectively: - onnxruntime: `data[1,0,0]=100`, `data[0,1,0]=200`, `data[0,2,0]=300` (correct) - TVM: `data[1,0,0]=100`, `data[0,0,1]=200`, `data[0,0,2]=300` (wrong — `max|diff| = 298`) ### Environment - OS: Linux - TVM: v0.24.dev0 (main branch, commit `262c6d2e0`, built 2026-02-11) - Python: 3.11 - onnx: 1.20.1 - onnxruntime: 1.24.1 ### Steps to reproduce ```python """Minimal repro: legal ONNX Scatter (opset 9) with broadcastable indices/updates is imported by the TVM relax ONNX frontend but produces wrong numeric output, while onnxruntime is correct.""" import numpy as np import onnx, onnxruntime from onnx import helper, TensorProto import tvm from tvm import relax from tvm.relax.frontend.onnx import from_onnx def make_model(): D = helper.make_tensor_value_info("data", TensorProto.FLOAT, [2, 3, 4]) I = helper.make_tensor_value_info("indices", TensorProto.INT64, [1, 3, 1]) U = helper.make_tensor_value_info("updates", TensorProto.FLOAT, [1, 3, 1]) O = helper.make_tensor_value_info("output", TensorProto.FLOAT, [2, 3, 4]) node = helper.make_node("Scatter", ["data", "indices", "updates"], ["output"], axis=0) m = helper.make_model(helper.make_graph([node], "g", [D, I, U], [O]), opset_imports=[helper.make_opsetid("", 9)]) m.ir_version = 8 return m model = make_model() onnx.checker.check_model(model) # (1) valid ONNX model data = np.arange(24, dtype="float32").reshape(2, 3, 4) indices = np.array([[[1], [0], [0]]], dtype="int64") # shape (1, 3, 1) updates = np.array([[[100], [200], [300]]], dtype="float32") # shape (1, 3, 1) feeds = {"data": data, "indices": indices, "updates": updates} y_ort = onnxruntime.InferenceSession(model.SerializeToString(), providers=["CPUExecutionProvider"]).run(None, feeds)[0] print("onnxruntime:", np.where(np.abs(y_ort - data) > 0)) # (2) correct mod = from_onnx(model, shape_dict={"data": (2, 3, 4), "indices": (1, 3, 1), "updates": (1, 3, 1)}) ex = relax.build(mod, target="llvm") y_tvm = relax.VirtualMachine(ex, tvm.cpu())["main"](data, indices, updates) y_tvm = y_tvm.numpy() if hasattr(y_tvm, "numpy") else np.asarray(y_tvm) print("TVM :", np.where(np.abs(y_tvm - data) > 0)) # (3) wrong positions, no error print("max|diff| =", float(np.abs(y_tvm.astype("float64") - y_ort.astype("float64")).max())) ``` Actual output: ``` onnxruntime: (array([0, 0, 1]), array([1, 2, 0]), array([0, 0, 0])) TVM : (array([0, 0, 1]), array([0, 0, 0]), array([1, 2, 0])) max|diff| = 298.0 ``` `onnxruntime` writes updates at `(0,1,0), (0,2,0), (1,0,0)` (the three spec positions); TVM writes at `(0,0,1), (0,0,2), (1,0,0)` — two of the three updates land in the wrong cells, silently. ### Additional context - The ONNX Scatter v9/v10 semantics are per-entry over `indices`' own shape (loop counters "from 0 up to the respective size in `updates` - 1"), so `indices`/`updates` **smaller than `data`** (size-1 dims, broadcastable) are well-defined. `onnx.checker` passes, onnxruntime runs them, and the official ONNX numpy reference implementation (`data[idx[:axis] + (indices[idx],) + idx[axis+1:]] = updates[idx]`) matches onnxruntime exactly. `Scatter` is deprecated only from opset 11 ("Please use ScatterElements"), so opset 9/10 models are the legal live cases. - Root cause: `Scatter._impl_v9` passes `indices`/`updates` un-normalized to `relax.op.scatter_elements`. That op assumes indices/updates broadcastable to `data` (torch `scatter_` semantics); with a smaller `indices` tensor it silently mis-computes. Broadcasting `indices`/`updates` to `data`'s shape first (e.g. `relax.op.broadcast_to`) makes `scatter_elements` at least behave consistently, but note that torch-style broadcast semantics still differ from ONNX's per-entry semantics for these shapes, so the fix must replicate the exact ONNX algorithm for `indices.shape != data.shape`. - Systematic differential survey (`data (2,3,4)`, `indices` each dim ∈ {1, data dim}, 3 axes × 3 seeds): of **72** broadcast models accepted by onnxruntime / `onnx.checker`, TVM returns wrong numeric output for **12** — the combos `indices=(1,3,1)/(2,3,1), axis=0` and `indices=(2,1,1)/(2,1,4), axis=2`, all 3 seeds. The conventional case `indices.shape == data.shape` matches onnxruntime exactly (`max|diff| = 0`), so the defect is confined to broadcastable `indices`. 2-D `data` was also tested and did not reproduce (requires rank ≥ 3). ### Triage * needs-triage * bug * relax * frontend/onnx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
