coffezhou opened a new issue, #18132: URL: https://github.com/apache/tvm/issues/18132
### Expected behavior TVM should run the model correctly. ### Actual behavior For the following model, <img width="269" height="460" alt="Image" src="https://github.com/user-attachments/assets/ba94f29e-b7c9-4c77-b222-ad028c7cc03d"; /> it can be executed by onnxruntime, the results are as follows: ```c ONNXRuntime: [array([[[[False, False, False, False], [False, False, False, False], [False, False, False, False], [False, False, False, False]]]])] ``` However, when compiling and running the model using TVM, TVM crashes: ```c File "/home/carla/Documents/tvm/python/tvm/runtime/vm.py", line 295, in invoke_stateful self._invoke_stateful(func_name) File "tvm/ffi/cython/./function.pxi", line 228, in tvm.ffi.core.Function.__call__ tvm.error.InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 18446744073709551553 at offset 0 in region that is 0bytes ``` ### Environment OS: Ubuntu 20.04 TVM: 0.22.dev0 (c6969d723) onnxruntime: 1.21.0 ### Steps to reproduce This bug can be reproduced by the following code with the model in the attachment. As shown in the code, the model can be executed by onnxruntime. However, TVM crashes when calling the invoke_stateful function. ```python import sys import numpy as np import onnx import onnxruntime import tvm from tvm import relax from tvm.relax.frontend.onnx import from_onnx import pickle def main(): onnx_model = onnx.load("111.onnx") with open("inputs.pkl", "rb") as fp: inputs = pickle.load(fp) print(inputs) try: ort_session = onnxruntime.InferenceSession( onnx_model.SerializeToString(), providers=["CPUExecutionProvider"] ) ort_output = ort_session.run([], inputs) except Exception as e: print(e) sys.exit(1) print("ONNXRuntime:\n", ort_output) # Convert the onnx model into relax through the onnx importer. tvm_model = from_onnx(onnx_model, keep_params_in_input=True) # Convert operators for inference mode. tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model) # Legalize any relax ops into tensorir. tvm_model = relax.transform.LegalizeOps()(tvm_model) # Separate model from parameters. tvm_model, params = relax.frontend.detach_params(tvm_model) # Prepare inputs. input_list = [ inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs ] if params: input_list += params["main"] # Compile the relax graph into a VM then run. #----------------------cpu----------------------- with tvm.transform.PassContext(opt_level=0): target = tvm.target.Target("llvm", host="llvm") relax_pipeline = relax.pipeline.get_default_pipeline(target) ex = relax.build(tvm_model, target="llvm", relax_pipeline=relax_pipeline) vm = relax.VirtualMachine(ex, tvm.cpu()) # Run model and check outputs. vm.set_input("main", *input_list) vm.invoke_stateful("main") tvm_cpu_output = vm.get_outputs("main") if __name__ == "__main__": main() ``` [testcase.zip](https://github.com/user-attachments/files/21178457/testcase.zip) ### Triage Please refer to the list of label tags [here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the relevant tags and add them below in a bullet format (example below). * needs-triage -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
