Thrsu opened a new issue, #15718: URL: https://github.com/apache/tvm/issues/15718
I encountered an error while using `tvm.relax` to execute an ONNX [model](https://github.com/Thrsu/onnx_model/blob/main/Pow.onnx) that contains a Pow operator. The issue arises when the two operands of the Pow operator have different data types, one being int32 and the other float32. During runtime, TVM throws an error with the following message: "TVMError: Data types int32 and float32 must be equal for binary operators." BTW, I noticed that this issue does not occur when using `tvm.relay` to load the model. ### Actual behavior ``` Traceback (most recent call last): ...... tvm_model = from_onnx(model, opset=18, keep_params_in_input=True) File "/workplace/software/tvm/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 2287, in from_onnx return g.from_onnx(graph, opset) File "/workplace/software/tvm/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 1932, in from_onnx self._construct_nodes(graph) File "/workplace/software/tvm/tvm/python/tvm/relax/frontend/onnx/onnx_frontend.py", line 2097, in _construct_nodes op = self.bb.normalize(op) File "/workplace/software/tvm/tvm/python/tvm/relax/block_builder.py", line 645, in normalize return _ffi_api.BlockBuilderNormalize(self, expr) # type: ignore File "/workplace/software/tvm/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 238, in __call__ raise get_last_ffi_error() tvm._ffi.base.TVMError: Traceback (most recent call last): 16: TVMFuncCall 15: _ZN3tvm7runtime13Pac 14: tvm::runtime::TypedPackedFunc<tvm::RelayExpr (tvm::relax::BlockBuilder, tvm::RelayExpr const&)>::AssignTypedLambda<tvm::runtime::Registry::set_body_method<tvm::relax::BlockBuilder, tvm::relax::BlockBuilderNode, tvm::RelayExpr, tvm::RelayExpr const&, void>(tvm::RelayExpr (tvm::relax::BlockBuilderNode::*)(tvm::RelayExpr const&))::{lambda(tvm::relax::BlockBuilder, tvm::RelayExpr const&)#1}>(tvm::runtime::Registry::set_body_method<tvm::relax::BlockBuilder, tvm::relax::BlockBuilderNode, tvm::RelayExpr, tvm::RelayExpr const&, void>(tvm::RelayExpr (tvm::relax::BlockBuilderNode::*)(tvm::RelayExpr const&))::{lambda(tvm::relax::BlockBuilder, tvm::RelayExpr const&)#1}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const 13: tvm::relax::Normalizer::Normalize(tvm::RelayExpr const&) 12: tvm::relax::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::VisitExpr(tvm::RelayExpr const&) 11: tvm::NodeFunctor<tvm::RelayExpr (tvm::runtime::ObjectRef const&, tvm::relax::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*)>::operator()(tvm::runtime::ObjectRef const&, tvm::relax::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>*) const 10: _ZZN3tvm5relax11ExprFunc 9: _ZThn200_N3tvm5relax1 8: tvm::relax::Normalizer::VisitExpr_(tvm::relax::CallNode const*) 7: tvm::relax::Normalizer::InferStructInfo(tvm::relax::Call const&) 6: _ZN3tvm7runtime13Pac 5: tvm::runtime::TypedPackedFunc<tvm::relax::StructInfo (tvm::relax::Call const&, tvm::relax::BlockBuilder const&)>::AssignTypedLambda<tvm::relax::StructInfo (*)(tvm::relax::Call const&, tvm::relax::BlockBuilder const&)>(tvm::relax::StructInfo (*)(tvm::relax::Call const&, tvm::relax::BlockBuilder const&))::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}::operator()(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*) const 4: _ZN3tvm5relax29InferS 3: tvm::relax::StructInfo tvm::relax::InferStructInfoBroadcast<tvm::runtime::DataType (*)(tvm::relax::Call const&, tvm::relax::BlockBuilder const&, tvm::relax::TensorStructInfo const&, tvm::relax::TensorStructInfo const&)>(tvm::relax::Call const&, tvm::relax::BlockBuilder const&, tvm::runtime::DataType (*)(tvm::relax::Call const&, tvm::relax::BlockBuilder const&, tvm::relax::TensorStructInfo const&, tvm::relax::TensorStructInfo const&)) 2: tvm::relax::InferBinaryArithOpOutDtype(tvm::relax::Call const&, tvm::relax::BlockBuilder const&, tvm::relax::TensorStructInfo const&, tvm::relax::TensorStructInfo const&) 1: tvm::relax::BlockBuilderImpl::ReportFatal(tvm::Diagnostic const&) 0: _ZN3tvm7runtime6detail File "/workplace/software/tvm/tvm/src/relax/ir/block_builder.cc", line 138 TVMError: Data types int32 and float32 must be equal for binary operators ``` ### Environment - TVM: v0.14.dev0-595-gfa0d35b - onnx: 1.14.0 - onnxruntime: 1.14.1 ### Steps to reproduce ```python import onnx import numpy as np import onnxruntime import tvm from tvm import relax from tvm.relax.frontend.onnx import from_onnx input_data = {} input_data['x'] = np.random.randint(1, 4, [3], dtype=np.int32) input_data['y'] = np.random.rand(3).astype(np.float32) onnx_model_path = "Pow.onnx" model = onnx.load(onnx_model_path) sess = onnxruntime.InferenceSession(model.SerializeToString()) onnx_output = sess.run(None, input_data) tvm_model = from_onnx(model, opset=18, keep_params_in_input=True) tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model) tvm_model = relax.transform.LegalizeOps()(tvm_model) tvm_model, params = relax.frontend.detach_params(tvm_model) with tvm.transform.PassContext(opt_level=3): ex = relax.build(tvm_model, target="llvm") vm = relax.VirtualMachine(ex, tvm.cpu()) inputs = {} inputs['x'] = tvm.nd.array(input_data['x'].astype(np.int32)) inputs['y'] = tvm.nd.array(input_data['y'].astype(np.float32)) input_tvm = [ inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs ] vm.set_input("main", *input_tvm) vm.invoke_stateful("main") tvm_output = vm.get_outputs("main").numpy() np.testing.assert_allclose(onnx_output[0], tvm_output, atol=1e-3, rtol=1e-3) ``` ### Triage * needs-triage I kindly request your assistance in resolving this bug, enabling TVM to handle the Pow operator with operands of different data types. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
