coffezhou opened a new issue, #17835:
URL: https://github.com/apache/tvm/issues/17835
### Expected behavior
For the following onnx model, the output of "v6_0" should be 0.
```python
[array(518, dtype=int64), array([ True, True, True, True, True, True,
True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True]),
array(0, dtype=int64),
array([25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25], dtype=uint8)]
```
### Actual behavior
However, when we compile the onnx model using relax.build with the default
optimization pipeline, tvm produces -1 of v6_0.
```python
(<tvm.nd.NDArray shape=(), cpu(0)>
array(518), <tvm.nd.NDArray shape=(24,), cpu(0)>
array([ True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True, True, True, True,
True, True, True, True, True, True]),
<tvm.nd.NDArray shape=(), cpu(0)>
array(-1),
<tvm.nd.NDArray shape=(24,), cpu(0)>
array([25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25, 25,
25, 25, 25, 25, 25, 25, 25], dtype=uint8))
```
### Environment
OS: Ubuntu 20.04
TVM: 0.20.dev0 (f6236ce41)
### Steps to reproduce
This bug can be reproduced by the following code with the model in the
attachment.
```python
from typing import Dict, List, Literal, Optional
import numpy as np
import onnx
import onnxruntime
from onnx import ModelProto, TensorProto, helper, mapping
import tvm
import tvm.testing
from tvm import relax
from tvm.relax.frontend.onnx import from_onnx
import pickle
def get_oracle(oracle_path):
with open(oracle_path, 'rb') as f:
oracle = pickle.load(f)
return oracle['input'], oracle['output']
def check(
model: ModelProto,
inputs: Optional[Dict[str, np.ndarray]] = None,
) -> None:
# Run the model through onnx to get the expected result.
try:
ort_session = onnxruntime.InferenceSession(
model.SerializeToString(), providers=["CPUExecutionProvider"]
)
ort_output = ort_session.run([], inputs)
except:
print("This model cannot be executed by onnxruntime!")
sys.exit(1)
print("onnxrumtime: ", ort_output)
# Convert the onnx model into relax through the onnx importer.
tvm_model = from_onnx(model, keep_params_in_input=True)
# Convert operators for inference mode.
tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model)
# Legalize any relax ops into tensorir.
tvm_model = relax.transform.LegalizeOps()(tvm_model)
# Separate model from parameters.
tvm_model, params = relax.frontend.detach_params(tvm_model)
# Prepare inputs.
input_list = [
inputs[key.name_hint] for key in tvm_model["main"].params if
key.name_hint in inputs
]
if params:
input_list += params["main"]
# Compile the relax graph into a VM then run.
#----------------------cpu-----------------------
with tvm.transform.PassContext(opt_level=0):
ex = relax.build(tvm_model, target="llvm", relax_pipeline="default")
vm = relax.VirtualMachine(ex, tvm.cpu())
# Run model and check outputs.
vm.set_input("main", *input_list)
vm.invoke_stateful("main")
tvm_cpu_output = vm.get_outputs("main")
#----------------------cpu-----------------------
print("tvm cpu", tvm_cpu_output)
def main(model_path, oracle_path):
onnx_model = onnx.load(model_path)
inputs, outputs = get_oracle(oracle_path)
check(onnx_model, inputs=inputs)
main("model.onnx", "oracle.pkl")
```
[testcase.zip](https://github.com/user-attachments/files/19732016/testcase.zip)
### Triage
Please refer to the list of label tags
[here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the
relevant tags and add them below in a bullet format (example below).
* needs-triage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
