digital-nomad-cheng opened a new issue, #15013: URL: https://github.com/apache/tvm/issues/15013
### Expected behavior I successfully dispatched the [yolov8](https://github.com/ultralytics/ultralytics) for tensorrt. But when I try to use the cutlass backend, tvm failed to partition it. ### Actual behavior The error happens when I call `mod = partition_for_cutlass(mod, params)` The error given: ``` mod = partition_for_cutlass(mod, params) File "/home/tvm/python/tvm/relay/op/contrib/cutlass.py", line 317, in partition_for_cutlass return seq(mod) File "/home/tvm/python/tvm/ir/transform.py", line 160, in __call__ return _ffi_transform_api.RunPass(self, mod) File "/home/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__ raise get_last_ffi_error() AttributeError: Traceback (most recent call last): 22: TVMFuncCall 21: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::TypedPackedFunc<tvm::IRModule (tvm::transform::Pass, tvm::IRModule)>::AssignTypedLambda<tvm::transform::__mk_TVM9::{lambda(tvm::transform::Pass, tvm::IRModule)#1}>(tvm::transform::__mk_TVM9::{lambda(tvm::transform::Pass, tvm::IRModule)#1}, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, tvm::runtime::TVMRetValue) 20: tvm::transform::Pass::operator()(tvm::IRModule) const 19: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 18: tvm::transform::SequentialNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 17: tvm::transform::Pass::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 16: tvm::relay::transform::FunctionPassNode::operator()(tvm::IRModule, tvm::transform::PassContext const&) const 15: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<tvm::runtime::TypedPackedFunc<tvm::relay::Function (tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)>::AssignTypedLambda<tvm::relay::transform::MergeComposite(tvm::runtime::Array<tvm::runtime::String, void> const&, tvm::runtime::Array<tvm::relay::DFPattern, void> const&, std::vector<tvm::runtime::PackedFunc, std::allocator<tvm::runtime::PackedFunc> > const&)::{lambda(tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)#1}>(tvm::relay::transform::MergeComposite(tvm::runtime::Array<tvm::runtime::String, void> const&, tvm::runtime::Array<tvm::relay::DFPattern, void> const&, std::vector<tvm::runtime::PackedFunc, std::allocator<tvm::runtime::PackedFunc> > const&)::{lambda(tvm::relay::Function, tvm::IRModule, tvm::transform::PassContext)#1})::{lambda(tvm::runtime::TVMArgs const&, tvm::runtime::TVMRetValue*)#1}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm:: runtime::TVMRetValue*) 14: tvm::relay::merge_composite::MergeComposite(tvm::relay::Function const&, tvm::runtime::Array<tvm::runtime::String, void> const&, tvm::runtime::Array<tvm::relay::DFPattern, void> const&, std::vector<tvm::runtime::PackedFunc, std::allocator<tvm::runtime::PackedFunc> > const&, tvm::IRModule const&) 13: tvm::relay::PartitionPattern(tvm::relay::DFPattern, tvm::RelayExpr, tvm::runtime::Map<tvm::runtime::String, tvm::runtime::ObjectRef, void, void>, tvm::runtime::PackedFunc) 12: tvm::relay::PatternPartitioner::Partition(tvm::relay::DFPattern const&, tvm::RelayExpr const&, tvm::runtime::Map<tvm::runtime::String, tvm::runtime::ObjectRef, void, void> const&, tvm::runtime::PackedFunc) 11: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&) 10: tvm::relay::MixedModeMutator::VisitLeaf(tvm::RelayExpr const&) 9: tvm::relay::PatternPartitioner::DispatchVisitExpr(tvm::RelayExpr const&) 8: _ZN3tvm5relay16MixedModeMutator17DispatchVisitExprERKNS_9RelayExp 7: tvm::relay::ExprMutator::VisitExpr(tvm::RelayExpr const&) 6: tvm::relay::ExprFunctor<tvm::RelayExpr (tvm::RelayExpr const&)>::VisitExpr(tvm::RelayExpr const&) 5: _ZZN3tvm5relay11ExprFunctorIFNS_9RelayExprERKS2_EE10InitVTableEvENUlRKNS_ 4: tvm::relay::ExprMutator::VisitExpr_(tvm::relay::FunctionNode const*) 3: tvm::relay::MixedModeMutator::VisitExpr(tvm::RelayExpr const&) 2: tvm::relay::MixedModeMutator::VisitLeaf(tvm::RelayExpr const&) 1: tvm::relay::PatternPartitioner::DispatchVisitExpr(tvm::RelayExpr const&) 0: tvm::runtime::PackedFuncObj::Extractor<tvm::runtime::PackedFuncSubObj<TVMFuncCreateFromCFunc::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#2}> >::Call(tvm::runtime::PackedFuncObj const*, tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) [clone .cold] File "/home/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 81, in cfun rv = local_pyfunc(*pyargs) File "/home/tvm/python/tvm/relay/op/contrib/cutlass.py", line 193, in check_conv2d_residual if not check_conv2d(call): File "/home/tvm/python/tvm/relay/op/contrib/cutlass.py", line 178, in check_conv2d return check_conv2d_common("nn.conv2d", "OHWI", call) File "/home/tvm/python/tvm/relay/op/contrib/cutlass.py", line 162, in check_conv2d_common data_layout = conv2d.attrs.data_layout AttributeError: 'NoneType' object has no attribute 'attrs' ``` I found that some of the types of the `call` passed into `get_root_call` is of type `<class 'tvm.relay.expr.TupleGetItem'>`, which leads it to return `None`. It should be of type, `<class 'tvm.relay.expr.Call'>` ### Environment os: ubuntu22.04 tvm: 0.12.0 ### Steps to reproduce Code to reproduce: ``` from ultralytics import YOLO import onnx import tvm from tvm import relay from tvm.contrib.cutlass import ( has_cutlass, num_cutlass_partitions, finalize_modules ) from tvm.relay.op.contrib.cutlass import partition_for_cutlass model = YOLO("yolov8n.pt") success = model.export(format="onnx") onnx_model = onnx.load_model("yolov8n.onnx") shape_dict = {"images": [1, 3, 640, 640]} mod, params = relay.frontend.from_onnx(onnx_model, shape_dict) print("Use cutlass codegen...") def convert_conv2d_layout(mod, desired_layouts): with tvm.transform.PassContext(opt_level=3): seq = tvm.transform.Sequential([relay.transform.ConvertLayout(desired_layouts)]) return seq(mod) print(mod) mod = convert_conv2d_layout(mod, {"nn.conv2d": ["NHWC", "OHWI"]}) mod = partition_for_cutlass(mod, params) mod = convert_conv2d_layout(mod, {"nn.conv2d": ["NHWC", "default"]}) num_partition = num_cutlass_partitions(mod) assert num_partition != 0, "Partition for cutlass failed!" print("num of partition using cutlass:", num_partition) with tvm.transform.PassContext(opt_level=3): lib = relay.build(mod, target=[target, cutlass], params=params) lib = finalize_modules(lib) ``` ### Triage Please refer to the list of label tags [here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the relevant tags and add them below in a bullet format (example below). * needs-triage * byoc:cutlass * relay:op -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
