zxybazh commented on issue #13639:
URL: https://github.com/apache/tvm/issues/13639#issuecomment-1370137144
Can reproduce same error, the stack trace is a bit different, run on latest
main 6dbb7e1a5e357ee829348a1c1953b32837e68289 with cuda 11.8, target
`nvidia/geforce-rtx-3070` and torch version `1.12.1+cu116`.
```
terminate called after throwing an instance of 'tvm::runtime::InternalError'
what(): [11:11:03]
/home/zxybazh/tvm-tensorir/src/runtime/cuda/cuda_device_api.cc:135:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (e == cudaSuccess || e == cudaErrorCudartUnloading) is
false: CUDA: unspecified launch failure
Stack trace:
0: tvm::runtime::CUDADeviceAPI::FreeDataSpace(DLDevice, void*)
at /home/zxybazh/tvm-tensorir/src/runtime/cuda/cuda_device_api.cc:131
1: tvm::runtime::NDArray::Internal::DefaultDeleter(tvm::runtime::Object*)
at /home/zxybazh/tvm-tensorir/src/runtime/ndarray.cc:107
2: tvm::runtime::Object::DecRef()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:805
3: tvm::runtime::Object::DecRef()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:801
4: tvm::runtime::ObjectPtr<tvm::runtime::Object>::reset()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:451
5: tvm::runtime::ObjectPtr<tvm::runtime::Object>::~ObjectPtr()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:400
6: tvm::runtime::ObjectRef::~ObjectRef()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:511
7: tvm::runtime::NDArray::~NDArray()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/ndarray.h:59
8: void std::_Destroy<tvm::runtime::NDArray>(tvm::runtime::NDArray*)
at /usr/include/c++/11/bits/stl_construct.h:140
9: void
std::_Destroy_aux<false>::__destroy<tvm::runtime::NDArray*>(tvm::runtime::NDArray*,
tvm::runtime::NDArray*)
at /usr/include/c++/11/bits/stl_construct.h:152
10: void std::_Destroy<tvm::runtime::NDArray*>(tvm::runtime::NDArray*,
tvm::runtime::NDArray*)
at /usr/include/c++/11/bits/stl_construct.h:185
11: void std::_Destroy<tvm::runtime::NDArray*,
tvm::runtime::NDArray>(tvm::runtime::NDArray*, tvm::runtime::NDArray*,
std::allocator<tvm::runtime::NDArray>&)
at /usr/include/c++/11/bits/alloc_traits.h:746
12: std::vector<tvm::runtime::NDArray,
std::allocator<tvm::runtime::NDArray> >::~vector()
at /usr/include/c++/11/bits/stl_vector.h:680
13: tvm::runtime::GraphExecutor::~GraphExecutor()
at
/home/zxybazh/tvm-tensorir/src/runtime/graph_executor/graph_executor.h:67
14:
tvm::runtime::SimpleObjAllocator::Handler<tvm::runtime::GraphExecutor>::Deleter_(tvm::runtime::Object*)
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/memory.h:138
15: tvm::runtime::ObjectPtr<tvm::runtime::GraphExecutor>::reset()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:451
16: tvm::runtime::ObjectPtr<tvm::runtime::GraphExecutor>::~ObjectPtr()
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/object.h:400
17:
tvm::runtime::GraphExecutorFactory::ExecutorCreate(std::vector<DLDevice,
std::allocator<DLDevice> > const&)
at
/home/zxybazh/tvm-tensorir/src/runtime/graph_executor/graph_executor_factory.cc:125
18: operator()
at
/home/zxybazh/tvm-tensorir/src/runtime/graph_executor/graph_executor_factory.cc:55
19: Call
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/packed_func.h:1213
20: tvm::runtime::PackedFuncObj::CallPacked(tvm::runtime::TVMArgs,
tvm::runtime::TVMRetValue*) const
at /home/zxybazh/tvm-tensorir/include/tvm/runtime/packed_func.h:1217
21: TVMFuncCall
at /home/zxybazh/tvm-tensorir/src/runtime/c_runtime_api.cc:477
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
