masahi commented on pull request #8285: URL: https://github.com/apache/tvm/pull/8285#issuecomment-903641187
This change doesn't solve the issue in https://github.com/apache/tvm/issues/8233, because `AllocDataSpace` can be called from `NDArray::Empty`: https://github.com/apache/tvm/blob/4d9bc9b4a3e9e8d3420efe60a52964fcd4c29c8d/src/runtime/ndarray.cc#L196-L197 That call is not protected by try/catch, so if almost all memory are held by `PooledAllocator` and `NDArray::Empty` is called, the program crashes with the following error: ``` terminate called after throwing an instance of 'tvm::runtime::InternalError' what(): [19:12:54] /home/masa/projects/dev/tvm/src/runtime/vulkan/vulkan_stream.cc:123: --------------------------------------------------------------- An error occurred during the execution of TVM. For more information, please see: https://tvm.apache.org/docs/errors.html --------------------------------------------------------------- Check failed: (__e == VK_SUCCESS) is false: Vulkan Error, code=-13: Unknown Vulkan error code Stack trace: 0: tvm::runtime::vulkan::VulkanStream::Synchronize() 1: _ZN3tvm7runtime6vulkan15VulkanDeviceAPI13FreeDataSpac 2: tvm::runtime::NDArray::Internal::DefaultDeleter(tvm::runtime::Object*) 3: tvm::runtime::NDArray::CopyTo(DLDevice const&) const 4: tvm::runtime::vm::CopyTo(tvm::runtime::ObjectRef, DLDevice const&) 5: std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::vm::VirtualMachine::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::$_6>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&) 6: TVMFuncCall ``` I think we need to revisit the memory release strategy of `PooledAllocator`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@tvm.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
