tqchen opened a new issue, #13678: URL: https://github.com/apache/tvm/issues/13678
In https://github.com/apache/tvm/pull/11003 we introduced NDArray::FromExternalDLTensor, which allows us to create an NDArray from a DLTensor struct. This method, however, is not safe, because the followup method can store the NDArray into data structures like, and there is no guarantee that the external DLTensor won't get freed before these data structure. A proper approach would be keeping the data as NDArray, which zero copy works. If we want to be able to exchange an external memory, we should do that through DLPack(which comes with DLManagedTensor*), and proper deleter to be able to track such dependencies and retain memory such. When we are chaining multiple models, in most cases such chaining can be done through natively allocated NDArray, if not possible having a way to construct such external memory into DLManagedTensor would be a safer way(then pass into the SetInput). Leaving a note here to see if we want to revisit this API behavior (I think ideally we should cleanup this API) cc @vvchernov @jwfromm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
