vvchernov commented on issue #13678: URL: https://github.com/apache/tvm/issues/13678#issuecomment-1369430160
Hello @tqchen! You are correct in your description and yes, for TVM is better to go along scenario with DLPack to prevent memory safity issue. Nevertheless the method with nesting of NDArray on the external memory (for zero copying) is convenient for some cases. Particularly, recently TVM was embedded to ONNX Runtime (ORT). To avoid excess copying between ORT and TVM input/output tensors this method was implemented. Of course, it is advanced tool and assumes client responsibility for memory management. Another thing about which I thought is performance measurements for TVM without input/output copying time. I understand that these parts can be hidden in pipelining mechanism, but it is not developed well on TVM python front-end. Recently I have implemented mechanism to set external output memory to VM instead of allocated inside VM. It gives possibility to client to get output tensors where he wants and keeps them for as long as he need even if TVM session is released. I think this case clearly show why method like NDArrary::FromExternalDLTensor is needed. P.S. Possibly description of the method should be extended to warn the client about the responsibility when using it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
