sunggg commented on code in PR #14282:
URL: https://github.com/apache/tvm/pull/14282#discussion_r1136411192
##########
src/runtime/relax_vm/builtin.cc:
##########
@@ -380,6 +380,40 @@
TVM_REGISTER_GLOBAL("vm.builtin.make_tuple").set_body([](TVMArgs args, TVMRetVal
*rv = arr;
});
+TVM_REGISTER_GLOBAL("vm.builtin.tensor_to_shape").set_body_typed([](NDArray
data) {
+ NDArray arr = data;
+ if (data->device.device_type != kDLCPU) {
+ arr = data.CopyTo(DLDevice{kDLCPU, 0});
Review Comment:
Based on my current understanding, we want to keep the shape-related
computation on the host side since its result could be related to the memory
planning.
Even for general GPU execution besides this case, we are running output
shape computation (inserted by `VMShapeLower` pass) on the host side.
##########
src/runtime/relax_vm/builtin.cc:
##########
@@ -380,6 +380,40 @@
TVM_REGISTER_GLOBAL("vm.builtin.make_tuple").set_body([](TVMArgs args, TVMRetVal
*rv = arr;
});
+TVM_REGISTER_GLOBAL("vm.builtin.tensor_to_shape").set_body_typed([](NDArray
data) {
+ NDArray arr = data;
+ if (data->device.device_type != kDLCPU) {
+ arr = data.CopyTo(DLDevice{kDLCPU, 0});
Review Comment:
Based on my current understanding, we want to keep the shape-related
computation on the host side since its result could be related to the memory
planning. Even for general GPU execution besides this case, we are running
output shape computation (inserted by `VMShapeLower` pass) on the host side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
