junrushao1994 commented on PR #12084: URL: https://github.com/apache/tvm/pull/12084#issuecomment-1183660649
I'm glad that #12019 fixes this issue that @tkonolige mentioned! so I believe it has unblocked you guys already. Indeed `StorageRewrite` could be buggy when encountering dynamic-shaped buffer allocation - I noticed the same thing personally - and we should get it fixed. It's good that we started discussing about it, on the other hand, we should also find the most proper fix in this case. Let me share some context first: on GPUs, intermediate buffers are usually constant-sized - even for dynamic shared memory, it always has an upper bound. Therefore, if TIR is specifically for CUDA/Vulkan codegen, I believe it's proper to make this assumption that each buffer should be constant-sized; On the other hand, dynamic-shape allocation could be useful on other devices like CPUs, where allocating dynamic sized memory is very straightforward without constraint. Therefore, to maintain TIR as generic, do you guys think there could be some alternative approaches, for example, pinging the buffer under loop `i` if its size depends on `i`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
