Lunderberg opened a new pull request, #15810: URL: https://github.com/apache/tvm/pull/15810
Prior to this commit, intermediate objects produced while executing a Relax function would persist until the end of the Relax function. While re-use of static allocations is handled by the `StaticPlanBlockMemory` transform, re-use of dynamic allocations is handled by the `relax_vm::PooledAllocator`. For large Relax functions representing end-to-end model execution, releasing memory from the VM registers to the `relax_vm::PooledAllocator` at the end of the function call may be insufficient. This commit introduces a new pass, `relax.transform.KillAfterLastUse`, which identifies the last usage of each Relax variable and inserts a `relax.memory.kill_tensor`, `relax.memory.kill_storage`, or `relax.vm.kill_object` call depending on the object type. This insertion is suppressed if a Relax variables is already killed, such as static allocations and tensors tracked by `StaticPlanBlockMemory`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
