MasterJH5574 opened a new pull request, #16318: URL: https://github.com/apache/tvm/pull/16318
Prior to this PR, the memory planning for different Relax functions are not independent -- storage tokens are shared across different Relax functions. This will incur memory overuse sometimes. For example, tensor `A` in `func1` has 128 bytes, tensor `B` in `func2` has 2048 bytes. If the memory planning decides to share the storage token for `A` and `B`, the shared token will have size 2048 bytes. Consider the case when at runtime only `func1` is executed, and `func2` is never invoked. In this case, only 128 bytes for tensor `A` is needed, while a total 2048-chunk is allocated in total, which is a 16x memory overuse. This PR makes the memory planning across different Relax function independent. That means in the example above, when only `func1` is executed, only 128 bytes will be allocated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
