tqchen commented on PR #18871: URL: https://github.com/apache/tvm/pull/18871#issuecomment-4000781007
I think starting with a uniform pool + lazy seems to be a good first step as that strictly improves over what we had. My understanding is we can always flush the queue when we run out of uniform pool. If you can send a PR that starts with uniform pool + lazy it would be great. I get how uniform buffer cache helps to reduce things further, and would be happy we explore that as a followup PR. Maybe one idea is that we have introduce a CacheState class that provides the relevant functionalities (wrt to cache key computation with comments on correctness, invalidation scenarios). Thanks @mitiskuma for looking into this. I think the two phase approach could give us more confidence here and also have a bit more thinkings to make the CacheState also robust. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
