MasterJH5574 opened a new pull request, #17036: URL: https://github.com/apache/tvm/pull/17036
This PR updates the PagedKVCache with the pinned memory support, which can reduce the copy overhead between CPU and GPU. This PR also bumps FlashInfer version, which now supports * specifying kernels to build via cmake, * pinned memory as host memory. We also update CMakeLists.txt and config.cmake to include the FlashInfer compile options. Prior to this PR, the kernels being built is hardcoded in FlashInfer header files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
