danieldk opened a new issue, #595: URL: https://github.com/apache/tvm-ffi/issues/595
Hi 👋. I am one of the core team members of the Hugging Face Kernels project: https://huggingface.co/docs/kernels/ The goal of the kernels project is to make it easy for kernel developers to write and distribute kernels and for kernel users to use kernels in their favorite models. The project consists of several parts: * `kernel-builder`: builds kernels for multiple systems (OS + arch), backends (CUDA, XPU, ROCm, Metal and multiple versions of each), and framework versions (e.g. multiple Torch versions). * Kernel Hub: which can be used for distribution of kernels. There are quite some kernels already available from us and from third-parties: https://huggingface.co/kernels * The `kernels` Python client. Using the Python client, users can download and load kernels using simply: ``` import kernels fa3 = kernels.get_kernel("kernels-community/flash-attn3", version=1) ``` `kernels` will take care of download and loading the right variant for a user's system. So far, we have focused primarily on Torch support, but I think TVM FFI fits in well with our goal of making it easy to build and load kernels, especially because it would make Hub kernels available for frameworks beyond Torch. A while ago, we implemented initial support for building and loading TVM FFI kernels. There is a minimal example at: https://github.com/huggingface/kernels/tree/main/examples/kernels/relu-tvm-ffi , with the built kernel at: https://huggingface.co/kernels/kernels-test/relu-tvm-ffi We would love to collaborate more if you are interested - we have a bunch of questions around encoding ABI versions, etc. If you are interested, we could set up a shared Slack channel or otherwise communicate through this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
