zhuwenxi commented on a change in pull request #47: URL: https://github.com/apache/tvm-rfcs/pull/47#discussion_r771079138
########## File path: rfcs/0046-Intel-LIBXSMM-integration.md ########## @@ -0,0 +1,97 @@ +# Summary +This RFC introduces the plan of integrating LIBXSMM into TVM. LIBXSMM leverages JIT code generator to produce high efficient kernels targeting x86 architectures. + +For details of LIBXSMM, please refer to: +* [LIBXSMM User Manual](https://libxsmm.readthedocs.io/en/latest/) +* [LIBXSMM github repo](https://github.com/hfp/libxsmm) + +# Motivation +TVM has shown satisfactory performance on MLP models with CPU. However there are still some defects in the assembly code generated by LLVM which block AutoTVM/AutoScheduler from achieving optimal on GEMM. + +LIBXSMM is a open source library developed by Intel Lab for accelerating small matrix multiplication. It leverages the JIT code generator to generate high efficient GEMM kernels for x86 CPU, which could be very close to hardware rootline. According to our evaluation, in “small” GEMM (cube_root(m * n * k) <= 256) , LIBXSMM shows a superior performance over the well-known BLAS library Intel MKL. + +By the way, given that LIBXSMM can generate quite efficient GEMM kernel implementation, it is also an ideal substitution for inner-kernel of normal size GEMM. According our experiments, the AutoTVM templates we wrote with LIBXSMM as register-block generation, has a much higher performance comparing to MKL and existing TOPI implementation. + +# Guide-level explanation +This proposal aims to integrate LIBXSMM into TVM to accelerate small GEMM and serve as inner-kernel to accelerate normal size GEMM. + +We will integrate LIBXSMM with TVM in following 3 components: +1. Add extern call “tvm.contrib.libxsmm.gemm” in “src/runtime/contrib” directory, and corresponding python interface in "python/tvm/contrib/" directory, so users can call them just as CBLAS; +2. Use BYOC to accelerate small GEMM (cube_root(m * n * k ) <= 256) and its epilogue fusion variations (bias/relu/sigmoid/bias_relu/bias_sigmoid); +3. AutoTVM template we wrote with LIBXSMM as inner kernel into TOPI, as a GEMM implementation candidate. + +# Reference-level explanation +1. Users can call libxsmm as CBLAS through extern call API. Review comment: 1. Relay op strategy support is already in our proposal. It's natural to add Relay op strategy when TOPI integration is done. OK, I'll make it clear in this RFC; 2. It does make sense to cover the target system, I'll add this part. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
