comaniac opened a new pull request #5919: URL: https://github.com/apache/incubator-tvm/pull/5919
RFC discussion: https://discuss.tvm.ai/t/byoc-runtime-json-runtime-for-byoc/6579 Currently, BYOC allows developers to choose either C source module or their customized module as the runtime for their accelerators. While we have provided an end-to-end execution flow of DNNL (i.e., MKL-DNN, OneDNN) using C source module, we found that many developers prefer to use a customized module to better integrate to their own runtime engine, such as TensorRT. As a result, this PR (collaborating with @zhiics) provides an end-to-end flow of DNNL using JSON runtime. Some detail highlights: * We provide JSON codegen and JSON runtime base classes. JSON codegen serializes a Relay subgaph to a JSON file; while JSON runtime base provides deserialization methods to interprete subgraphs in JSON format. Developers can derive JSON codegen to easily customize their codegen, or even directly use JSON codegen if their runtime engine accepts standard TVM graph runtime JSON. * We make a case study of leveraging JSON runtime with DNNL. The DNNL JSON runtime now supports conv2d, dense, relu, batch_norm, and add. As a result, it is able to run MobileNet. Note that DNNL JSON runtime only creates one DNNL execution engine for a subgraph, so it is much more efficient compared to the C source module version, which creates a DNNL engine for each *operator* in a subgraph. * DNNL JSON runtime handles constant tensors following the new mechanism in #5770. * DNNL codegen with C source module will be preserved for illustraction purpose, and we use cmake to control which DNNL codegen should be used. Specifically, `USE_DNNL_CODEGEN ON` and `USE_DNNL_CODEGEN JSON` enable the JSON runtime (and this is the default runtime for DNNL). When following the tutorial, which we will update after this PR, users may use `USE_DNNL_CODEGEN C_SRC` to enable C source module so that they can learn how it work. cc @masahi @mbaret @tqchen ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
