areusch commented on a change in pull request #7164: URL: https://github.com/apache/tvm/pull/7164#discussion_r551655848
########## File path: docs/dev/microtvm_design.rst ########## @@ -0,0 +1,340 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at +.. http://www.apache.org/licenses/LICENSE-2.0 +.. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +************************** +microTVM Design Document +************************** + +.. contents:: Table of Contents + :depth: 3 + +Background +=========== + +TVM is a model deployment framework that has demonstrated good performance across a wide range of +models on traditional operating systems. Given TVM's layered approach to compilation, it is a +natural extension to target bare metal devices. While most of the compilation flow does not need to +change for a proof-of-concept implementation on such devices, the runtime cannot depend on: + +* **Virtual Memory**, and by extension any system-provided ``malloc``. Additionally, bare metal + devices typically have very limited memory (measured in KB). Because of this, libraries designed + for such platforms typically need to be more judicious in using memory, and need to release + memory when it is not in use. +* Traditional OS abstractions, such as **files**, **libraries**, and **kernel functions**. Some + projects implement support for these, but they are by no means standard. +* Support for programming languages other than **C**. + +Such changes require a different appraoch from the TVM C++ runtime typically used on traditional +Operating Systems. + +Typical Use +=========== + +This section discusses our vision of the "typical" microTVM use case. Each component used to achieve +this typical use case is intended to be designed for flexibility, but this unifying vision serves to +motivate the inclusion of each part of the design. + +.. image:: microtvm_workflow.svg + +The parts of this process are described below: + +#. **Model Import**. The user imports an existing model or describes a new model to TVM, producing a + *Relay module*. + +#. **Model Transformations**. The user can apply transformations, such as quantization, to the + model. After each transformation, the user should still have a Relay module. + +#. **Compilation** (Scheduling and Code Generation). TVM implements each operator into Tensor IR by + assigning a schedule and schedule configuration to each Relay operator. Then, code (C source or + compiled object) is generated for each operator. + +#. **Integration**. The generated code is integrated along with the TVM C Runtime library into a + user-supplied binary project. In some cases (such as when the project is standardized across + multiple SoC/development boards), this process is handled automatically. + +#. **Deployment**. The project is built and the residual firmware binary is flashed onto the device. + Model inference is driven either by TVM using an on-device RPC server, or on the device using the + on-device Graph Runtime. + +Design Goals +============ + +microTVM aims to achieve these design goals: + +1. **Portable Code**. microTVM can translate any Relay model into C code that can compile with only + a C standard library. +2. **Minimal Overhead**. microTVM generates target-specific, highly optimized code. As much overhead + from the runtime should be removed. +3. **Accessible Code**. microTVM considers C source code as a first-class output mechanism so that + it is easier for a firmware engineer to understand and tweak. microTVM Review comment: deleted, thanks ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
