hogepodge commented on a change in pull request #27: URL: https://github.com/apache/tvm-rfcs/pull/27#discussion_r706370057
########## File path: rfcs/0027-formalize-documentation-organization.md ########## @@ -0,0 +1,442 @@ +- Feature Name: `Formalize TVM Documentation Organization` +- Start Date: 2021-09-01 +- RFC PR: [apache/tvm-rfcs#0027](https://github.com/apache/tvm-rfcs/pull/0027) +- GitHub Issue: [apache/tvm#00xx](https://github.com/apache/tvm/issues/0000) + +# Summary +[summary]: #summary + +This RFC proposes a refactoring of TVM documentation. The goal of this refactor +is to create a document architecture that classifies four major documentation +types: + +* Tutorials, +* How-tos, +* Deep Dives, +* and Reference + +then organizes the documents based on those types. The desired result is to +make it easier for the entire TVM community to find documentation that meet +their needs, whether they are new users or experienced users. Another goal is +to make it easier for the developer community to contribute to the TVM +documentation. It recognizes that while in most communities there is a distinct +divide between the user and the developer communities, there can be significant +overlap given the nature of TVM as an optimizing compiler. + +# Motivation +[motivation]: #motivation + +TVM has seen an explosion of growth since it was released as an open source +project, and formally grauated as an official Apache Software Foundation +project. The vision of the Apache TVM Project is to host a "diverse community +of experts and practitioners in machine learning, compilers, and systems +architecture to build an accessible, extensible, and automated open-source +framework that optimizes current and emerging machine learning models for any +hardware platform." + +The TVM community has done an excellent job in producing a wide range of +documents to describe how to successfully install, use, and develop for TVM. +The documenation project grew with the community, to address the immediate +needs of the developer community. However, one consistent piece of feedback is +that the documentation is difficult to navigate, with beginner material mixed +in with advanced material. As a result, it can be difficult for new users to +find the exact information they need, and can work against the vision of the +project. + +This RFC aims to refactor the organization of the TVM docs, loosely following +the [formal documentation style described by +Divio](https://documentation.divio.com). This system has been chosen because it +is a: + +> "simple, comprehensive and nearly universally-applicable scheme. It is proven +> in practice across a wide variety of fields and applications." + +This RFC is primarily concerned with the organization of the documents, and not +the content. As such, the implementation of this RFC would move documents, and +only create new documents as top-level placeholders, indexes, and documentation +about the system itself. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## The Four Document Types + +### Introductory Tutorials + +These are step by step guides to introduce new users to a project. An +introductory tutorial is designed to get a user engaged with the software +without necessarily explaining why the software works the way it does. Those +explanations can be saved for other document types. An introductory tutorial +focuses on a successful first experience. These are the most important docs to +turning newcomers into new users and developers. A fully end-to-end tutorial, +from installing TVM and supporting ML software, to creating and training a +model, to compiling to different architectures will give a new user the +opportunity to use TVM in the most efficient way possible. A tutorial teaches a +beginner something they need to know. This is in contrast with a how-to, which +is meant to be an answer to a question that a user with some experience would +ask. + +Tutorials need to be repeatable and reliable, because the lack of success means +a user will look for other solutions. + +### How-to Guides + +These are step by step guides on how to solve particular problems. The user can +ask meaningful questions, and the documents provide answers. An examples of +this type of document might be, “how do I compile an optimized model for ARM +architecture?” or “how do I compile and optimize a TensorFlow model?” These +documents should be open enough that a user could see how to apply it to a new +use case. Practical usability is more important than completeness. The title +should tell the user what problem the how-to is solving. + +How are tutorials different from how-tos? A tutorial is oriented towards the +new developer, and focuses on successfully introducing them to the software and +community. A how-to in contrast focuses on accomplishing a specific task within +the context of basic understanding. A tutorial helps to onboard, a how-to helps +to accomplish a task. + +### Reference + +Reference documentation describes how the software is configured and operated. +APIs, key functions, commands, and interfaces are all candidates for reference +documentation. These are the technical manuals that let users build their own +interfaces and programs. They are information oriented, focused on lists and +descriptions. You can assume that the audience has a grasp on how the software +works and is looking for specific answers to specific questions. Ideally, the +reference documentation should have the same structure as the code base and +generated automatically as much as possible. + +### Explanations (Deep Dive) + +Background material on a topic. These documents help to illuminate and +understand the application environment. Why are things the way they are? What +were the design decisions, what alternatives were considered, what are the RFCs +describing the existing system. This includes academic papers and links to +publications relevant to the software. Within these documents you can explore +contradictory and conflicting position, and help the reader make sense of how +and why the software was built the way it is. It’s not the place for how-to’s +and technical descriptions, and instead focuses on higher level concepts. + +## Special considerations for TVM + +The TVM comunity has some special considerations that require deviation from +the simple docs style outlined by Divio. The first is that there is frequently +overlap between the user and developer communities. Many projects document the +develper and user experience with separate systems, but it is appropriate to +consider both in this system, with differentiations where appropriate. As a +result the tutorials and how-tos will be divided between "User Guides" and +"How-to Guides". + +The next is that there are special topics within the TVM community that benefit +from additional attention. These topics include, but are not limited to, +microTVM and VTA. Special "Topic Guides" can be created to index existing +material, and provide context on how to naviagate that material most +effectively. + +To facilitate newcomers, a special "Getting Started" section with installation +instructions, a overview of why to use TVM, and other first-experience douments +will be produced. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## Document Organization + +### Top Level Organization + +* Getting Started +* User Guide +* Topic Guide +* Developer Guide +* TVM Architecture Guide +* API Reference (reference) +* Index + +### Organization with Major Sections + +* Getting Started + * About TVM + * Installing TVM + * Contributor Guide +* User Guide + * Tutorial + * How To +* Topic Guide + * MicroTVM Guide (index to existing docs) + * VTA (index to existing docs) +* Developer Guide + * Contributor Tutorial (new, to be written) + * How To +* Architecture Guide + * Architecture Overview (new, diagram/map, to be written) + * ... +* API Reference (reference) + * Generated C++ Docs… + * Generated Python Docs… +* Index + +### Organization with Detailed Description + +* Getting Started + * About TVM + * Installing TVM + * Contributor Guide + * Community Guideline + * Performing Code Reviews + * Committer Guide + * Writing Document and Tutorials + * Code Guide and Tips + * Error Handling Guide + * Submitting a Pull Request + * Git Usage Tips + * Apache TVM Release Process +* User Guide + * Tutorial + * Introduction + * An Overview of TVM and Model Optimization + * Installing TVM + * Compiling and Optimizing a Model with TVMC + * Compiling and Optimizing a Model with the Python Interface (AutoTVM) + * Working with Operators Using Tensor Expression + * Optimizing Operators with Schedule Templates and AutoTVM + * Optimizing Operators with Auto-scheduling + * Cross Compilation and RPC + * Introduction to TOPI + * Quick Start Tutorial for Compiling Deep Learning Models + * How To + * Install TVM + * Install from Source + * Docker Images + * Compile Deep Learning Models + * Deploy Deep Learning Models + * Work With Relay + * Work with Tensor Expression and Schedules + * Optimize Tensor Operators + * Auto-Tune with Templates and AutoTVM + * Use AutoScheduler for Template-Free Auto Scheduling + * Work With microTVM +* Topic Guide + * MicroTVM Guide (index to existing docs) + * -> Work With microTVM + * -> microTVM Architecture + * VTA (index to existing docs) +* Developer Guide + * Contributor Tutorial + * ... + * How To + * Write an operator + * Write a backend + * ... +* Architecture Guide + * Architecture Overview + * Research Papers + * Frontend + * Relay: Graph-level design: IR, pass, lowering + * TensorIR: Operator-level design: IR, schedule, pass, lowering + * TOPI: Pre-defined operators operator coverage + * AutoScheduler / AutoTVM: Performance tuning design + * Runtime & microTVM design + * Customization with vendor libraries BYOC workflow + * RPC system + * Target system +* API Reference (reference) + * Generated C++ Docs… + * Generated Python Docs… +* Index + +## Document Code Organization + +This refactor will require a shift of how the documents are organized. In +general, Tutorials and How-Tos are written as Sphinx Gallery documents, +allowing for the generation of text, python source, and Jupyter Notebooks. This +allows the user to consume these working code samples in a number of ways, but +comes at the cost of fixed format that can be confusing to navigate. To help +mitigate this, the tutorials and how-tos will be broken up into a more fine +grained directory structure. For example: + + tvm/ + gallery/ + dev_how_tos/ + compile_models/ + ... + how_tos/ + tutorial/ + +Rather than render the gallery in one pass as a nested structure (resulting in +a single page with multiple sections), instead each directory will be rendered +independently. This will aid in navigation through the galleries, and also give +more fine-grained grouping of similar topics. The naming of the directory +reflects the organization of Sphinx documentation folder, for example: + + tvm/ + docs/ + deep_dive/ + how_tos/ + index.rst + **compile_models/** + ... + reference/ + **tutorial/** + dev_deep_dive/ + dev_how_tos/ + dev_reference/ + dev_tutorial/ + +Depending on the type of documentation, some of the directories may be +generated. For example, the tutorial and compile_models directories are +auto-generated by Sphinx Gallery. To add a new Sphinx Gallery requires the +following steps: + +1. Create a gallery subdirectory with the how-to or tutorial documents +2. Create entries in the docs conf.py example_dirs and gallery_dirs variables + to reference the source and target directories. +3. Update the appropriate index pages in the docs subdirectories to add the new + directories to the Sphinx table of contents. + +# Drawbacks +[drawbacks]: #drawbacks + +One consistent drawback of this approach is how major subprojects are handled. +For example, microTVM may require a specific set of tutorials and how-tos, but +these can become mixed in with other TVM specific documents. This will be +mitigated through two means: + +* Subdirectories within the How-Tos can target specific topics. +* Landing pages can be created for specific topics that collect links to all of + the pages related to that topic. + +Another drawback is that this format may require a user to dig deeper on the +first run experience, requiring them to dig into a tutorial or how-to to +install the software. This can be mitigated by refactoring the landing page to +include a “Quick Start” guide for installing the TVM software. + +Throughout the open source ecosystem, there is often a distinction between +documentation for users and documentation for developers. The TVM community is +unique in that frequently users will need to extend TVM to accomplish some +goal, for example adding a new backend for code generation. This issue is +addressed by dividing the user and developer topics, but keeping them within +the same documentation system. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +This style of documentation has been formalized by developed by Divio 3 and +deployed throughout the open source communities. Although it can be difficult +to characterize documents within the system (“Should this be a developer or +user doc?” “Is this a tutorial or a how-to?”), working within the constraints +of a formalized system brings many benefits: + +It helps prevent documentation sprawl. Rather than create new top-level +headings to capture new ideas, new ideas are logically documented at different +levels of detail within the for existing types. It creates a consistent user +experience. Users know exactly where to look depending on their needs. New +users will find a path to success through tutorials, while existing users who +need to solve common tasks can look to the how-tos for guidance. It encourages +new documentation. Developers have a framework for what docs should look like, +and where they should go. It takes advantage of current content. A +proof-of-concept implementation of this method consisted largely of moving new +documents. It creates a framework to improve existing content. Many how-tos +duplicate steps repeatedly. This will allow us to identify the duplications and +refactor the documents into more targeted forms. In researching documentation +systems, there aren’t many formalized systems that have been published. Review comment: Fixed, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
