denise-k commented on a change in pull request #50: URL: https://github.com/apache/tvm-rfcs/pull/50#discussion_r782486145
########## File path: rfcs/0050-roadmaps.md ########## @@ -0,0 +1,320 @@ +- Feature Name: roadmaps +- Start Date: 2022-01-07 +- RFC PR: [apache/tvm-rfcs#0050](https://github.com/apache/tvm-rfcs/pull/0050) +- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000) +- pre-RFC: https://discuss.tvm.apache.org/t/pre-rfc-tvm-roadmap/11171 +- pre-RFC: https://discuss.tvm.apache.org/t/pre-rfc-roadmap-rfcs/11711 +- Co-Authors: Denise Kutnick (@denise-k), Andrew Reusch (@areusch) + +NOTE(areusch): This RFC is a combination of the two above pre-RFCs, and limited copy-editing has +been done to preserve the flow of the document. + +## Summary + +This RFC proposes to add product roadmaps to TVM. + +Roadmaps should be seen as a way of unifying the planning process of TVM, all the way from ideas to +PR merges. The roadmaps discussed in this pre-RFC are intentionally designed to integrate with TVM’s +existing planning tools (e.g. Github tracking issues, RFCs), while adding an additional space for +sharing and collaboration earlier in the R&D process. + +This proposal includes two categories of roadmaps: + +- **TVM Global Roadmap**: This roadmap aims to provide a holistic view of all components within TVM, + focusing on features which broadly affect the entire project or add significant functionality. +- **TVM Component Roadmaps**: These roadmaps aim to provide a more detailed view of a subsection of + TVM: including vertical efforts involving TVM's intermediate representations (such as Relay or + TIR) and the horizontal efforts which span across TVM's full stack (such as Automation or + Documentation). Details about each of the components in TVM will be discussed later in this + pre-RFC. + +## Motivations + +- Integrate TVM’s technical vision and task tracking mechanisms together +- Provide everyone with visibility into all of TVM’s projects +- Enable individuals and organizations that contribute to TVM to influence the vision of TVM via + roadmaps +- Encourage individuals and organizations that contribute to TVM to share the details of their + longer-term efforts +- Ensure a openly-governed, equitable process towards creating new roadmaps for TVM + +## Guide-Level Explanation + +This guide-level explanation will describe the basic structure of the roadmap, without focusing +heavily on the intended contents. + +### Tooling + +The roadmap utilizes Github Projects as the underlying tooling mechanism. This maximizes reuse of +existing content tracked in Github, and is very user-friendly for existing Github users. + +This RFC proposes to place the TVM roadmaps directly in the `apache/tvm` repository, since the +roadmap is intended to be a community-focused project, and having the roadmaps directly in TVM would +help to maximize visibility to the TVM community. + + + +### Sample Roadmap + + + +Above is a sample roadmap for **TVM CI and Testing**, with annotations to show several important +features within the roadmap: + +1. **Background & Motivations** **Column:** This section is located on the left-most part of the + roadmap, and is intended to define a few key focus areas of the project and their overall success + criteria. In the **TVM CI and Testing** roadmap, this section contains definitions of Coverage, + Ease of Use, Runtime, Stability, and Reporting. +2. **Quarterly Planning Columns**: Following **Background & Motivations**, a roadmap has 4 quarterly + planning columns (one column per upcoming quarter through the next year). Each Quarterly planning + column has these cards: + - **Themes & Goals:** The **Themes & Goals** are listed in the top-most card of each column of + the roadmap, and are intended to show how the focus areas of the project are addressed that + quarter. + - Themes are chosen from those defined in the **Background & Motivations** section (such as + Stability or Runtime). + - Goals describe how the **Roadmap Items** listed in each column contribute to the success + criteria listed in **Background & Motivations**. + + For example, in Q3 of 2021 (the last 3 months of development in TVM), two of the major focus + areas in **TVM CI and Testing** were Stability and Runtime. + + - **Roadmap Items:** Each column of development contains roadmap items which contribute to each + of the **Themes & Goals** described above. These items are meant to unify feature development + within each project, whether it is coming from an RFC, Github task tracking, or directly from + within the roadmap. Items on the roadmap generally have feature ownership at least partially + identified at a minimum. +3. **Backlog Column**: Items on the roadmap which don't have an owner or a proposed development + timeline fall into the **Backlog Items** column. This is also the default location where new + roadmap issues are placed. + +## Roadmap Process + +This section proposes the **Roadmap-RFC** process, which aims to encapsulate the processes discussed +in the Overview section. The **Roadmap-RFC** is used to create, modify, or delete roadmaps. Details +on the process are listed below. + +### Template of a Roadmap's Contextual Information + +The following fields provide some contextual information for each roadmap. + +- **Roadmap Name:** Name of the roadmap +- **Roadmap Maintainers**: These community members are primarily responsible for adding, modifying, + and removing roadmap items. **Any community member can be listed as a maintainer of a roadmap, + regardless of contributor status within the Apache organization.** + - Note: If a Roadmap-RFC's proposed maintainer does not have TVM Committer or PMC status within + the Apache organization, they will receive an invitation for a Triage role in Apache TVM as soon + as their Roadmap-RFC is accepted. This will allow them to maintain and triage roadmaps while + maintaining the Apache community contribution process. +- **Roadmap Summary/Description:** A brief description of the roadmap + +### Establishing Scope and Themes for a new Roadmap + +For each roadmap, a set of **scope** and **themes** should also be defined, in order for TVM's +community members to quickly learn about a roadmap's key focus areas. + +**Establishing Scope** + +- How are the tasks tracked in this roadmap grouped together? How can we think about this grouping + distinct from those made in other roadmaps? It's okay for there to be overlap with other roadmaps, + but the scope defined here should motivate a separate roadmap. +- Is the proposed roadmap intended to represent a perpetually ongoing set of efforts, or is there an + end goal which will close/finalize the roadmap? +- Does the proposed roadmap have any scope overlaps with any existing roadmaps? If so, please list + them. + +**Establishing Themes** + +- List 4-6 proposed "themes" of the roadmap, intended to convey the purpose of the roadmap and the + types of tasks that should be added. + - Some examples of themes are `programmability`, `portability`, and `performance`. + - Themes are intended to group items within a roadmap. This helps us to understand the scope of + the roadmap. +- For each theme, include a set of definitions specific to the proposed roadmap. + - What does this theme mean in the context of this roadmap? + - Are there multiple definitions for this theme? For example, `performance` could be interpreted + as tuning times, runtime latency, or a number of other definitions. +- For each theme, include a set of success criteria specific to the proposed roadmap. + - What types of metrics would be relevant to this roadmap? + +### Adding Items to a Roadmap + +A roadmap's maintainers are the primary folks responsible for adding, modifying, and removing items +for a roadmap. This isn't a hard and fast rule—for example, it may be expedient for other community +members to triage new roadmap items into a roadmap's backlog. However, the maintainers should be +considered the "owners" of a roadmap, and generally no rigid process is defined around modifying the +items in a roadmap. + +Roadmaps are defined using [GitHub +Projects](https://docs.github.com/en/issues/trying-out-the-new-projects-experience/about-projects) +and can include GitHub Issues, Pull Requests, and simple note cards. Maintainers should strive to +place mainly **GitHub Issues** in Roadmaps to make it possible for the community to learn more about +ongoing work and reduce triage burden. + +Each item on a roadmap is intended to track one of these community processes: + +- **pre-RFCs** serve as a way to begin discussions on planned work at the earliest stage of + maturity. **pre-RFCs** are typically posted in the [TVM Discuss + forums](http://discuss.tvm.apache.org) in order to solicit TVM Community feedback. For an example + of a **pre-RFC**, see the screenshot of Andrew R's proposal to *Convert RST Docs to Markdown* + below. + + **pre-RFCs** can be tracked on a Roadmap by preemptively creating a GitHub Task-tracking Issue + in [tvm-rfcs](https://github.com/apache/tvm-rfcs). +  + +- **RFCs** serve as a way to share feature proposals with the TVM Community. The + [tvm-rfcs](https://github.com/apache/tvm-rfcs) repo is used for the creation and acceptance of + **RFCs**. For some examples of **RFCs**, see the screenshot of open pull requests in + [tvm-rfcs](https://github.com/apache/tvm-rfcs) below. + + Open **RFCs** can be directly linked into any roadmap. Once an **RFC** is accepted, please use + the **Github Task-Tracking** process to track **RFC** Execution. +  + +- **Github Task-Tracking Issues** are used in [tvm](https://github.com/apache/tvm) to share the + progress of accepted RFCs over time. For an example of a **Github Task-Tracking Issue**, see the + screenshot of Andrew L's RFC to *Add Mixed-Precision Support to TVM* below. + + These task-tracking issues can be directly linked into any roadmap. +  + +- **Bugfixes** are actionable GitHub Issues not necessarily connected to a Task-Tracking + Issue. Generally, they require only 1 PR and the work is clearly specified in the issue. For an + example of a **bugfix** Github Issue, see the screenshot of a flaky CI test report below. + + You can directly link **bugfixes** in a roadmap by adding their corresponding GitHub Issues as a + card. Generally, it’s the maintainer’s responsibility to combine related **bugfixes** in order + to keep the roadmap concise. For example, if you have 10 flaky tests in CI, it might make sense + to have a global tracking issue for flaky CI tests which points to each individual bug. +  + Review comment: This is a great callout. I agree that a roadmap's maintainers should have the ability to track these types of issues on the roadmap. Perhaps this falls under Github Task-Tracking, and we should just change the wording so that we're not boxing that particular category in as "accepted RFCs only" - what are your thoughts @driazati @areusch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
