YuchenJin commented on PR #89: URL: https://github.com/apache/tvm-rfcs/pull/89#issuecomment-1220178200
Hi @leandron, thanks for your feedback! :) We share a common goal of minimizing disruption while incrementally improving TVM. One of the main questions is how to bring in the improvements. That’s indeed what we have carefully thought about. One thing we found in building the unity connection is that we need to **co-design things together**. Take first-class symbolic shape support as an example: Suppose we want to apply a `flatten` operation to flatten a tensor, here is the symbolic shape representation in Relax: ```python b: Tensor[(m, 224)] a: Tensor[(m * 224, )] = flatten(b) ``` In Relay, we have `?` to denote an unknown dimension. So the above program in Relay is: ```python b: Tensor[(?, 224)] a: Tensor[(?, )] = flatten(b) ``` Without symbolic shape, we lose the shape relation between tensor `a` and tensor `b`, which prevents good optimization opportunities for example memory planning to reuse the memory between `a` and `b` since we know at compile time they occupy the same amount of memory. Supporting this requires introducing native symbolic shape support as opposed to a separate mechanism. It's worth noting that from the example above, the shape mechanism in Relax is very different from what we currently have in Relay. Additionally, the other improvements also benefit from first-class symbolic shape. For example, `call_tir` signature has `output_shape` which can be represented by symbolic shape (since TensorIR supports symbolic shape), and TE language is based on symbolic shape hence we can directly generate PrimFunc with `emit_te` (see [Relax-TE integration section](https://github.com/YuchenJin/tvm-rfcs/blob/relax-upstream-rfc/rfcs/0089-relax-upstreaming.md#44-relax-tetopi-integration)). Introducing each component separately will make the design less cohesive. Directly introducing this symbolic shape support to existing IR would mean **one-stop transition** to the current Relax, which is not incremental improvements as we hope for. Relax can be viewed as complementary to Relay. Relay focuses on high-level op transformations, while the current Relax passes focus on TIR-graph co-transformations that can enable flexible fusion and layout rewrite, which is hard to achieve in Relay. As of this RFC, we do not seek to change the default build pipeline or replace Relay. In this RFC, we only introduce Relax as an optional component for those community members who need it. It is a common practice in other ML frameworks, for example, PyTorch brought in TorchFX as an optional (vertical) component to support graph exporting, while maintaining TorchScript. We totally agree that TVM default flow evolution is important, and we should carefully discuss that with the community in future RFCs. [Evolving default build](https://discuss.tvm.apache.org/t/establish-tvm-unity-connection-a-technical-strategy/13344#evolving-default-build-14) has been briefly discussed in [Establish TVM Unity Connection — A Technical Strategy](https://discuss.tvm.apache.org/t/establish-tvm-unity-connection-a-technical-strategy/13344), and there will be an upcoming [tvm.compile RFC](https://discuss.tvm.apache.org/t/establish-tvm-unity-connection-a-technical-strategy/13344#relation-to-upcoming-tech nical-rfcs-2) to discuss the long-term strategy to consolidate default build flows in TVM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
