csullivan opened a new pull request #7688:
URL: https://github.com/apache/tvm/pull/7688
This PR adds generic support for the use of storage scope in Relay's graph
runtime lowering path. The key conceptual additions made are:
- Introduce a target dependent CollectStorageInfo pass which returns a
storage mapping for the outputs of each Expr in a Relay function. The consumers
of the output storage scope annotations are:
- The graph runtime memory planner, which can utilize the output storage
scope when doing token allocation optimization
- The graph runtime codegen, which annotates the graph json with these
storage scopes.
- The target dependent CollectBufferBinds packed function which translates
these storage scopes into buffers to be bound when lowering a Relay primitive
func in CompileEngine (via the binds field of tvm::lower/build)
- Finally the graph runtime, which utilizes the storage scope annotations
in the graph json when setting up data space storage
The two introduced CollectStorageInfo and CollectBufferBinds are dispatched
to via a target dependent packed function. If no implementation exists for a
given target, the storage scope of each Expr defaults to global scope and the
runtime is unaffected.
RFC in progress, once posted I will add a link here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
