manupa-arm commented on pull request #8472:
URL: https://github.com/apache/tvm/pull/8472#issuecomment-881370674
Hi @tqchen,
Thanks for the elaboration on the choices that we could take to support
constants in TIR.
> For the particular application in mind usually, C3 certainly brings the
least amount of indirection. However, it also brings deeper surgery to the IR
architecture itself. In the extreme case, one might fall into a that introduces
new IR node for each kind of application to touch meta-data.
I think it should be stated --link-params are just one of useful usage of
having the ability to express constant data in TIR. However, having the ability
to express constant in TIR allows to introduce constants that are properties of
the operators, mutations required by the HW, etc. Therefore, this is an
extension of the IR rather than a surgery to the IR. I'd view surgery as
rooting out an establied TIR construct and relocating/redefining with to
support more features/usecases. Moreover, the process of compilation for
variety of targets will include many constants that undergoes different
transformation for optimization and also to be prepared as requirement for the
HW to execute/decode them.
I think we could agree that having to introduce an IR node for each
meta-data (of the model) might not be desirable. However, we feel ability to
express constants in the TIR goes beyond them being meta-data. This is because
the constants does not necessarily are meta-data of the model. Frankly, Im not
even sure why we call constants involved in a computation as meta-data ?
Therefore, its better to understand why we are categorizing constant
non-scalar data as meta-data (always) is quite important in making the decision.
> The main possible problem in C2 is that we want to make sure meta data in
attrs remains constant during rewrite, so that the my_param_var won't get
rewritten to something else, and passes does not need to consider the context
to check if all variables are bound.
So to summarize, the tir builtin will provide a write-acess restriction,
correct? I feel that is an overkill where we should use a 'const' specifier (or
qualify the buffer_var to just loadable and not storeable) to restrict if thats
required. Then again, not sure what value it brings because users will not
write TIR directly -- unlike 'const' specifier in other languages.
> C1 is the approach that falls within the current architecture of meta data
handling. While I agree that it may not be the most direct option at the
moment, it will bring the least amount of impact to the TIR architecture
itself. We also want to make sure that the metadata handling mechanism works
for cases like this and possible future ones, and improvement if things does
not work out well.
I'd agree with this if we have good reasons why constant non-scalar data is
always treated as metadata. That is dependent on the resolution for the above
point. Then again, I think we should welcome positive extention to the IR which
does not involve a surgery.
> Popping up a level, we want to carefully think about the general need to
referring to a constant meta data, and perhaps build some more native mechanism
than C1, but also generalizes the particular case of allocate_const. For
example, maybe we want to allow relay and TIR to share a same mechanism of such
constant data reference. That is the long term solution that we might want to
think about along the directions(e.g. bring the constant map as part of
IRModule).
While all meta-data are constants, we would need to think whether all
constants are meta-data :) . Again, related to the same point above.
Let say, we had good reasons to decide all constants are meta-data. Then,
we'd propose not to pull out constants that are accessed by primfuncs early on
to IRModule scope -- that could happen later. This allows different compilation
flows to contribute the constants to for global optimizations later on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
