chunit-quic commented on code in PR #13212:
URL: https://github.com/apache/tvm/pull/13212#discussion_r1109437322
##########
python/tvm/relay/frontend/common.py:
##########
@@ -1067,6 +1067,20 @@ def __init__(self, span):
self._span =
tvm.relay.Span(tvm.relay.SourceName(span.decode("utf-8")), 0, 0, 0, 0)
else:
assert False, f"unsupported span type: {type(span)}"
+ self.suffix_str = "_PART_"
+ self.counter = 0
+ self.distance_from_leaf = -1
+
+ def _create_span(self):
+ """Adds suffix_str + counter value to _span.source_name.name,
+ to create a unique source_name for the Relay layer
+ """
+ if self.distance_from_leaf == 0:
+ return tvm.relay.Span(tvm.relay.SourceName(self._span), 0, 0, 0, 0)
+ self.distance_from_leaf -= 1
+ span_str = "{}{}{}".format(self._span.source_name.name,
self.suffix_str, str(self.counter))
+ self.counter += 1
+ return tvm.relay.Span(tvm.relay.SourceName(span_str), 0, 0, 0, 0)
Review Comment:
Hi @arina-grovety and @lhutton1
Thank you for informing us at once :D. Sorry that I didn't notice the
rewriting part in the invoked pass. Before going to details, our suggestion
will be "write a post-processing to tag span suffix" specifically for this kind
of scenario. So that we could have a workaround and still keep suffix being
discarded as default.
Here are the reasons in short below:
1. We still cannot find methods to handle the impacts after introducing the
suffix as default.
2. Span somehow will be changed according to the functionality of pass.
Now I would like to go through the process about why we would suggest
writing the post-processing pass. I will try to describe the goals of this PR,
and what problems might be encountered. Please kindly correct me if I still
miss something. :)
# Goal
Obtain the mapping between the offloaded result and the frontend operators.
The flow should be:
```
frontend model ->
Relay IR (IR1) ->
Relay IR transfomred by partition_for_ethosu() (IR2)->
tvmc.complie_model ->
Relay IR transfomred by convert_graph_layout and partition_functions (IR3)->
dump initial_relay offloaded result
```
Based on the codes
PR([compiler.py:347](https://github.com/apache/tvm/blob/325b2ef24496eadc14836cb2c8a168a06dff37e8/python/tvm/driver/tvmc/compiler.py#L347)),
we would like to get the mapping via IR3 and IR2.
# Requirement
We need a unique indicator to maintain the relation offloaded result. If the
`source_name` of `span` is unique, it might be a choice. Therefore here comes
two proposals:
1. Add suffix to span as default.
2. Add unique id to span.
# Challenges we will encounter
In this section we would try to describe what challenges we encountered.
Since the expressions are modified by partition_functions, we cannot use the
hash key of expressions. Now let's check the proposals in the previous section.
1. Add suffix to span as default.
The drawback of setting suffix as default is still significant. Like the
discussions in the
[preRFC](https://discuss.tvm.apache.org/t/pre-rfc-tvm-explorer-infrastructure/13457/29#how-many-benefits-can-we-get-from-suffix-2).
We still don't encourage to add it back as default.
2. Add unique id to span
An obvious obstacle we will face is, it becomes tough to compare the
equality between span once we add an unique id. Another problem is, with the
change of expressions, span might be changed also according to the
functionality of pass.
Take the concept of foldconstant pass for example, consider the following
transformation:
```python
IR1:
fn(%input) {
%0 = add(meta[relay.Constant] /* span=const_1*/, meta[relay.Constant]
/* span=const_2*/]) /* span=add*/
%1 = subtract(%input, %0) /* span=sub */
}
# After foldconstant pass it should look like
IR2:
fn(%input) {
%0 = subtract(%input, meta[relay.Constant] /* span=[const_1, const_2,
add]*/) /* span=sub */
}
```
The span of the folded constant expressions is from two Constant and an
add Call. Therefore, even if we have the unique id of const_1, const_2 and
add span, we still cannot have a better experience to get the relation.
Note that if you are interested in how to construct a many-to-one span, it
is [one of the ongoing
features](https://discuss.tvm.apache.org/t/pre-rfc-tvm-explorer-infrastructure/13457#pass-source-information-builder-6)
in the preRFC of TVM Explorer infra. We will try to submit that PR recently.
# Conclusion
From my point of view, it is still challenging for us to add back the
suffix, and unique ID
for span might not be easy to deal with either. However, we do observe that
making span as an unique indicator is quite useful in some cases. Therefore we
will suggest writing a distinct pass to tag suffix to the span. With the help
of the suffix-tagging pass, we can have a flexibility to work with and still
keep the suffix problem away from the build flow by default.
Here is a sudo persudo code for your reference.
```python
from collections import defaultdict
class _SuffixTagger(ExprMutator):
"""_SuffixTagger"""
def __init__(self):
ExprMutator.__init__(self)
# key: span or source name, value: counter, indexed from 0
self.lookup = defaultdict(int)
self.suffix = "_PART_"
def _tag_suffix(self, span):
# To avoid error once we introduce the SequentialSpan in the future
#https://discuss.tvm.apache.org/t/pre-rfc-tvm-explorer-infrastructure/13457#pass-source-information-builder-6
# Don't need this if currently
if isinstance(span, relay.Span):
ori_name = span.source_name.name
new_name = ori_name + self.suffix + self.lookup[ori_name]
lookup[ori_name] += 1
return tvm.relay.Span(
tvm.relay.SourceName(new_name), span.line, span.end_line,
span.column, span.end_column
)
return span
def visit(self, expr):
if hasattr(expr, "span"):
return super().visit(expr)
def visit_call(self, call):
new_args = [self.visit(arg) for arg in call.args]
new_op = self.visit(call.op)
return _expr.CallWithFields(
call, new_op, new_args, call.attrs, call.type_args, None,
self._tag_suffix()
)
```
Thank you for reading such long post. If there is any question please don't
hesitate to let me know. I will get back to you soon. :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
