mbrookhart opened a new pull request #7368:
URL: https://github.com/apache/tvm/pull/7368
I recently spent a lot of time fighting dynamic rank issues in a kind of
crazy ONNX model. Fixing it required doing incremental dynamic-to-static before
type inference. This PR basically changes the logic of dynamic to static from
this:
```
infer type on the whole function
constant fold the whole function
then go find dynamic ops who's inputs are constant
replace them with static ops
Rinse and repeat
```
to this:
```
go find dynamic ops
infer type and constant fold their inputs
If the inputs are now constant, replace them with static ops
```
This has the advantage that it can analyze those crazy dynamic rank and
control flow graphs and simplify them, but it has the disadvantage that it's
slower than the previous version because we call infertype and constant folding
many more times.
Performance checking shows that this takes a BERT compile from ~15 seconds
to ~60 seconds. This should be fixable when incremental type inference becomes
available.
Thanks,
Matthew
cc @masahi @jroesch @tmoreau89 @jwfromm @electriclilies
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
