comaniac commented on issue #5277: [BYOC] Refine AnnotateTarget and MergeCompilerRegion Passes URL: https://github.com/apache/incubator-tvm/pull/5277#issuecomment-611078702 > Thanks for this, a few high level comments first. > > * Could we separate some of these changes out into separate PRs? In particular, separating > bug fixes from features so we can get them in quickly. It's hard to separate bug fixes to another PR because some fixes were done with refactoring. > * Am I right to think that the intended flow here is to only run AnnotateTarget once? I'm not sure how we would use it in the case where we have two codegens, one that wants to see the graph with QNN ops and the other than wants the graph without them. Yes I suppose the flow is intended to run AnnotateTarget only once, but I'm not sure I get your issue. If a user wants to have a QNN model then she will run the QNN pass first anyhow to generate QNN ops in the graph. Under this assumption, if a codegen doesn't want to see QNN ops, it should not have fannotate and AnnotateTarget will annotate QNN ops to other codegens or just default. > * What's the advantage of this multi-target approach over running the entire partitioning pipeline multiple times? I think I understand your concern more after this question. You're saying if a user wants to offload a graph to multiple targets then she will run AnnotateTarget->MergeCompilerRegion->PartitionGraph multiple times. IMHO, this is not an ideal flow due to the following reasons: 1. If the flow is designed for offloading only one target, it's hard to consider global optimization over multiple targets in the future. For example, 1st run with TargetA creates 2 subgraphs due to restriction set; 2nd run with TargetB creates 1 subgraph (so 3 subgraphs in total). However, assuming the second subgraph for TargetA can also be offloaded to TargetB, meaning that if we run TargetB first than we will get 2 subgraphs in total. 2. If the flow may take a graph that has been partitioned by another target in advance, all passes in this flow will need to deal with partitioned functions. It's hard to maintain and even harder to integrate global optimization in the above situation. 3. The position of PartitionGraph in the pass manager is meaningful. We guarantee external functions generated by this pass can be processed by following passes, but not others.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
