masahi opened a new pull request, #13259: URL: https://github.com/apache/tvm/pull/13259
I'm working on improving tuning time on quantized models using tensor core. One of the reasons it is slow is that the function below, which is called thousands of times during each evo search initial population sampling, takes more than 50 msec: https://github.com/apache/tvm/blob/a4840e7de38c5a2000917f2101f3ec4a374bcd39/src/meta_schedule/search_strategy/evolutionary_search.cc#L506-L517 The time it takes can be broken down into the followings: * `ApplyToTrace`: 35 msec * `RewriteTensorize`: 7-8 msec * `VerifyGPUCode`: 7-8 msec `RewriteTensorize` is relatively slow because it invokes `sch->Tensorize(...)` which does some heavy lifting. Since it is called thousands of times, it quickly adds up. It is wasteful if the tensorized schedule is rejected by `VerifyGPUCode` immediately after that. Since `tensorize` doesn't affect the validity of a sample, we can improve tuning time slightly by running `VerifyGPUCode` first and do `RewriteTensorize` only on valid samples. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
