comaniac commented on pull request #5613: URL: https://github.com/apache/incubator-tvm/pull/5613#issuecomment-630521369
> @comaniac I think the `dense_large_batch` is quite complicated, therefore adjusting the `dense_large_batch` directly to handle super large batch size is error-prone. In order to make `dense_large_batch` works for super large batch size, it seems that `block_cand`, 'vthread_cand' and `n_thread_cand` need to be adjusted, but I am afraid changing such parameters can bring in bad consequences. Besides, super large batch size is not very common, therefore creating a new schedule to handle such situations may be more convenient? I personally don't prefer adding more schedule templates for different workloads, because 1) we will end up with lots of ad-hoc schedules, and 2) we will need to tune all possible schedule templates during the already time-consuming AutoTVM process. As a result, unless the target workloads are specialized but common and widely used, I would not suggest to do so. On the other hand, I was working on the `dense_large_batch` and I know that it wasn't well-polished yet. The candidates (e.g., `block_cand`) are tuning space pruning heuristics. You can firstly remove them in the template and integrate with yours to see if that works for your workloads as well as some common CV workloads with large batch sizes. Then I can later file another PR to add the candidates back. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
