tqchen edited a comment on issue #15465: [RFC] Integrate TVM into Apache MXNet URL: https://github.com/apache/incubator-mxnet/issues/15465#issuecomment-509712700 Technically it is easy to drop in openmp as a threadpool backend of TVM. It is just not the default one. Historically tvm defaulted to openmp then evolved its own version of thread pool for better perf. The current thread pool already has accepted the OMP env variable(OMP_NUM_THREADS), making it consistent with the omp settings. Most of the operators touched so far are single-threaded so won’t be affected by the discussion. So to summarize the the answers to @ZhennanQin 's points. (1) and (2) likely won't be an issue because: TVM's thread number configurations and core settings are consistent with OMP(OMP_NUM_THREADS). (3) Might be an issue when consecutive operators are both compute-intensive and use different backends. Given that the initial operators that are going to be supported likely to use only single threads for memory-intensive ones, it won't be an issue. If in the case where compute-intensive ones are provided by TVM, we need to do benchmarks to decide which one is better. Mainly because according to @yidawang, using tvm's current pool has around 15% gains over omp. This gain might override the latency hit we get in the beginning. Note that the existing libraries MXNet use like NNPack has its own thread pools. Nevertheless, if necessary, tvm runtime could switch to use openmp for threads. Given that this is a technical issue that may or may not affect how to get the best speed of the newly added features. I would recommend us to keep it in mind and do benchmarks to decide the best option when we find it is necessary.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
