kpuatamazon commented on issue #17980: When compiled with MKL, fully_connected calls DNNL while dot and batch_dot call MKL URL: https://github.com/apache/incubator-mxnet/issues/17980#issuecomment-609906743 > I don't think it's possible at this moment. Can we please have a build option to force MKL usage for all GEMM calls? This is a pretty big performance regression. > I think the real question is how to improve the performance of FullyConnected when DNNL is used. Hmmm, when DNNL delegates to MKL, that happens internally to DNNL, right? If the MXNet wrapper is the problem, then why is delegating to MKL much faster? Applying my workaround: ``` cmake -GNinja -DUSE_CUDA=OFF -DCMAKE_BUILD_TYPE=Release -D_DNNL_USE_MKL=FULL -DMKLINC=/opt/intel/mkl/include .. ninja -j 30 export LD_PRELOAD=/opt/intel/mkl/lib/intel64/libmkl_rt.so export OMP_NUM_THREADS=4 ./mult_bench.py #NB I edited the script to label things more clearly. ``` Output: ``` Shape (5, 512, 512) 0.0000628 seconds for fullyconnected (MKL workaround) 0.0000488 seconds for dot (MKL) Shape (5, 512, 1536) 0.0000816 seconds for fullyconnected (MKL workaround) 0.0000661 seconds for dot (MKL) Shape (5, 512, 2048) 0.0001109 seconds for fullyconnected (MKL workaround) 0.0000957 seconds for dot (MKL) Shape (5, 2048, 512) 0.0001078 seconds for fullyconnected (MKL workaround) 0.0000954 seconds for dot (MKL) Shape (4, 512, 512) 0.0000501 seconds for fullyconnected (MKL workaround) 0.0000434 seconds for dot (MKL) ``` There is something less efficient about `FullyConnected` than `dot` but it's nowhere near explaining DNNL's slowdown. FYI I usually wear this hat (as opposed to @kpu) on Mondays so expect slower responses on other days.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
