bartekkuncer edited a comment on pull request #19562: URL: https://github.com/apache/incubator-mxnet/pull/19562#issuecomment-742603472
> Hi, can we compare https://github.com/apache/incubator-mxnet/pull/19601 ? @kpuatamazon Sorry for the late response, was waiting for the layer norm optimization in oneDNN. Below are the results I got using marian and my onednn implementation. It looks like oneDNN is faster in most cases. Can you tell me what CPU are you using? threads amount | 28 threads | 28 threads | 4 threads | 4 threads -- | -- | -- | -- | -- execution | marian | mx w/ onednn | marian | mx w/ onednn 1000x5 | 0,0213 | 0,032 | 0,0272 | 0,029 1000x100 | 0,03 | 0,0859 | 0,0664 | 0,0421 300x512 | 0,0255 | 0,1505 | 0,0667 | 0,073 500x512 | 0,0305 | 0,2436 | 0,0859 | 0,0449 1000x2048 | 0,1396 | 0,0616 | 0,4103 | 0,2594 1000x3 | 0,0156 | 0,0239 | 0,018 | 0,0242 45x512 | 0,0152 | 0,038 | 0,0252 | 0,0377 1000x5x100 | 0,0652 | 0,0457 | 0,2194 | 0,117 1000x8x100 | 0,0963 | 0,0538 | 0,3147 | 0,1594 300x512x512 | 8,0384 | 7,8368 | 31,491 | 28,0296 500x512x10 | 0,6231 | 0,3557 | 3,9009 | 1,0595 1000x5x2048 | 2,1114 | 1,574 | 4,9501 | 3,8672 1000x2048x3 | 2,6274 | 1,3618 | 12,9816 | 5,8291 45x512x512 | 2,2648 | 1,7278 | 5,4215 | 4,6426 1000x5x30x200 | 3,6479 | 3,5311 | 13,3026 | 11,3592 100x100x10x300 | 3,6913 | 3,9369 | 13,4574 | 10,93 300x512x45x20 | 24,2647 | 16,6294 | 146,4946 | 76,6008 50x512x40x30 | 5,3653 | 4,0195 | 27,1334 | 15,8884 100x2048x10x10 | 6,2846 | 3,4051 | 34,9686 | 13,3237 1000x3x10x200 | 0,9976 | 0,5507 | 2,1544 | 1,0273 45x52x300x45 | 4,7193 | 3,9909 | 22,6179 | 14,0623 100x5x30x20x10 | 0,7225 | 0,3956 | 4,5843 | 1,3655 100x4x100x10x30 | 2,7491 | 1,9519 | 10,9089 | 6,3355 300x52x2x45x20 | 5,7044 | 3,9567 | 30,7142 | 15,74 500x52x3x40x30 | 13,8315 | 10,8504 | 80,604 | 46,8657 100x28x10x10x10 | 0,6829 | 0,4126 | 4,3192 | 1,2705 100x3x10x18x200 | 2,1217 | 1,752 | 5,232 | 4,2747 45x512x30x4x45 | 15,4316 | 12,9341 | 86,8337 | 54,7685 I built mxnet using: cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DUSE_BLAS=mkl -DUSE_CUDA=0 -DUSE_LAPACK=0 -DUSE_GPERFTOOLS=0 -DUSE_OPENCV=0 .. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
