wuxun-zhang commented on issue #17159: Performance regression from 1.4.1 to 1.5.1 URL: https://github.com/apache/incubator-mxnet/issues/17159#issuecomment-568836514 Thanks @TaoLv for reminding the above different settings. @jonatan1626 Just updated the collected results on **c5.18xlarge**. I got the very different performance numbers of mxnet-mkl 1.5.1 with yours. Could you double check in your side? **With below commands (using all physical cores on 1 socket):** ``` export OMP_NUM_THREADS=18 export KMP_AFFINITY=granularity=fine,noduplicates,compact,1,0 numactl --physcpubind=0-17 --membind=0 python gluon_resnext50.py --iteration 10000 ``` ``` 1.4.1 Install: pip install mxnet-mkl==1.4.1 Output: Model: resnext-50 ----Latency---- Model Partition Time: 0 Model Load Time: 68.3131217956543 First Inference: 90.35015106201172 p99: 14.545679092407227 p90: 14.46223258972168 p50: 14.419078826904297 Avg: 14.41427937695006 StdDev: 0.09572181406460434 ``` ``` 1.5.1 Install: pip install mxnet-mkl==1.5.1 Output: Model: resnext-50 ----Latency---- Model Partition Time: 0 Model Load Time: 68.64523887634277 First Inference: 93.96791458129883 p99: 16.599690914154053 p90: 16.532421112060547 p50: 16.48259162902832 Avg: 16.486327974449974 StdDev: 0.09714292044513262 ``` ``` 1.6.0 Install: pip install mxnet-mkl==1.6.0b20191025 Output: Model: resnext-50 ----Latency---- Model Partition Time: 0 Model Load Time: 74.61214065551758 First Inference: 100.08096694946289 p99: 16.619694232940674 p90: 16.5402889251709 p50: 16.489028930664062 Avg: 16.493797707719867 StdDev: 0.09944463033333369 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
