pengzhao-intel commented on issue #14173: [WIP] MXNet AMP (automatic mixed precision) URL: https://github.com/apache/incubator-mxnet/pull/14173#issuecomment-476056958 Thanks @ptrendx @eric-haibin-lin this is no FP16 GEMM from Intel MKL/MKL-DNN till now so I think we have to fall back to FP32 GEMM or simulated FP16 GEMM (maybe from `mshadow` or other BLAS library). The current design is good for APEX. I am thinking about a general solution for mixed precision training, such as INT8, FP16, BF16, FP32. The different HW supports for the different data type, like INT8 in CPU and small GPU chip and BF16 on CPU and FP16 on GPU. So we need more flexibility for data type change or other library integration. We are going to support BF16 and the ideal situation is the user can switch FP16 and BF16 based on the devices transparently. We are looking into the changes and will give back more details later.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
