ptrendx commented on issue #7996: Question about Float16 URL: https://github.com/apache/incubator-mxnet/issues/7996#issuecomment-331678105 If you choose fp16 dtype then training values storage is fp16, compute accuracy is fp32/TensorCore on Volta. By default, there is no fp32 master weight, but for SGD optimizer you can set multi_precision=True which will result in fp32 master weight. I have a PR #7654 that will make mixed precision work with all optimizer (although slower than a dedicated path like in sgd). If you use train_imagenet.py script with fp16 then it sets multi_precision option to True. Before there was MSHADOW_USE_PASCAL compile flag that would enable fp16 compute (what caffe calls true fp16 training) but we decided that (especially with Volta) it is not worth the confusion it makes. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
With regards, Apache Git Services
