IvyGongoogle commented on issue #14159: [Feature Request] Support fp16 for c++ api URL: https://github.com/apache/incubator-mxnet/issues/14159#issuecomment-485865319 > @IvyGongoogle Are your inputs and weights in fp16. Your change should work to run fp16 inference. What batch size are you using and what is the model ? For smaller batch sizes you may not see a big speedup. Also what hardware are you running it on ? yes, my inputs and weights is in fp16 and batch_size is 16/32, but after my change above, the inference using `src/c_api/c_predict_api.cc` can run, but the speed using fp16 still is same with fp32, or slower. However, as you know, when we run `example/image-classification/benchmark_score.py`, using fp16 is twice faster than fp32, so I doubt that why python and C++ inference api using fp16 have so big difference?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
