zhhoper commented on issue #17231: cannot quantization example
URL: 
https://github.com/apache/incubator-mxnet/issues/17231#issuecomment-574543735
 
 
   @wuxun-zhang @ZhennanQin I run the example using mxnet 1.6.0, it seems to 
work ok. However, the run time of quantized model is much slower (more than 10 
times) than the original one. Is there anything I need to set up in order to 
speed up the quantized model?
   I test resnet152 
   For float32:
   command:
   python imagenet_inference.py 
--symbol-file=./model/imagenet1k-resnet-152-symbol.json 
--param-file=./model/imagenet1k-resnet-152-0000.params --num-skipped-batches=50 
--batch-size=64 --num-inference-batches=500 --dataset=./data/val_256_q90.rec 
--ctx=cpu
   Output:
   INFO:logger:batch size = 64 for inference
   INFO:logger:rgb_mean = 0,0,0
   INFO:logger:rgb_std = 1,1,1
   INFO:logger:label_name = softmax_label
   INFO:logger:Input data shape = (3, 224, 224)
   INFO:logger:Dataset for inference: ./data/val_256_q90.rec
   [07:03:16] ../src/io/iter_image_recordio_2.cc:831: Create ImageRecordIter2 
optimized for CPU backend.Use omp threads instead of preprocess_threads.
   [07:03:16] ../src/io/iter_image_recordio_2.cc:178: ImageRecordIOParser2: 
./data/val_256_q90.rec, use 16 threads for decoding..
   [07:03:16] ../src/base.cc:84: Upgrade advisory: this mxnet has been built 
against cuDNN lib version 7401, which is older than the oldest version tested 
by CI (7600).  Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
   INFO:logger:Loading symbol from file 
/home/ubuntu/software/incubator-mxnet/example/quantization/./model/imagenet1k-resnet-152-symbol.json
   [07:03:18] ../src/nnvm/legacy_json_util.cc:209: Loading symbol saved by 
previous version v0.8.0. Attempting to upgrade...
   [07:03:18] ../src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
   INFO:logger:Loading params from file 
/home/ubuntu/software/incubator-mxnet/example/quantization/./model/imagenet1k-resnet-152-0000.params
   INFO:logger:Skipping the first 50 batches
   INFO:logger:Running model ./model/imagenet1k-resnet-152-symbol.json for 
inference
   [07:03:19] ../src/executor/graph_executor.cc:1982: Subgraph backend MKLDNN 
is activated.
   INFO:logger:Finished inference with 32000 images
   **INFO:logger:Finished with 22.124158 images per second**
   WARNING:logger:Note: GPU performance is expected to be slower than CPU. 
Please refer quantization/README.md for details
   INFO:logger:('accuracy', 0.7676875)
   INFO:logger:('top_k_accuracy_5', 0.93034375)
   
   For quantized model
   command:
   python imagenet_inference.py 
--symbol-file=./model/imagenet1k-resnet-152-quantized-5batches-naive-symbol.json
 --param-file=./model/imagenet1k-resnet-152-quantized-0000.params 
--num-skipped-batches=50 --batch-size=64 --num-inference-batches=500 
--dataset=./data/val_256_q90.rec --ctx=cpu
   output:
   INFO:logger:batch size = 64 for inference
   INFO:logger:rgb_mean = 0,0,0
   INFO:logger:rgb_std = 1,1,1
   INFO:logger:label_name = softmax_label
   INFO:logger:Input data shape = (3, 224, 224)
   INFO:logger:Dataset for inference: ./data/val_256_q90.rec
   [00:37:40] ../src/io/iter_image_recordio_2.cc:831: Create ImageRecordIter2 
optimized for CPU backend.Use omp threads instead of preprocess_threads.
   [00:37:40] ../src/io/iter_image_recordio_2.cc:178: ImageRecordIOParser2: 
./data/val_256_q90.rec, use 16 threads for decoding..
   [00:37:40] ../src/base.cc:84: Upgrade advisory: this mxnet has been built 
against cuDNN lib version 7401, which is older than the oldest version tested 
by CI (7600). Set MXNET_CUDNN_LIB_CHECKING=0 to quiet this warning.
   INFO:logger:Loading symbol from file 
/home/ubuntu/software/incubator-mxnet/example/quantization/./model/imagenet1k-resnet-152-quantized-5batches-naive-symbol.json
   INFO:logger:Loading params from file 
/home/ubuntu/software/incubator-mxnet/example/quantization/./model/imagenet1k-resnet-152-quantized-0000.params
   INFO:logger:Skipping the first 50 batches
   INFO:logger:Running model 
./model/imagenet1k-resnet-152-quantized-5batches-naive-symbol.json for inference
   [00:37:43] ../src/executor/graph_executor.cc:1982: Subgraph backend MKLDNN 
is activated.
   INFO:logger:Finished inference with 32000 images
   **INFO:logger:Finished with 1.495486 images per second**
   WARNING:logger:Note: GPU performance is expected to be slower than CPU. 
Please refer quantization/README.md for details
   INFO:logger:('accuracy', 0.76328125)
   INFO:logger:('top_k_accuracy_5', 0.92859375)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to