anijain2305 commented on a change in pull request #5848:
URL: https://github.com/apache/incubator-tvm/pull/5848#discussion_r445326823
##########
File path: tests/python/frontend/tflite/test_forward.py
##########
@@ -2445,6 +2467,112 @@ def test_forward_qnn_mobilenet_v3_net():
tvm.testing.assert_allclose(tvm_sorted_labels, tflite_sorted_labels)
+
+def _quantize_tf_hub_keras_model(url, height, width):
+ keras_model = tf.keras.Sequential([hub.KerasLayer(url,
output_shape=[1001])])
+ data = pre_processed_image(height, width)
+
+ # Set the input shapes of the keras model
+ keras_model._set_inputs(data)
+
+ # Get the converter
+ converter =
interpreter_wrapper.TFLiteConverter.from_keras_model(keras_model)
+
+ # To create quantized values with dynamic range of activations, needs
representative dataset
+ def representative_data_gen():
+ for i in range(1):
+ yield [data]
+
+ converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
+ converter.representative_dataset = representative_data_gen
+ return converter.convert()
+
+
+def test_forward_tflite2_qnn_resnet50():
+ """Test the Quantized TFLite version 2.1.0 Resnet50 model."""
+ if package_version.parse(tf.VERSION) >= package_version.parse('2.1.0'):
+ # Quantize the model
+ url = "https://tfhub.dev/tensorflow/resnet_50/classification/1";
+ tflite_model_buf = _quantize_tf_hub_keras_model(url, 224, 224)
+ data = pre_processed_image(224, 224)
+
+ tflite_output = run_tflite_graph(tflite_model_buf, data)
+ tflite_predictions = np.squeeze(tflite_output)
+ tflite_sorted_labels = tflite_predictions.argsort()[-3:][::-1]
+ tvm_output = run_tvm_graph(tflite_model_buf, np.array(data), 'input_1')
+ tvm_predictions = np.squeeze(tvm_output)
+ tvm_sorted_labels = tvm_predictions.argsort()[-3:][::-1]
+ tvm.testing.assert_allclose(tvm_sorted_labels, tflite_sorted_labels)
Review comment:
Yes, rounding differences lead to these differences (although minimal
effect on Top1/5 accuracy). I tried softmax'd output for older TFLite 1.13
tests and labels looked better for testing. I will check the TFLite 2.1.0 tests
(I dont think picture would change)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
