gromero commented on PR #11108:
URL: https://github.com/apache/tvm/pull/11108#issuecomment-1110471946
> @gromero @leandron I have no problem. Could you provide an example of test
case for other tvmc frontend, I will update in this PR
@jiangjiajun So, since I understand we need to test the code path between
tvmc' s `main()` (in `main.py`) and `compile_model()` (in `compiler.py`), which
includes exercising any logic/check to validate the `args` attributes passed to
`fronends.load_model()` plus the load method itself (i.e. it's a command line
test), I suggest something like:
```
diff --git a/tests/python/driver/tvmc/test_command_line.py
b/tests/python/driver/tvmc/test_command_line.py
index 5b15492aa..1d9180fed 100644
--- a/tests/python/driver/tvmc/test_command_line.py
+++ b/tests/python/driver/tvmc/test_command_line.py
@@ -20,9 +20,11 @@ import pytest
import shutil
from pytest_lazyfixture import lazy_fixture
+from unittest import mock
+
from tvm.driver.tvmc.main import _main
from tvm.driver.tvmc.model import TVMCException
-
+from tvm.driver.tvmc import compiler
@pytest.mark.skipif(
platform.machine() == "aarch64",
@@ -155,3 +157,25 @@ def test_tvmc_tune_file_check(capsys, invalid_input):
)
on_assert_error = f"'tvmc tune' failed to check invalid FILE:
{invalid_input}"
assert captured.err == expected_err, on_assert_error
+
[email protected]
+def paddle_model(paddle_resnet50):
+ # If we can't import "paddle" module, skip testing paddle
+ if pytest.importorskip("paddle", reason="'paddle' module not
installed"):
+ return paddle_resnet50
+
[email protected]("model", [lazy_fixture("paddle_model"), ])
+# compile_model() can take too long and is tested elsewhere, hence it's
mocked below
[email protected](compiler, "compile_model")
+def test_tvmc_compile_input_model(mock_compile_model, tmpdir_factory,
model):
+
+ output_dir = tmpdir_factory.mktemp("output")
+ output_file = output_dir / "model.tar"
+
+ compile_cmd = f"tvmc compile --target 'llvm' {model} --model-format
paddle --output {output_file}"
+ run_arg = compile_cmd.split(" ")[1:]
+
+ _main(run_arg)
+
+ mock_compile_model.assert_called_once()
```
This way we could also easily extend `test_tvmc_compile_input_model` to
test other input formats. For instance, a `tflite` model, by adding:
```
diff --git a/tests/python/driver/tvmc/test_command_line.py
b/tests/python/driver/tvmc/test_command_line.py
index 505faea65..8e4a116b4 100644
--- a/tests/python/driver/tvmc/test_command_line.py
+++ b/tests/python/driver/tvmc/test_command_line.py
@@ -164,7 +164,14 @@ def paddle_model(paddle_resnet50):
if pytest.importorskip("paddle", reason="'paddle' module not
installed"):
return paddle_resnet50
[email protected]("model", [lazy_fixture("paddle_model"), ])
+
[email protected]
+def tflite_model(tflite_mobilenet_v1_1_quant):
+ if pytest.importorskip("tflite", reason="'tflite' module not
installed"):
+ return tflite_mobilenet_v1_1_quant
+
+
[email protected]("model", [lazy_fixture("paddle_model"),
lazy_fixture("tflite_model"), ])
# compile_model() can take too long and is tested elsewhere, hence it's
mocked below
@mock.patch.object(compiler, "compile_model")
# @mock.patch.object(compiler, "compile_model")
```
I've also used mock() for `compile_model()` because it takes too long and
it's also tested elsewhere.
Also `compile_cmd` in the test won't have `--model-format paddle` after the
fix in the sense we've discussed, so it would work for other formats too.
Note I've mentioned the `tflite` format just as an example on how to extend
the test in the future, it's out of scope for the PR.
You probably know it already, but just in case, you can use pytest `-k`
flags to select only that test to run, for example:
```
$ pytest -k test_tvmc_compile_input_model -vvvv ./test_command_line.py
[02:37:03] /home/gromero/git/tvm/src/target/target_kind.cc:163: Warning:
Unable to detect CUDA version, default to "-arch=sm_20" instead
enabled targets: llvm
pytest marker:
====================================================================================
test session starts
=====================================================================================
platform linux -- Python 3.9.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 --
/usr/bin/python3
cachedir: .pytest_cache
rootdir: /home/gromero/git/tvm/tests/python/driver/tvmc
plugins: lazy-fixture-0.6.3
collected 10 items / 8 deselected / 2 selected
test_command_line.py::test_tvmc_compile_input_model[paddle_model] PASSED
[ 50%]
test_command_line.py::test_tvmc_compile_input_model[tflite_model] SKIPPED
('tflite' module not installed)
[100%]
========================================================================= 1
passed, 1 skipped, 8 deselected in 1.67s
=========================================================================
```
Feel free to adapt it or use your own test version.
HTH.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
