Lunderberg commented on a change in pull request #7: URL: https://github.com/apache/tvm-rfcs/pull/7#discussion_r698489224
########## File path: rfcs/0007-parametrized-unit-tests.md ########## @@ -0,0 +1,568 @@ +- Feature Name: Parametrized Unit Tests +- Start Date: 2021-05-10(fill me in with today's date, YYYY-MM-DD) +- RFC PR: [apache/tvm-rfcs#0007](https://github.com/apache/tvm-rfcs/pull/0007) +- GitHub PR: [apache/tvm#8010](https://github.com/apache/tvm/issues/8010) + +# Summary +[summary]: #summary + +This RFC documents how to implement unit tests that depend on input +parameters, or have setup that depends on input parameters. + +# Motivation +[motivation]: #motivation + +Some unit tests should be tested along a variety of parameters for +better coverage. For example, a unit test that does not depend on +target-specific features should be tested on all targets that the test +platform supports. Alternatively, a unit test may need to pass +different array sizes to a function, in order to exercise different +code paths within that function. + +The simplest implementation would be to write a test function that +loops over all parameters, throwing an exception if any parameter +fails the test. However, this does not give full information to a +developer, as a failure from any parameter results in the entire test +to be marked as failing. A unit-test that fails for all targets +requires different debugging than a unit-test that fails on a single +specific target, and so this information should be exposed. + +This RFC adds functionality for implementing parameterized unit tests, +such that each set of parameters appears as a separate test result in +the final output. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +## Parameters + +To make a new parameter for unit tests to use, define it with the +`tvm.testing.parameter` function. For example, the following will +define a parameter named `array_size` that has three possible values. +This can appear either at global scope inside a test module to be +usable by all test functions in that module, or in a directory's +`conftest.py` to be usable by all tests in that directory. + +```python +array_size = tvm.testing.parameter(8, 256, 1024) +``` + +To use a parameter, define a test function that accepts the parameter +as an input. This test will be run once for each value of the Review comment: Updated, but with a comma instead of a parenthetical. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
