On Wed, Sep 16, 2015 at 05:19:28PM +0200, Christophe Milard wrote: > On 2015-09-16 10:42, Stuart Haslam wrote: > > Update sections describing how a specific platform may skip tests by > > marking them as inactive. > > > > Signed-off-by: Stuart Haslam <[email protected]> > > --- > > doc/implementers-guide/implementers-guide.adoc | 58 > > +++++++++++++++++++++----- > > 1 file changed, 48 insertions(+), 10 deletions(-) > > > > diff --git a/doc/implementers-guide/implementers-guide.adoc > > b/doc/implementers-guide/implementers-guide.adoc > > index 090d4e5..181d5c2 100644 > > --- a/doc/implementers-guide/implementers-guide.adoc > > +++ b/doc/implementers-guide/implementers-guide.adoc > > @@ -110,7 +110,7 @@ Module test and naming convention > > *<Module>_test_+++*+++* + > > where the suffix idendify the test. > > > > -* Test arrays, i.e. arrays of CU_TestInfo, listing the test functions > > belonging to a suite, are called: > > +* Test arrays, i.e. arrays of odp_testinfo_t, listing the test functions > > belonging to a suite, are called: > > *<Module>_suite+++[_*]+++* + > > where the possible suffix can be used if many suites are declared. > > > > @@ -118,7 +118,7 @@ Module test and naming convention > > *<Module>+++_suite[_*]_init()+++* and *<Module>+++_suite[_*]_term()+++* > > respectively. + > > where the possible extra middle pattern can be used if many suites are > > declared. > > > > -* Suite arrays, i.e. arrays of CU_SuiteInfo used in executables (C_UNIT > > registry) are called: > > +* Suite arrays, i.e. arrays of odp_suiteinfo_t used in executables (C_UNIT > > registry) are called: > > *<Module>+++_suites[_*]+++* + > > where the possible suffix identifies the executable using it, if many. > > > > @@ -232,14 +232,52 @@ Defining platform specific tests > > Sometimes, it may be necessary to call platform specific system calls to > > check some functionality: For instance, testing odp_cpumask_* could involve > > checking the underlying system CPU mask. On linux, such a test would > > require using the CPU_ISSET macro, which is linux specific. Such a test > > would be written in '<PLATFORM_SPECIFIC>/cpumask/...' The contents of this > > directory would be very similar to the contents of the platform agnostic > > side cpu_mask tests (including a Makefile.am...), but platform specific > > test would be written there. > > '<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the > > platform specific tests (by listing their module name in SUBDIRS and > > therefore calling the appropriate Makefile.am) and then it would call both > > the platform agnostic executable(s) and the platform specific test > > executable. > > > > -Skipping tests during development > > -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > -During ODP development, it may be useful to skip some test. This can be > > achieved by creating a new test executable (still on the platform side), > > picking up the required tests from the platform agnostic libtest<module>.la. > > +Marking tests as inactive > > +^^^^^^^^^^^^^^^^^^^^^^^^^ > > +The general policy is that a full run of the validation suite (a "make > > check") must pass at all times. However a particular platform may have one > > or more test cases that are known to be unimplemented either during > > development or permanently, so to avoid these test cases being reported as > > failures it's useful to be able to skip them. This can be achieved by > > creating a new test executable (still on the platform side), giving the > > platform specific initialisation code the opportunity to modify the > > registered tests in order to mark unwanted tests as inactive while leaving > > the remaining tests active. It's important that the unwanted tests are > > still registered with the test framework to allow the fact that they're not > > being tested to be recorded. > > + > > +The odp_cunit_update() function is intended for this purpose, it is used > > to modify the properties of previously registered tests, for example to > > mark them as inactive. Inactive tests are registered with the test > > framework but aren't executed and will be recorded as inactive in test > > reports. > > + > > +In 'test/validation/foo/foo.c', define all tests for the 'foo' module; > > + > > +[source,c] > > +------------------ > > +odp_testinfo_t foo_tests[] = { > > + ODP_TEST_INFO(foo_test_a), > > + ODP_TEST_INFO(foo_test_b), > > + ODP_TEST_INFO_NULL > > +}; > > + > > +odp_suiteinfo_t foo_suites[] = { > > + {"Foo", foo_suite_init, foo_suite_term, foo_tests}, > > + ODP_SUITE_INFO_NULL > > +}; > > +------------------ > > + > > +In 'platform/<platform>/test/foo/foo_main.c', register all the tests > > defined in the 'foo' module, then mark a single specific test case as > > inactive; > > + > > +[source,c] > > +------------------ > > +static odp_testinfo_t foo_tests_updates[] = { > > + ODP_TEST_INFO_INACTIVE(foo_test_b), > > + ODP_TEST_INFO_NULL > > +}; > > + > > +static odp_suiteinfo_t foo_suites_updates[] = { > > + {"Foo", foo_suite_init, foo_suite_term, foo_tests_updates}, > > + ODP_SUITE_INFO_NULL > > +}; > > + > > +int pktio_main(void) > > should be foo_main > > > +{ > > + int ret = odp_cunit_register(foo_suites); > > > > -The top Makefile would then call only the platform specific executable, > > hence skipping the tests which have been omitted. > > + if (ret == 0) > > + ret = odp_cuint_update(foo_suites_updates); > > > > -TIP: You probably want to copy the platform-agnostic module main function > > and prune it from the undesired tests when writing your own platform > > specific main, for a given module. > > + if (ret == 0) > > + ret = odp_cunit_run(); > > > > -Permanently skipping test > > -^^^^^^^^^^^^^^^^^^^^^^^^^^ > > -If a platform wants to permanently skip a test (i.e. a part of the ODP API > > is and will not be supported on that platform), it is recommended to use > > the function odp_cunit_TBD() to removed the tests or suite from the list of > > tests. This gives a chance to the test environment to trace this removal. > > + return ret; > > +} > > +------------------ > > Maybe it should say that is is possible, during devloppement, to skip all > validation tests and have only platform specific if needed, and then beeing > able to add the common tests one by one.
OK I'll expand it. > I know we don't exactely have the same view here... but we are getting closer > :-) Actually this bit I agree with. Selectively running tests is fine, it's swapping them out that I'm not convinced about (but I'm happy enough to ignore it until it actually causes a problem). -- Stuart. _______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
