Hi,
Yet another try to reach a acceptable solution regarding the test
environment, following tuesday meeting.
Here comes the n+2th proposal...:-)
The goal is now to improve this proposal with your comments, and then
finally pick up the best alternative between the proposal from last week
(running the tests from the validation side, assumed to be optimal as no
further comments where recieved :-) ) and this new one running the tests
from the platform side.
M is number of modules.
<Module> is the module name.
test/validation would contain:
-a Makefile.am which would
-recursively build in all M test/validation/<Module> directories,
-link all the M <Module>/lib<Module>.a (see below) together in a superlib
-possibly run all the M <Module>/<Module> executables (see below) when
target "check-agnostic" is made. (i.e. on 'make check-agnostic'; nothing
would be ran on 'make check' here.)
-M directories called <Module>: each of these directory
/validation/<Module> would contain:
-a set of .c files defining:
* tests functions, called <Module>_<testname>_test(): These functions are
not exported by default (i.e. they are static), but this could be changed
in the future, if we'd want to give the freedom for the platform to play
with individual tests.
* test arrays (CU_TestInfo), called <Module>_<test-array-name>_tests: These
symbols are not exported by default (i.e. they are static), but this could
be changed in the future, if we'd want to give the freedom for the platform
to play with individual arrays of tests.
* test suites (CU_SuiteInfo), called <Module>_<test-suites-name>_suites:
These symbols are exported, giving the possibility to the platform to
handle the suites as it wishes (but they would remain "atomic" as long as
no other symbol is exported). This would be used, for instance, for testing
init: Three suites would be defined and called by a single main on
linux-generic (where many odp-init() and odp-term() in a raw does not
hurt), while platforms which needs it could create 3 different mains from
these 3 symbols.
-a set of .h files defining the above exported symbols
-a <Module>.c file (using validation/common/...) defining a main
running all suites belonging to the module (roughly as today)
-a Makefile.am which would build (both for "make check-agnostic"
and "make check"):
* lib<Module>.a by linking together all the C files, exept
<Module>.c
* <Module> by linking together <Module>.c and lib<Module>.a
platform/<platform>/test would contain:
-a Makefile.am which would
-recursively build in all existing (0 to M)
platform/<platform>/test/<Module> directories (see below),
-run a list of M executables when "make check" is made. These executable
could be one of:
*/test/validation/<Module>/<Module> if there no platform specific for this
module.
This means that platform/<platform>/test/Makefile directely points the
executable in the test/validation/<Module> part, thus avoiding the need for
a specific directory and Makefile on the platform side when there is
nothing platform specific with a module.
*something in platform/<platform>/test/<Module> which could be:
.a program (script,C), setting up the platform environment, and calling
/test/validation/<Module>/<Module>
.a program (script? and C) using the test suites exported by
test/validation/<Module>/lib<Module>.a (or the superlib)
-a list of optional platform/<platform>/test/<Module> directories
containing:
-scripts and or specific programs for this module (possibly linked with
test/validation/<Module>/lib<Module>.a (or the superlib))
-a Makefile.am building whatever is needed for the given module.
As an example, for init on platform that do not support multiple odp-init
and odp-term in a raw one would build:
3 mains, using respectively init_ok_suites, init_abort_suites, and
init_log_suites. These 3 mains could be called from a script, init.sh,
which would be the executable given in platform/<platform>/test/ Makefile.am
the +:
- no need for any directory nor Makefile on the platform side when there is
nothing platform specific.
- ability to still run platform agnostic tests from the validation side by
making "make check-agnostic"
(may not be very useful, though, when odp-init() needs platform
options...)
- ability to restrict the freedom given to the platform by restricting the
symbols exported in lib<Module>.a (and superlib)
- platforms can use whatever the platform offers to setup the tests.
- ... please comment!
the -:
- a module is restricted to 1 executable at least on the validation side.
(on platforms supporting it,
a script or independent program could be used to gather the different
executable as one, see init above)
- slighly more complicated structure: the mains present under
/test/validation/<Module>/<Module>
may discrectely be overloaded by some main on the platform side.
- the more freedom is given to the platform, the less comparable the test
result get...but this is now controled
by the restriction on the symbols exported in lib<Module>.a (and superlib)
- ... please comment!
common to all alternatives is the problem of the test reports. what is
reported by make-check vs Cunit.
maybe a later question...
Christophe.
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
