On 6 May 2015 at 11:26, Christophe Milard <[email protected]> wrote: > > Hi, > > Yet another try to reach a acceptable solution regarding the test > environment, following tuesday meeting. > Here comes the n+2th proposal...:-) > The goal is now to improve this proposal with your comments, and then finally > pick up the best alternative between the proposal from last week (running the > tests from the validation side, assumed to be optimal as no further comments > where recieved :-) ) and this new one running the tests from the platform > side. > > M is number of modules. > <Module> is the module name. > > test/validation would contain: > > -a Makefile.am which would > > -recursively build in all M test/validation/<Module> directories, > > -link all the M <Module>/lib<Module>.a (see below) together in a superlib > > -possibly run all the M <Module>/<Module> executables (see below) when target > "check-agnostic" is made. (i.e. on 'make check-agnostic'; nothing would be > ran on 'make check' here.) > > > -M directories called <Module>: each of these directory /validation/<Module> > would contain: > -a set of .c files defining: > > * tests functions, called <Module>_<testname>_test(): These functions are not > exported by default (i.e. they are static), but this could be changed in the > future, if we'd want to give the freedom for the platform to play with > individual tests. > > * test arrays (CU_TestInfo), called <Module>_<test-array-name>_tests: These > symbols are not exported by default (i.e. they are static), but this could be > changed in the future, if we'd want to give the freedom for the platform to > play with individual arrays of tests. > > * test suites (CU_SuiteInfo), called <Module>_<test-suites-name>_suites: > These symbols are exported, giving the possibility to the platform to handle > the suites as it wishes (but they would remain "atomic" as long as no other > symbol is exported). This would be used, for instance, for testing init: > Three suites would be defined and called by a single main on linux-generic
This will break KS2, the reason they are separate mains is becasue any call to global_init must be its own executable for some platforms. > (where many odp-init() and odp-term() in a raw does not hurt), while > platforms which needs it could create 3 different mains from these 3 symbols. Why would we make other platform that currently reuse these tests write their own tests ? > -a set of .h files defining the above exported symbols > -a <Module>.c file (using validation/common/...) defining a main > running all suites belonging to the module (roughly as today) > -a Makefile.am which would build (both for "make check-agnostic" and > "make check"): > * lib<Module>.a by linking together all the C files, exept > <Module>.c > * <Module> by linking together <Module>.c and lib<Module>.a > > > platform/<platform>/test would contain: > > -a Makefile.am which would > > -recursively build in all existing (0 to M) platform/<platform>/test/<Module> > directories (see below), > > -run a list of M executables when "make check" is made. These executable > could be one of: > > */test/validation/<Module>/<Module> if there no platform specific for this > module. > > This means that platform/<platform>/test/Makefile directely points the > executable in the test/validation/<Module> part, thus avoiding the need for a > specific directory and Makefile on the platform side when there is nothing > platform specific with a module. > > *something in platform/<platform>/test/<Module> which could be: > > .a program (script,C), setting up the platform environment, and calling > /test/validation/<Module>/<Module> > > .a program (script? and C) using the test suites exported by > test/validation/<Module>/lib<Module>.a (or the superlib) > > -a list of optional platform/<platform>/test/<Module> directories containing: > > -scripts and or specific programs for this module (possibly linked with > test/validation/<Module>/lib<Module>.a (or the superlib)) > > -a Makefile.am building whatever is needed for the given module. > > As an example, for init on platform that do not support multiple odp-init and > odp-term in a raw one would build: > > 3 mains, using respectively init_ok_suites, init_abort_suites, and > init_log_suites. These 3 mains could be called from a script, init.sh, which > would be the executable given in platform/<platform>/test/ Makefile.am > > > the +: > - no need for any directory nor Makefile on the platform side when there is > nothing platform specific. > - ability to still run platform agnostic tests from the validation side by > making "make check-agnostic" > (may not be very useful, though, when odp-init() needs platform options...) > - ability to restrict the freedom given to the platform by restricting the > symbols exported in lib<Module>.a (and superlib) > - platforms can use whatever the platform offers to setup the tests. > - ... please comment! > > the -: > - a module is restricted to 1 executable at least on the validation side. (on > platforms supporting it, > a script or independent program could be used to gather the different > executable as one, see init above) > - slighly more complicated structure: the mains present under > /test/validation/<Module>/<Module> > may discrectely be overloaded by some main on the platform side. > - the more freedom is given to the platform, the less comparable the test > result get...but this is now controled > by the restriction on the symbols exported in lib<Module>.a (and superlib) > - ... please comment! > > common to all alternatives is the problem of the test reports. what is > reported by make-check vs Cunit. > maybe a later question... > > Christophe. Patches I see 1. split crypto executable into async, syncrononous and common executables 2. Remove common cunit main, replace it with a script that generates a main.c for each test 3. rename the suites with unique sensible names 4. make a library of all the common test code per module -- Mike Holmes Technical Manager - Linaro Networking Group Linaro.org │ Open source software for ARM SoCs _______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
