On 6 May 2015 at 22:48, Mike Holmes <[email protected]> wrote: > On 6 May 2015 at 11:26, Christophe Milard <[email protected]> > wrote: > > > > Hi, > > > > Yet another try to reach a acceptable solution regarding the test > environment, following tuesday meeting. > > Here comes the n+2th proposal...:-) > > The goal is now to improve this proposal with your comments, and then > finally pick up the best alternative between the proposal from last week > (running the tests from the validation side, assumed to be optimal as no > further comments where recieved :-) ) and this new one running the tests > from the platform side. > > > > M is number of modules. > > <Module> is the module name. > > > > test/validation would contain: > > > > -a Makefile.am which would > > > > -recursively build in all M test/validation/<Module> directories, > > > > -link all the M <Module>/lib<Module>.a (see below) together in a superlib > > > > -possibly run all the M <Module>/<Module> executables (see below) when > target "check-agnostic" is made. (i.e. on 'make check-agnostic'; nothing > would be ran on 'make check' here.) > > > > > > -M directories called <Module>: each of these directory > /validation/<Module> would contain: > > -a set of .c files defining: > > > > * tests functions, called <Module>_<testname>_test(): These functions > are not exported by default (i.e. they are static), but this could be > changed in the future, if we'd want to give the freedom for the platform to > play with individual tests. > > > > * test arrays (CU_TestInfo), called <Module>_<test-array-name>_tests: > These symbols are not exported by default (i.e. they are static), but this > could be changed in the future, if we'd want to give the freedom for the > platform to play with individual arrays of tests. > > > > * test suites (CU_SuiteInfo), called <Module>_<test-suites-name>_suites: > These symbols are exported, giving the possibility to the platform to > handle the suites as it wishes (but they would remain "atomic" as long as > no other symbol is exported). This would be used, for instance, for testing > init: Three suites would be defined and called by a single main on > linux-generic > > This will break KS2, the reason they are separate mains is becasue any > call to global_init must be its own executable for some platforms. > > > (where many odp-init() and odp-term() in a raw does not hurt), while > platforms which needs it could create 3 different mains from these 3 > symbols. > > Why would we make other platform that currently reuse these tests > write their own tests ? >
OK. I thought the idea was to get one executable per module when possible. I do not see any problem allowing many, and keeping the 3 executable as of today for init even in validation. > > > -a set of .h files defining the above exported symbols > > -a <Module>.c file (using validation/common/...) defining a main > running all suites belonging to the module (roughly as today) > > -a Makefile.am which would build (both for "make check-agnostic" > and "make check"): > > * lib<Module>.a by linking together all the C files, > exept <Module>.c > > * <Module> by linking together <Module>.c and > lib<Module>.a > > > > > > platform/<platform>/test would contain: > > > > -a Makefile.am which would > > > > -recursively build in all existing (0 to M) > platform/<platform>/test/<Module> directories (see below), > > > > -run a list of M executables when "make check" is made. These executable > could be one of: > > > > */test/validation/<Module>/<Module> if there no platform specific for > this module. > > > > This means that platform/<platform>/test/Makefile directely points the > executable in the test/validation/<Module> part, thus avoiding the need for > a specific directory and Makefile on the platform side when there is > nothing platform specific with a module. > > > > *something in platform/<platform>/test/<Module> which could be: > > > > .a program (script,C), setting up the platform environment, and calling > /test/validation/<Module>/<Module> > > > > .a program (script? and C) using the test suites exported by > test/validation/<Module>/lib<Module>.a (or the superlib) > > > > -a list of optional platform/<platform>/test/<Module> directories > containing: > > > > -scripts and or specific programs for this module (possibly linked with > test/validation/<Module>/lib<Module>.a (or the superlib)) > > > > -a Makefile.am building whatever is needed for the given module. > > > > As an example, for init on platform that do not support multiple > odp-init and odp-term in a raw one would build: > > > > 3 mains, using respectively init_ok_suites, init_abort_suites, and > init_log_suites. These 3 mains could be called from a script, init.sh, > which would be the executable given in platform/<platform>/test/ Makefile.am > > > > > > the +: > > - no need for any directory nor Makefile on the platform side when there > is nothing platform specific. > > - ability to still run platform agnostic tests from the validation side > by making "make check-agnostic" > > (may not be very useful, though, when odp-init() needs platform > options...) > > - ability to restrict the freedom given to the platform by restricting > the symbols exported in lib<Module>.a (and superlib) > > - platforms can use whatever the platform offers to setup the tests. > > - ... please comment! > > > > the -: > > - a module is restricted to 1 executable at least on the validation > side. (on platforms supporting it, > > a script or independent program could be used to gather the different > executable as one, see init above) > > - slighly more complicated structure: the mains present under > /test/validation/<Module>/<Module> > > may discrectely be overloaded by some main on the platform side. > > - the more freedom is given to the platform, the less comparable the > test result get...but this is now controled > > by the restriction on the symbols exported in lib<Module>.a (and > superlib) > > - ... please comment! > > > > common to all alternatives is the problem of the test reports. what is > reported by make-check vs Cunit. > > maybe a later question... > > > > Christophe. > > Patches I see > 1. split crypto executable into async, syncrononous and common executables > ok > 2. Remove common cunit main, replace it with a script that generates a > main.c for each test. > ok:I will create a main for each executable (I don't think I want to commit a script to do that, but I will surely use scripts when doing the job). > 3. rename the suites with unique sensible names > ok: That would prepare for exporting the suites in a lib. > 4. make a library of all the common test code per module > I am not sure everyone want to give this freedom to platforms.... I'll call for a quick meeting at 14:00. hope we can come to an agreement :-) /Christophe. > > -- > Mike Holmes > Technical Manager - Linaro Networking Group > Linaro.org │ Open source software for ARM SoCs >
_______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
