Will investigate On 18 November 2015 at 14:40, Bill Fischofer <[email protected]> wrote:
> This patch won't apply for me on any branch (master, next, or api-next). > Needs a rebase? > > Bill > > On Wed, Nov 18, 2015 at 12:09 PM, Mike Holmes <[email protected]> > wrote: > >> Signed-off-by: Mike Holmes <[email protected]> >> --- >> doc/implementers-guide/implementers-guide.adoc | 265 >> +++++++++++++++++-------- >> 1 file changed, 185 insertions(+), 80 deletions(-) >> >> diff --git a/doc/implementers-guide/implementers-guide.adoc >> b/doc/implementers-guide/implementers-guide.adoc >> index 0033ba3..8b9c729 100644 >> --- a/doc/implementers-guide/implementers-guide.adoc >> +++ b/doc/implementers-guide/implementers-guide.adoc >> @@ -11,9 +11,11 @@ Further details about ODP may be found at >> http://opendataplane.org[ODP homepage] >> >> >> :numbered: >> -The include structure >> ---------------------- >> -The implementers view of the include source tree allows the common API >> definitions and documentation to be reused by all the platforms defined in >> the tree, but leave the actual definitions to be defined by the specific >> platform. >> +== The include structure == >> + >> +The implementers view of the include source tree allows the common API >> +definitions and documentation to be reused by all the platforms defined >> in the >> +tree, but leave the actual definitions to be defined by the specific >> platform. >> >> .Implementers include structure >> ---- >> @@ -29,94 +31,145 @@ The implementers view of the include source tree >> allows the common API definitio >> │ ├── <implementation name>/ >> │ │ ├── include/ >> │ │ │ ├── odp/ >> -│ │ │ │ ├── In-line function definitions of the public API for >> this platform >> -│ │ │ │ │ seen by the applicationx. >> +│ │ │ │ ├── In-line function definitions of the public API for >> this >> +│ │ │ │ │ platform seen by the application. >> │ │ │ │ │ >> │ │ │ │ └── plat/ >> -│ │ │ │ └── Platform specific types, enums etc as seen by >> the application >> -│ │ │ │ but require overriding by the implementation. >> +│ │ │ │ └── Platform specific types, enums etc as seen by the >> +│ │ │ │ application but require overriding by the >> +│ │ │ │ implementation. >> │ │ │ │ >> │ │ │ └── Internal header files seen only by the implementation. >> ---- >> >> -The doxygen description of the API definition is held in the public api >> file 'include/odp/api'. >> -This file is included by a counterpart in 'platform/<implementation >> name>/include/odp'. >> -The include of the public API is AFTER the platform specific definitions >> to allow the platform to provide definitions that match the underlying >> hardware. >> -The implementation code includes 'platform/<implementation >> name>/include/plat' and this then provides the source files with a complete >> definition the ODP API to be implemented. >> -Applications in turn include the include/odp.h file which includes the >> 'platform/<implementation name>/include/plat' files to provide a complete >> definition of the API. >> +The doxygen description of the API definition is held in the public api >> file >> +'include/odp/api'. >> +This file is included by a counterpart in >> +'platform/<implementation name>/include/odp'. >> +The include of the public API is AFTER the platform specific definitions >> to >> +allow the platform to provide definitions that match the underlying >> hardware. >> +The implementation code includes 'platform/<implementation >> name>/include/plat' >> +and this then provides the source files with a complete definition the >> ODP API >> +to be implemented. >> +Applications in turn include the include/odp.h file which includes the >> +'platform/<implementation name>/include/plat' files to provide a complete >> +definition of the API. >> >> -The validation Suite >> --------------------- >> -ODP provides a comprehensive set of API validation tests that are >> intended to be used by implementers during development and by application >> developers to verify that a particular implementation meets their >> requirements. >> +== The validation Suite == >> + >> +ODP provides a comprehensive set of API validation tests that are >> intended to be >> +used by implementers during development and by application developers to >> verify >> +that a particular implementation meets their requirements. >> >> The list of these tests is expected to grow as ODP grows. >> >> -The list of test executables is run by the automake test harness, when >> running "make check". >> -Therefore, as required by this harness, each executable should return 0 >> on success (tests passed), 77 on inconclusive, or any other values on >> failure. >> -The automake functionality shows a status line (PASSED/FAIL...) for each >> of the ran test executables. >> +The list of test executables is run by the automake test harness, when >> running >> +"make check". >> +Therefore, as required by this harness, each executable should return 0 >> on >> +success (tests passed), 77 on inconclusive, or any other values on >> failure. >> +The automake functionality shows a status line (PASSED/FAIL...) for each >> of the >> +ran test executables. >> >> -It is expected that ODP developers will need to run tests as early as >> possible in the development cycle, before all APIs have been implemented. >> -Besides, although there are no APIs that are formally listed as >> optional, it is also expected that there may be cases where a subset of >> APIs remain unimplemented on a particular platform. >> -Moreover, some platforms may require specific initialization/termination >> code prior/after running the standard tests. >> +It is expected that ODP developers will need to run tests as early as >> possible >> +in the development cycle, before all APIs have been implemented. >> +Besides, although there are no APIs that are formally listed as >> optional, it is >> +also expected that there may be cases where a subset of APIs remain >> +unimplemented on a particular platform. >> +Moreover, some platforms may require specific initialization/termination >> code >> +prior/after running the standard tests. >> >> -To accommodate with these platform disparities, the ODP validation has >> been divided in two distinct areas: >> +To accommodate with these platform disparities, the ODP validation has >> been >> +divided in two distinct areas: >> >> * The platform agnostic area, >> * A platform dependent area (one per platform). >> >> -Platform agnostic >> -~~~~~~~~~~~~~~~~~ >> -This grouping defines tests that are expected to be executable and >> succeed on any platform, though possibly with very different performances, >> depending on the underlying platform. >> -They are written in plain C code, and may only use functions defined in >> the standard libC library (besides the ODP functions being tested, of >> course). >> -No other languages (like scripting) are allowed as their usage would >> make assumptions on the platform capability. >> +=== Platform agnostic === >> + >> +This grouping defines tests that are expected to be executable and >> succeed on >> +any platform, though possibly with very different performances, >> depending on the >> +underlying platform. >> +They are written in plain C code, and may only use functions defined in >> the >> +standard libC library (besides the ODP functions being tested, of >> course). >> +No other languages (like scripting) are allowed as their usage would make >> +assumptions on the platform capability. >> >> This area is located at: 'test/validation/' >> >> -The ODP API itself is ordered by module, where each module groups the >> set of ODP API functions related to the same "topic". >> -Examples of modules includes "classification" (API functions dealing >> with ingres packets classification), time (functions dealing with time, >> excluding timers which have their own module), timer,... >> -The complete module list can be seen at: >> http://docs.opendataplane.org/linux-generic-doxygen-html/modules.html[ODP >> Modules] + >> -Within the platform agnostic area, the tests are also grouped by >> modules, matching the ODP API modules: 'test/validation/' mainly contains a >> list of directories matching each module name (as defined by the doxygen >> "@defgroup" or "@ingroup" statement present in each API ".h" file). >> +The ODP API itself is ordered by module, where each module groups the >> set of ODP >> +API functions related to the same "topic". >> +Examples of modules includes "classification" (API functions dealing >> with ingres >> +packets classification), time (functions dealing with time, excluding >> timers >> +which have their own module), timer,... >> +The complete module list can be seen at: >> + >> http://docs.opendataplane.org/linux-generic-doxygen-html/modules.html[ODP >> Modules] + >> +Within the platform agnostic area, the tests are also grouped by modules, >> +matching the ODP API modules: 'test/validation/' mainly contains a list >> of >> +directories matching each module name (as defined by the doxygen >> "@defgroup" or >> +"@ingroup" statement present in each API ".h" file). >> >> -Within each of these directories, a library (called >> "libtest<module>.la") and its associated ".h" file (called "<module>.h") >> defines all the test functions for this module as well as few other >> functions to initialize, terminate, and group the tests. >> -An executable called "<module>_main*", is also built. It is permissible >> to generate more than one executable to cover the functionality in the test >> library for the module. >> +Within each of these directories, a library (called >> "libtest<module>.la") and >> +its associated ".h" file (called "<module>.h") defines all the test >> functions >> +for this module as well as few other functions to initialize, terminate, >> and >> +group the tests. >> +An executable called "<module>_main*", is also built. It is permissible >> to >> +generate more than one executable to cover the functionality in the test >> library >> +for the module. >> These executable(s) shall call all the tests for this module. + >> See <<anchor-1, Module test and naming convention>> for more details. >> >> -It is important to be aware that the tests defined for a given module >> (defined in 'test/validation/<module>') are focused to test the ODP >> functions belonging to this module, but are not limited to use this >> module's ODP functions only: many modules needs some interaction with some >> other module to be tested. >> -The obvious illustration of this is for module "init" whose functions >> are required by all tests of all other modules (as ODP needs to be >> initialized to test anything else). + >> +It is important to be aware that the tests defined for a given module >> +(defined in 'test/validation/<module>') are focused to test the ODP >> functions >> +belonging to this module, but are not limited to use this module's ODP >> functions >> +only: many modules needs some interaction with some other module to be >> tested. >> +The obvious illustration of this is for module "init" whose functions are >> +required by all tests of all other modules (as ODP needs to be >> initialized to >> +test anything else). + >> >> -There is a "Makefile.am" located at the top of the platform agnostic >> area. Its role is limited to the construction of the different test >> libraries and the "<module>_main*" executables. No tests are run from this >> area when "make check" is performed. >> +There is a "Makefile.am" located at the top of the platform agnostic >> area. Its >> +role is limited to the construction of the different test libraries and >> the >> +"<module>_main*" executables. No tests are run from this area when "make >> check" >> +is performed. >> >> -CUnit >> -^^^^^^ >> -Within a given test executable CUnit is used to run the different tests. >> The usage of CUnit implies the following structure: >> +==== CUnit ==== >> + >> +Within a given test executable CUnit is used to run the different tests. >> The >> +usage of CUnit implies the following structure: >> >> * Tests are simple C functions. >> -* Tests are grouped in arrays called test suites. Each test suite can be >> associated with a suite initialization/termination function(s), called by >> CUnit before and after the whole suite is run. >> -* An array of test suites (and associated init/term functions) defines >> the test registry run by the test executable. >> +* Tests are grouped in arrays called test suites. Each test suite can be >> +associated with a suite initialization/termination function(s), called >> by CUnit >> +before and after the whole suite is run. >> +* An array of test suites (and associated init/term functions) defines >> the test >> +registry run by the test executable. >> >> -Moreover, two extra functions can be used to initialize/terminate the >> test executable (these are not part of CUnit). + >> +Moreover, two extra functions can be used to initialize/terminate the >> test >> +executable (these are not part of CUnit). + >> A test executable return success (0) if every test of each suite succeed. >> >> -More details about http://cunit.sourceforge.net/doc/index.html[CUnit >> users guide] >> +More details about >> +http://cunit.sourceforge.net/doc/index.html[CUnit users guide] >> >> [[anchor-1]] >> -Module test and naming convention >> -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> +==== Module test and naming convention ==== >> + >> >> * Tests, i.e. C functions which are used in CUnit test suites are named: >> *<Module>_test_+++*+++* + >> where the suffix idendify the test. >> >> -* Test arrays, i.e. arrays of odp_testinfo_t, listing the test functions >> belonging to a suite, are called: >> +* Test arrays, i.e. arrays of odp_testinfo_t, listing the test functions >> + belonging to a suite, are called: >> *<Module>_suite+++[_*]+++* + >> where the possible suffix can be used if many suites are declared. >> >> * CUnit suite init and termination functions are called: >> *<Module>+++_suite[_*]_init()+++* and >> *<Module>+++_suite[_*]_term()+++* respectively. + >> - where the possible extra middle pattern can be used if many suites >> are declared. >> + where the possible extra middle pattern can be used if many suites are >> + declared. >> >> -* Suite arrays, i.e. arrays of odp_suiteinfo_t used in executables >> (CUnit registry) are called: >> +* Suite arrays, i.e. arrays of odp_suiteinfo_t used in executables >> + (CUnit registry) are called: >> *<Module>+++_suites[_*]+++* + >> where the possible suffix identifies the executable using it, if many. >> >> @@ -128,18 +181,28 @@ Module test and naming convention >> *<Module>_init* >> *<Module>_term* >> >> -All the above symbols are part of the generated libtest<Module>.la >> libraries. The generated main executable(s) (named <module>_+++main[_*]+++, >> where the optional suffix is used to distinguish the executables belonging >> to the same module, if many) simply call(s) the related >> <Module>_main+++[_*]+++ from the library. >> +All the above symbols are part of the generated libtest<Module>.la >> libraries. >> +The generated main executable(s) (named <module>_+++main[_*]+++, where >> the >> +optional suffix is used to distinguish the executables belonging to the >> same >> +module, if many) simply call(s) the related <Module>_main+++[_*]+++ from >> the >> +library. >> >> -Platform specific >> -~~~~~~~~~~~~~~~~~ >> -These tests are located under 'platform/<platform>/test'. There is one >> such area for each platform implementing ODP. >> -This location will be referred as <PLATFORM_SPECIFIC> in the rest of >> this document. >> +=== Platform specific === >> >> -The normal case >> -^^^^^^^^^^^^^^^ >> -If the considered platform needs no platform specific tests, this >> directory simply needs to contain a single Makefile.am listing each of the >> executables (named <module>_main) built from the platform agnostic area. >> The executables are listed in the automake TEST variable and will therefore >> be run on "make check". >> +These tests are located under 'platform/<platform>/test'. There is one >> such area >> +for each platform implementing ODP. >> +This location will be referred as <PLATFORM_SPECIFIC> in the rest of this >> +document. >> >> -For the linux-generic platform, most tested modules fall into this >> category: currently, the 'platform/linux-generic/test/Makefile.am' looks as >> follows: >> +==== The normal case ==== >> + >> +If the considered platform needs no platform specific tests, this >> directory >> +simply needs to contain a single Makefile.am listing each of the >> executables >> +(named <module>_main) built from the platform agnostic area. The >> executables are >> +listed in the automake TEST variable and will therefore be run on "make >> check". >> + >> +For the linux-generic platform, most tested modules fall into this >> category: >> +currently, the 'platform/linux-generic/test/Makefile.am' looks as >> follows: >> >> [source,am] >> ---- >> @@ -175,22 +238,39 @@ endif >> >> ---- >> >> -With the exception for module pktio, all other modules testing just >> involves calling the platform agnostic <module>_main executables (in >> test/validation). >> +With the exception for module pktio, all other modules testing just >> involves >> +calling the platform agnostic <module>_main executables (in >> test/validation). >> >> -Using other languages >> -^^^^^^^^^^^^^^^^^^^^^ >> -The pktio module, above, is actually tested using a bash script. This >> script is needed to set up the interfaces used by the tests. The pktio_run >> script eventually calls the platform agnostic >> 'test/validation/pktio/pktio_main' after setting up the interfaces needed >> by the tests. >> -Notice that the path to the script, 'pktio/pktio_run', is pointing to a >> file within the <PLATFORM_SPECIFIC> tree so is private to this platform. >> Any languages supported by the tested platform can be used there, as it >> will not impact other platforms. >> -The platform "private" executables (such as this script), of course, >> must also return one of the return code expected by the automake test >> harness (0 for success, 77 for skipped, other values for errors). >> +==== Using other languages ==== >> >> -Defining test wrappers >> -^^^^^^^^^^^^^^^^^^^^^^ >> -The pktio case above is actually using a script as wrapper around the >> "standard" (platform independent) test executable. Wrappers can also be >> defined by using the LOG_COMPILER variable of automake. >> -This is applicable in cases where the same wrapper should be used for >> more then one test, as the test name is passed has parameter to the >> wrapper. A wrapper is just a program expecting one argument: the test name. >> +The pktio module, above, is actually tested using a bash script. This >> script is >> +needed to set up the interfaces used by the tests. The pktio_run script >> +eventually calls the platform agnostic >> 'test/validation/pktio/pktio_main' after >> +setting up the interfaces needed by the tests. >> +Notice that the path to the script, 'pktio/pktio_run', is pointing to a >> file >> +within the <PLATFORM_SPECIFIC> tree so is private to this platform. Any >> +languages supported by the tested platform can be used there, as it will >> not >> +impact other platforms. >> +The platform "private" executables (such as this script), of course, >> must also >> +return one of the return code expected by the automake test harness >> +(0 for success, 77 for skipped, other values for errors). >> >> -Automake also supports the usage different wrappers based of the >> executable filename suffix. See >> https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness] >> for more information. >> +==== Defining test wrappers ==== >> >> -To add a wrapper around the executed test, just add the following >> LOG_COMPILER definition line in the '<PLATFORM_SPECIFIC>/Makefile.am': >> +The pktio case above is actually using a script as wrapper around the >> "standard" >> +(platform independent) test executable. Wrappers can also be defined by >> using >> +the LOG_COMPILER variable of automake. >> +This is applicable in cases where the same wrapper should be used for >> more then >> +one test, as the test name is passed has parameter to the wrapper. A >> wrapper is >> +just a program expecting one argument: the test name. >> + >> +Automake also supports the usage different wrappers based of the >> executable >> +filename suffix. See >> + >> https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html[Parallel-Test-Harness] >> +for more information. >> + >> +To add a wrapper around the executed test, just add the following >> LOG_COMPILER >> +definition line in the '<PLATFORM_SPECIFIC>/Makefile.am': >> >> [source,am] >> ---- >> @@ -221,18 +301,40 @@ echo "Do something to clean up the mess here :-)" >> exit $res >> ---- >> >> -Note how the above script stores the return code of the test executable >> to return it properly to the automake test harness. >> +Note how the above script stores the return code of the test executable >> to >> +return it properly to the automake test harness. >> >> -Defining platform specific tests >> -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> -Sometimes, it may be necessary to call platform specific system calls to >> check some functionality: For instance, testing odp_cpumask_* could involve >> checking the underlying system CPU mask. On linux, such a test would >> require using the CPU_ISSET macro, which is linux specific. Such a test >> would be written in '<PLATFORM_SPECIFIC>/cpumask/...' The contents of this >> directory would be very similar to the contents of the platform agnostic >> side cpu_mask tests (including a Makefile.am...), but platform specific >> test would be written there. >> -'<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the >> platform specific tests (by listing their module name in SUBDIRS and >> therefore calling the appropriate Makefile.am) and then it would call both >> the platform agnostic executable(s) and the platform specific test >> executable. >> +==== Defining platform specific tests ==== >> >> -Marking tests as inactive >> -^^^^^^^^^^^^^^^^^^^^^^^^^ >> -The general policy is that a full run of the validation suite (a "make >> check") must pass at all times. However a particular platform may have one >> or more test cases that are known to be unimplemented either during >> development or permanently, so to avoid these test cases being reported as >> failures it's useful to be able to skip them. This can be achieved by >> creating a new test executable (still on the platform side), giving the >> platform specific initialisation code the opportunity to modify the >> registered tests in order to mark unwanted tests as inactive while leaving >> the remaining tests active. It's important that the unwanted tests are >> still registered with the test framework to allow the fact that they're not >> being tested to be recorded. >> +Sometimes, it may be necessary to call platform specific system calls to >> check >> +some functionality: For instance, testing odp_cpumask_* could involve >> checking >> +the underlying system CPU mask. On linux, such a test would require >> using the >> +CPU_ISSET macro, which is linux specific. Such a test would be written in >> +'<PLATFORM_SPECIFIC>/cpumask/...' The contents of this directory would >> be very >> +similar to the contents of the platform agnostic side cpu_mask tests >> +(including a Makefile.am...), but platform specific test would be >> written there. >> +'<PLATFORM_SPECIFIC>/Makefile.am' would then trigger the building of the >> +platform specific tests (by listing their module name in SUBDIRS and >> therefore >> +calling the appropriate Makefile.am) and then it would call both the >> platform >> +agnostic executable(s) and the platform specific test executable. >> >> -The odp_cunit_update() function is intended for this purpose, it is used >> to modify the properties of previously registered tests, for example to >> mark them as inactive. Inactive tests are registered with the test >> framework but aren't executed and will be recorded as inactive in test >> reports. >> +==== Marking tests as inactive ==== >> + >> +The general policy is that a full run of the validation suite (a "make >> check") >> +must pass at all times. However a particular platform may have one or >> more test >> +cases that are known to be unimplemented either during development or >> +permanently, so to avoid these test cases being reported as failures >> it's useful >> +to be able to skip them. This can be achieved by creating a new test >> executable >> +(still on the platform side), giving the platform specific >> initialisation code >> +the opportunity to modify the registered tests in order to mark unwanted >> tests >> +as inactive while leaving the remaining tests active. It's important >> that the >> +unwanted tests are still registered with the test framework to allow the >> fact >> +that they're not being tested to be recorded. >> + >> +The odp_cunit_update() function is intended for this purpose, it is used >> to >> +modify the properties of previously registered tests, for example to >> mark them >> +as inactive. Inactive tests are registered with the test framework but >> aren't >> +executed and will be recorded as inactive in test reports. >> >> In 'test/validation/foo/foo.c', define all tests for the 'foo' module: >> >> @@ -250,7 +352,8 @@ odp_suiteinfo_t foo_suites[] = { >> }; >> ------------------ >> >> -In 'platform/<platform>/test/foo/foo_main.c', register all the tests >> defined in the 'foo' module, then mark a single specific test case as >> inactive: >> +In 'platform/<platform>/test/foo/foo_main.c', register all the tests >> defined in >> +the 'foo' module, then mark a single specific test case as inactive: >> >> [source,c] >> ------------------ >> @@ -280,4 +383,6 @@ int foo_main(void) >> >> So 'foo_test_a' will be executed and 'foo_test_b' is inactive. >> >> -It's expected that early in the development cycle of a new >> implementation the inactive list will be quite long, but it should shrink >> over time as more parts of the API are implemented. >> +It's expected that early in the development cycle of a new >> implementation the >> +inactive list will be quite long, but it should shrink over time as more >> parts >> +of the API are implemented. >> -- >> 2.5.0 >> >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> https://lists.linaro.org/mailman/listinfo/lng-odp >> > > -- Mike Holmes Technical Manager - Linaro Networking Group Linaro.org <http://www.linaro.org/> *│ *Open source software for ARM SoCs
_______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
