On 12-03-10 09:02 PM, Chase Douglas wrote: > On 03/09/2012 05:33 PM, Gaetan Nadon wrote: >> On 12-03-09 05:49 PM, Chase Douglas wrote: >>> Copied from XORG_ENABLE_UNIT_TESTS. >>> >>> Signed-off-by: Chase Douglas<[email protected]> >>> --- >>> I pasted in a minimum version of 1.17.0, but I don't know if that's >>> how we do >>> things. >> yes, there is a bit of guess work here but it is the right number. >>> >>> This will be needed for xorg-gtest and integration testing in other >>> projects. >>> >>> xorg-macros.m4.in | 29 +++++++++++++++++++++++++++++ >>> 1 files changed, 29 insertions(+), 0 deletions(-) >>> >>> diff --git a/xorg-macros.m4.in b/xorg-macros.m4.in >>> index 4966df3..9cd73c0 100644 >>> --- a/xorg-macros.m4.in >>> +++ b/xorg-macros.m4.in >>> @@ -1056,6 +1056,35 @@ AC_MSG_CHECKING([whether to build unit test >>> cases]) >>> AC_MSG_RESULT([$enable_unit_tests]) >>> ]) # XORG_ENABLE_UNIT_TESTS >>> >>> +# XORG_ENABLE_INTEGRATION_TESTS (enable_unit_tests=auto) >>> +# ------------------------------------------------------ >>> +# Minimum version: 1.17.0 >>> +# >>> +# This macro enables a builder to enable/disable integration testing >>> +# It makes no assumption about the test cases' implementation >>> +# Test cases may or may not use Automake "Support for test suites" >>> +# >>> +# Interface to module: >>> +# ENABLE_INTEGRATION_TESTS: used in makefiles to conditionally >>> build tests >>> +# enable_integration_tests: used in configure.ac for additional >>> configuration >>> +# --enable-integration-tests: 'yes' user instructs the module to >>> build tests >>> +# 'no' user instructs the module not to >>> build tests >>> +# parm1: specify the default value, yes or no. >>> +# >>> +AC_DEFUN([XORG_ENABLE_INTEGRATION_TESTS],[ >>> +AC_REQUIRE([XORG_MEMORY_CHECK_FLAGS]) >>> +m4_define([_defopt], m4_default([$1], [auto])) >>> +AC_ARG_ENABLE(integration-tests, >>> AS_HELP_STRING([--enable-integration-tests], >>> + [Enable building integration test cases (default: ]_defopt[)]), >>> + [enable_integration_tests=$enableval], >>> + [enable_integration_tests=]_defopt) >>> +m4_undefine([_defopt]) >>> +AM_CONDITIONAL([ENABLE_INTEGRATION_TESTS], >>> + [test "x$enable_integration_tests" != xno]) >>> +AC_MSG_CHECKING([whether to build unit test cases]) >>> +AC_MSG_RESULT([$enable_integration_tests]) >>> +]) # XORG_ENABLE_INTEGRATION_TESTS >>> + >>> # XORG_WITH_GLIB([MIN-VERSION], [DEFAULT]) >>> # ---------------------------------------- >>> # Minimum version: 1.13.0 >> >> You must have considered using ENABLE_UNIT_TESTS but came to the >> conclusion that it would not be suitable. On the other hand it seems to >> do exactly the same thing. Can you share your conclusions? >> >> The motivation behind ENABLE_UNIT_TEST is to solve the problem where a >> few modules started adding unit testing, each of them using a different >> configuration option, working a little differently. This makes it hard >> for builders to run unit testing on all modules as they must read and >> understand each module code and write their build script accordingly. No >> ones fault, it's the nature of X being distributed in over 240 separate >> modules. >> >> The only mission of the macro is to provide a common configure option >> (--enable-unit-tests) with a matching ENABLE_UNIT_TESTS variable. >> Nothing else is planned for this macro. It does not require or depend on >> XORG_WITH_GLIB or XORG_LD_WRAP. >> >> I am not familiar with gtest, it's hard for me to say if >> --enable-unit-tests would be appropriate. You can look at xserver and >> libXt as examples of modules using it. > > The issue is that integration testing usually has different > dependencies from unit testing. Unit testing usually has no > dependencies, or maybe a dependency on a test framework. However, > integration testing usually has dependencies of a working stack. For > example, input integration testing for the X server will depend on: > > xorg-gtest > libX11 > libXi > libutouch-evemu > linux > root privileges, or special udev rules > > And development headers for it all. Integration testing for other > subsystems would have different stack requirements. > > One example where builds will want unit tests but not integration > tests is package builds. Debian package builds build and run the unit > tests during the build and fails the build if the unit tests fail. > However, integration tests during debian package builds isn't really > feasible without potentially opening security holes. > > I think unit and integration tests are sufficiently different that a > separate configuration option is required. > This is what I was hoping for. I assume that this new macro can be used by many other packages, not just xorg-gtest.
Would you mind adding the description in the commit text? I'd like to avoid others to randomly pick integration test macro over unit test macro or vice-versa. This is all the documentation we've got, aside from this list. Feel free to push. I'll make a release after I see a few runs on tinderbox. Reviewed-by: Gaetan Nadon<[email protected]> > Thanks, > > -- Chase > _______________________________________________ [email protected]: X.Org development Archives: http://lists.x.org/archives/xorg-devel Info: http://lists.x.org/mailman/listinfo/xorg-devel
