Hi all, Jeff Lavallee and I got together a couple of weeks ago and enabled the test profiles thing in Module::Build.
Patch and tests are here: http://scratchcomputing.com/tmp/MB-test_profiles.trunk8067.patch http://scratchcomputing.com/tmp/test_type.t http://scratchcomputing.com/tmp/test_types.t Repository: http://scratchcomputing.com/svn.mb/trunk (started from a drop of https://svn.perl.org/modules/Module-Build/trunk/ r8049) Summary: This allows you to define $foo test types in the constructor and ACTION_test$foo methods in your subclass. The tests will not be run under `./Build test`, but all defined types will be run under `./Build testall`. More info below. Some uses: o author/quality tests o networked tests o gui tests o database tests And anything else that might not be required for installation and/or might not work without special setup (e.g. be online, setup a database, have a $DISPLAY set, etc.) I've included documentation for the new "testall" target. I'm not certain where to put the "How this works" documentation, but I'll be happy to add it in whatever is the correct place. NOTE: All of the tests pass except for the it-was-like-that-when-I-got-here t/module_info.t (which I understand to be purposefully broken by Ken in r7859 from the log message "Add failing test for double-assignment".) Details and explanation follow. Why not a plugin: We don't have a plugin system yet. Also, Schwern convinced me that field-deployment would be simpler if this went in the MB core, which makes it easier for authors to get test results for special cases. Why: In my case, it was gui tests, in Jeff's case, you need a subscription to a Yahoo service. I believe some other projects have similar issues, so I think this will be very generally useful. Currently, the only way to gracefully skip tests which cannot be run without special setup involves checking an environment variable or other condition at the beginning of each non-core test and setting skip_all. This is sub-optimal both in that you must paste the checks at the front of each test file, and in that the test still does not run even when it was specifically requested by the --test-files argument unless the variable got set. Further, setting the variable doesn't stop the normal tests from running unless you have way too much prelude code in every test file. Before: ./Build test --test-files t/foo.gt # skips! try: SOMESILLY_VAR=1 ./Build test # runs them all! pull hair... finally: SOMESILLY_VAR=1 ./Build test --test-files t/foo.gt Hmm. and then I want to run all-but-only-those? err? ... SOMESILLY_VAR=1 ./Build test $(for i in $(grep -lr \ SOMESILLY_VAR t/); do echo "--test-files" $i;done) After: ./Build testgui OR ./Build test --test-files t/foo.gt OR ./Build testgui --test-files t/gui_01/ So, this makes it easier to write and run the tests, and is discoverable by the installing user via `./Build help`. How it works: To define a type "special", the author defines test_types => {special => '.st'} in the constructor parameters. This registers the type for use in `testall`, but you currently still need to define ACTION_testspecial in your subclass (we could inject a method for you, but I'm inclined to leave that up to a plugin.) This action (minimally) looks like so: sub ACTION_testspecial {$_[0]->generic_test(type => 'special');} You might have reservations about having tests which don't match /.*\.t$/, but it doesn't bother me. If it bugs you, the facility is there for defining actions independently of file extensions. For example, you could say test_types => {database => undef} And then have your ACTION_testdatabase() twiddle some environment variables which activate the tests and/or run .t files from elsewhere and/or do something that doesn't involve .t files at all. It will still run during `./Build testall` and you now have a place to document it which will be accessible via `./Build help testdatabase`. Thanks, Eric -- Issues of control, repair, improvement, cost, or just plain understandability all come down strongly in favor of open source solutions to complex problems of any sort. --Robert G. Brown --------------------------------------------------- http://scratchcomputing.com ---------------------------------------------------
