On 1 Nov 2007, at 15:38, Michael Peters wrote:
Isn't this the problem that SKIP was designed to fix? If it needs to
dynamically
determined if a test should run, isn't the test itself the best
place to do
that? xt tests shouldn't be run unless explicity done so by the
runner of the
tests (a developer) who presumably knows that they are doing.
The feature request that started all this relates to a large test
suite most of which needs to be skipped if a graphical environment
isn't available - so presumably it'd involve executing hundreds of
tests just to find out that they skipped.
Instead of "something else" telling the harness which tests to run,
why not have
the harness run them all and the individual tests can decide if they
don't want
to go through with it or not. If "something else" tells the harness
which tests
to run, we loose the information about which tests didn't run and why.
Having to run each test to find out it didn't want to run doesn't
scale well. Having a mechanism to dynamically select subsets of tests
can only encourage people to write more tests I'd have thought.
If we decide on a set of rules for which xt/* tests are executed under
which circumstances it also gives us a language in which to express
those rules - so instead of being hard wired into TAP::Harness we'll
have a little TSP document that could be used with any TAP parser.
I've also seen a discussion of test ordering and the practice of
numbering tests to guarantee a ordering recently; ordering and more
complex dependencies could also be handled by a t/controller.t that
emits TSP.
Testing in Perl is really nice and simple and I'm not seeing the
benefits or the
problems being solved with this complexity that can't already be
solved using
something else.
You could presumably have said the same about the idea of TAP based
testing. You can after all just have a bunch of programs that pass or
fail and run them from the shell.
I'm being slightly facetious there :)
I agree that parsimony should guide our decisions but I'm not finding
it hard to think of use cases for TSP.
--
Andy Armstrong, Hexten
