# from Smylers
# on Friday 17 August 2007 03:13 pm:
>Eric Wilhelm writes:
>Why can't gui tests simply be in t/ and be skipped if the appropriate
>environment isn't available? That way all users who are in that
>particular environment run the tests, rather than only those who've
>discovered the xt/gui/ directory.
"Can't you just" paste some 'if($ENV{WHATEVER})' code into all of your
tests?
Yes, "I can just".
I don't want to.
I don't want the test suite to start 100 test processes and skip 90 of
them.
I don't want to maintain some randomish, pasted 'if($ENV{WHATEVER})'
code in 20+ test suites.
I don't want other CPAN authors to have to do these things either.
I don't want other CPAN authors to have to discover "the recommended
way" to do these things.
>> What I would ultimately like to see happen is that these
>> subdirectories fall into some form of standard(ish) naming scheme
>> which would allow testers and qa efforts to run particular subsets.
>> That is, the smoke box (or rigorous end-user upgrade verification
>> process) could set variables to test the author, 'gui', 'network',
>> and 'postgres' tests[1] without stumbling into e.g. "must have an
>> X.com web-services account".
>
>That requires a central standardized repository of permitted
>subdirectories and what needs to be done to run the tests there.
It might be useful, but is not strictly required. I would still prefer
that to "everything is turing-complete and self-selecting". Plus,
self-selecting tests bring us back to the same point which started all
of this which is that not all of them are absolutely critical to the
installation.
Part of my original point is that it would be useful[1] to define a
standardized mapping of what functionality or resources are required to
run typical subsets:
author: some pod testing, kwalitee, etc prereqs
network: network access
gui: a gui
Yes, the rest is a bit more complicated. Here, I'll define the
arbitrary invent-your-own-thing ad-hoc asylum location right now:
misc: everything else
> Surely it's better to let each distribution keep control of this, by
> specifying in code what's needed to run those particular tests (and
> skipping otherwise)?
It is only better in specific cases of "totally unlike everything else".
Self-selecting tests are clearly (to me) worse for things as simple and
typical as pod, network, and gui because it is ad-hoc,
non-discoverable, and requires pasting code or modifying all of the
tools.
There is nothing about categorization which prohibits ad-hoc from
occurring within it. However, you cannot easily have categorization
within ad-hoc.
[1] useful for developers, smoker testers, qa-testers, and end-users.
Each of which have various reasons to want to run or not run various
categories of tests (and do it in a standard way.)
--Eric
--
If the above message is encrypted and you have lost your pgp key, please
send a self-addressed, stamped lead box to the address below.
---------------------------------------------------
http://scratchcomputing.com
---------------------------------------------------
