Eric Wilhelm wrote:
>>> The only impossible spot is when tests are inside e.g. a
>>> runtime dispatched method, no? (And, given the procedural paradigm,
>>> that seems to be an odd case.)
>> No, that's not odd at all. Any data driven testing system will be that
>> way. Tests are run based on some config file (I know Ovid uses YAML
>> for something like this), etc.
>
> That still doesn't imply that we can't somehow count the number of tests
> with a computer instead of relying on humans to screw it up. If some
> combination of static analysis and early runtime can come up with a
> count, then it becomes possible to automatically plan without counting.
>
> So, whatever constructs make that impossible might be worth pondering.
With apologies to Godwin: As a programming discussion grows longer, the
probability of someone thinking they can solve the Halting Problem approaches
one.
Why can't the computer count the number of tests? Because it CAN NOT KNOW how
many tests there are. Here's why:
pass() for 1..rand 5;
Here is a more realistic example.
my @things = <$fh>;
for my $thing (@things) {
is $thing, "something";
}
Here's another:
$object->run_tests;
Because, in Perl and other languages, until you run it you can't know what
class $object is going to be, or what its inheritance tree will look like, and
once you do figure out which run_tests() will run (if any) you're back to the
problem of figuring out how many tests are in run_tests().
If we were using a mathematically provable functional programming language,
say Haskell, there are some cases where you can do static analysis and figure
out how many tests are going to run. If you got really clever I'll bet you
could do it for most cases. But when a test involves any randomness or
outside input it is impossible to know. The best you can do is know when you
can not know.
For those who don't believe me, feel free to push this rock up and down the
hill of your choice until enlightenment or death occurs.
For those of you in the "I can write something good enough" camp, feel free to
use Shlomi's Test::Count.
http://search.cpan.org/perldoc?Test::Count
--
164. There is no such thing as a were-virgin.
-- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army
http://skippyslist.com/list/
