On Fri, Mar 09, 2001 at 09:56:37AM -0500, barries wrote:
> You may want to settle on using underscores everywhere or nowhere in the
> interests of consistency.
Yeah. Underscores read better but type slower. Hmmm...
> Perhaps it would be a tad easier to assume noplan if there's no plan
> call present. Then Test::Harness would complain about not seeing
> output.
It would be easier, but it would be alot easier to accidentally forget
to declare a plan up front. No, you have to declare some sort of
plan, even if its that you have no plan.
Something to make it easier though would be this syntax:
use Testing no_plan;
use Testing skip_all;
use Testing plan tests => 23;
That gets rid of the clumsy BEGIN block (which isn't *strictly*
necessary anyway).
> Perhaps this would work if noplan emitted a line at the end like
> "Test exiting after $Num_tests\n" or something and Test::Harness
> whined if that line were not present.
I'd like to alter what Test::Harness accepts as little as possible,
more in the interest of keeping the harness rules simple than anything
else. Wedging test names in will be difficult enough.
> > # Various ways to say "ok"
> > ok($this eq $that, $test_name);
> >
> > expect($this, $that, $test_name);
> > expect($this, qr/that/, $test_name);
>
> Thank you for preserving qr/foo/ and not '/foo/'.
I'm loathe to even keep qr// as a special case. Maybe
expect ($this, $that, $test_name);
expect_qr($this, qr/that/, $test_name);
I like that better. I know its rare to test for equality between two
regex references, but someone's going to complain about it. It'll
also make writing the code simpler. Less special input cases to deal
with.
> > skip($why, $test_name);
> >
> > todo($why, {$this eq $that}, $test_name);
>
> s/{/sub {/ ; # sub-avoidance only works in first parm, IIRC:
You're right. In fact, I'd explicitly tested that... Silly me.
> Hmmm, turning that around, perhaps:
>
> package Testing ;
>
> my $in_todo ; # Globals to let ok(), expect() know they're in a todo
> my $todo_why ; # and why they're in it
>
>
> sub todo (&$) {
> my ( $func, $why ) = @_ ;
>
> croak "No reason given for todo." unless defined $why && length $why ;
>
> local $in_todo = 1 ;
> local $todo_why = $why ;
> eval { $func->() } ;
> return print "ok ... # TODO $why\n" if $@ ;
> }
>
> Then todo-ing a test might look like:
>
> todo {
> ok( .... ),
> } "why" ;
>
> I had to require $why in the prototype to force an error if a user puts
> a comma after the } in that type of example.
Ooooh, I kind of like that last one!
I'm working with the Aegis CASE system. One of the things you can do is
require that a new test fails against the baseline code (ie. works with
your patch, fails without it). Good in practice, but your test has to
fail, but not puke. So I find myself writing alot of this:
if( My::Module->can('new_method') ) {
ok( My::Module->new_method );
my @args = qw(blurp blurp blurp);
ok( My::Module->new_method(@args) );
}
else {
ok( 0, "My::Module->can('new_method')" );
}
What if instead I wrote it like this:
todo {
ok( My::Module->new_method );
my @args = qw(blurp blurp blurp);
ok( My::Module->new_method(@args) );
} q{My::Module->can't("new_method") yet},
My::Module->can('new_method');
If the third argument is true, ok() see that its inside
Testing::todo() (either through caller or with local variables) and
print out the magic "# TODO $Skip_Why" comment! Otherwise it runs
normally. The third argument would be optional. Its basically an
until statement. "This is a todo test until My::Module::new_method()
is defined."
skip() could work similarly.
todo { test } $why, $until;
skip { test } $why, $until;
skip {
ok( fork );
ok( flock );
} q{There is no fork},
$^O =~ /^MacOS/;
We might have to work on the cannonical formatting (it looks a bit
cluttered at the end), but I think we're onto something here.
> > # Utility comparison functions.
> > eqarray(\@this, \@that);
> > eqhash(\%this, \%that);
> > eqset(\@this, \@that);
>
> How deep are these equality operators?
The equality operators go all the way down, and they use each other.
However, I haven't tested them against circular structures, etc...
That might be tricky.
> It'd be happy to see these done
> quietly by expect(), to keep the API simple. So if expect sees two refs
> of the same type passed in, it would deep-compare them, for some def of
> deep. I think
>
> expect( \%this, \%that, $test_name ) ;
>
> works ($test_name to be optional for me) well in a DWIMMERLY fashion.
No, that's magic. Magic bad. Tests should be fairly strict.
Consider testing a cached object constructor to make sure it returns
the same object referents (or perhaps a singleton class):
my $this_obj = My::Cached::Class->new($id);
my $same_obj = My::Cached::Class->new($id);
expect( $this_obj, $same_obj, 'caching is working' );
That should be a straight C<ref $this_obj eq ref $same_obj>.
Fortunately, it wouldn't be hard at all to write a new Testing::DWIM
module built on top of Testing:
sub Testing::DWIM::expect {
my($this, $that) = @_;
if( ref $this eq 'HASH' and ref $that eq 'HASH' ) {
return eq_hash($this, $that);
}
elsif( ref $this eq 'ARRAY' and ref $that eq 'ARRAY' ) {
return eq_array($this, $that);
}
else {
return Testing::expect($this,$that);
}
}
> > not ok 5 - simple exponential
>
> Please don't forget to print the caller's file and line here.
Repeating the same file over and over again will rapidly get annoying, but
the line number will be useful yes.
> Might also be cool to allow the test program to set a "confess" flag
> that gives you stack dumps for failing tests, I have cases where
> ok() is buried inside a utility sub and thus it's hard to tell what
> file and line the croak is on.
Good idea. Would that be on a per-test basis, or overall? Maybe
something like:
use Testing plan tests => 23, confess => 1;
And a more general
use Testing plan tests => 23, on_not_ok => \&Cry;
Both might be useful.
> > ok 6 - force == mass * acceleration
> >
> > The later gives you some idea of what failed. It also makes it easier
> > to find the test in your script, simply search for "simple
> > exponential".
>
> Hmmm, should testing warn if you give the same name to two tests?
Might be worthwhile to make that a plan option. Dunno if it should be
default.
I often find myself doing things like this:
my $foo = Foo::Class->new($id);
ok( $foo->name, 'Foo::Class has a name');
ok( $foo->number, ' and a number');
my $bar = Foo::Class::Bar->new($id);
ok( $bar->name, 'Foo::Class::Bar has a name');
ok( $bar->number, ' and a number');
Often a result of a cut-and-paste + search-and-replace.
> > ok($this eq $that);
> > ok($this eq $that, $test_name);
>
> This is implicitly promoting the call with lower diagnostic value.
You're right. (I habitually put calls in order of least arguments)
> Hey, howabout Huffman coding the API so the more useful, presumably more
> used, call is shortest:
>
> ok( $this, $that, $test_name ) ;
> test_assert( $this eq $that, $test_name ) ;
>
> Also continues the tradition of ok()ing things for most tests.
Hmm, looking at the Class::DBI tests about half of them could use
expect() and half couldn't. Some could be split up into an expect()
and an ok()... but its still split. Granted that's not exactly a
statistical samping, but it is a fairly complicated test.
Maybe we could cut down expect() into something smaller. Ummm,
test()? ;) A little too generic is the only problem.
--
Michael G. Schwern <[EMAIL PROTECTED]> http://www.pobox.com/~schwern/
Perl6 Quality Assurance <[EMAIL PROTECTED]> Kwalitee Is Job One
Do you have a map? Because I keep getting lost in your armpits.
