On 20/07/06, chromatic <[EMAIL PROTECTED]> wrote:
On Thursday 20 July 2006 02:59, Fergal Daly wrote:
> > There already exist two long-accepted, well-understood, coded, tested,
> > and debugged ways to answer both questions. I don't see the value in
> > adding a third, especially when it's not substantially better than either
> > and can be wrong in other circumstances!
>
> What circumstances? Example please.
In a deep call stack beyond the $Level.
Example code please. You're saying there's a situation where using
$Level produces the incorrect output. I'm saying there isn't.
Everything I'm saying is based on my opinion that $Level is reliable,
everything you're saying is based on your opinion that it is not
reliable.
As evidence, I submit this small piece of code
----
use Test::More 'no_plan';
fail("failure at line 3");
sub foo { fail("failure at line 5") };
foo();
----
which outputs the following
----
not ok 1 - failure at line 3
# Failed test 'failure at line 3'
# in fail.t at line 3.
not ok 2 - failure at line 5
# Failed test 'failure at line 5'
# in fail.t at line 5.
1..2
# Looks like you failed 2 tests of 2.
----
and as you can see, it correctly identifies the lines in my test
script that call into the testing library.
> Without stack traces, here's the only way I know to use subroutines
> safely in test scripts.
> sub foo {
> my ($thing, $name) = @_;
> is($thing->x(), "x", "checking x in $name"); # line 4
> is($thing->y(), "y", "checking y in $name");
> }
You forgot $Level or :TestHelper. Was this an accident or deliberate?
Accident.
> > Your patch relies on $Level too, as it calls the caller() *method* within
> > Test::Builder. If someone has not set it correctly, your patch will
> > report the wrong information.
>
> Someone? Who? If there's a bug in one of the Test::XXX modules then
> yes the stack trace could be wong but that's a bug that needs to be
> fixed and is not a problem with my patch.
I thought that was the problem you were trying to fix: reporting the wrong
location.
No, as far as I'm concerned Test::Builder never reports the wrong
location (modulo bugs _outside_ of Test::Builder).
If your patch doesn't fix that, I'm not sure what the point is. Why report
extra information if you know it's not reliable?
The information is perfectly reliable, it's just not enough in some
circumstances.
> > sub an_extra_level
> > {
> > ok( $_[0], $_[1], $_[2] );
> > }
> >
> > sub another_extra_level
> > {
> > an_extra_level( @_ );
> > }
> >
> > sub yet_another_extra_level
> > {
> > another_extra_level( @_ );
> > }
> >
> > sub still_more_extra_levels
> > {
> > yet_another_extra_level( @_ );
> > }
> >
> > still_more_extra_levels( $have, $want, $diagnostic );
> >
> > Where do you start debugging that?
> With a stack trace it's pretty easy.
Yes, you *merely* check *every* level reported in the stack trace to find the
place that the call to the test function *may* have occurred, hoping that
someone set $Level appropriately.
You have completely misunderstood why I want a stack trace because you
still think that $Level is unrealiable.
If $Level has been set incorrectly then that's a bug, plain and
simple. Garbage in garbage out. I'm not try to fix other people's
bugs.
Let me give another more complex example (one which cannot be got
around by adding test descriptions)
sub is_foo {
my $thing = shift;
is($thing->x, "x", "checking x"); # 3
is($thing->y, "y", "checking y"); # 4
}
sub has_2_foos {
my $thing = shift;
is_foo($thing->a); # 8
is_foo($thing->b); # 9
}
my $t1 = Thing->new(1);
my $t2 = Thing->new(2);
has_2_foos($t1); # 15
has_2_foos($t2); # 16
If one of those tests fails, we need 3 pieces of information to locate it
1 was it $t1 or $t2
2 were we looking at ->a or ->b
3 were we looking at ->x or ->y
the test name will give us #3 and we can get one of #1 or #2 by
setting $Level but we can never get both.
A stack trace would look like
# failed "checking x"
# at line 3
# called at line 9
# called at line 15
and we would know that the problem was with $t1->b->x
F
I don't really want to grovel through extra information that I know probably
isn't useful by default. If Schwern is amenable, I wouldn't complain too
much if this were an option for the person running the test (probably set
through an environment variable), but it's too much information that I don't
trust to be useful for a default.
> > That's why there are test descriptions too.
> But some people don't like having to provide names
Yes, some people write bad code. I don't think there's an algorithmic way to
fix that.
-- c
