On Wednesday 19 July 2006 15:17, demerphq wrote:
> On 7/19/06, chromatic <[EMAIL PROTECTED]> wrote:
> > > Excuse me? Where did I say the code was "broken"?
> >
> > Wasn't that the implication when you said you've seen misleading line
> > numbers many times?
>
> No. I didnt imply anything. I spelled it out quite clearly.
Fine, then you said "quite clearly" that it was broken.
> I said that the code to find the line number uses a heurisitic that
> gets things wrong, and that I didn't see any way to improve the
> heurisitc.
But there is one! Give it the explicit knowledge that you, the writer of the
test suite, has.
> And therefore that the line number information is an
> unreliable way to find what code actually failed the test, which in my
> experience is a problem properly solved by using test names.
If you don't give it a test name, you have the exact same problem, only worse,
because there's absolutely *no* information. Test::Builder has a heuristic
for giving the location of failures and it works *most* of the time.
Test::Builder has no heuristic for giving test names in the absence of test
names, and thus it works *none* of the time.
> > > use Test::More tests => 3;
> > >
> > > sub my_ok {
> > > ok($_[0],$_[1]);
> > > }
> >
> > I don't know why you'd expect this to report the right line numbers; this
> > code really *is* broken.
>
> No, its not broken, using subroutines is not broken.
No, using buggy, incomplete subroutines is. At least complaining that an
incomplete, buggy subroutine does the wrong thing is silly. Of course it
does the wrong thing. You wrote it to do the wrong thing.
Would you complain that my favorite chocolate chip cookie recipe is awful if
you deliberately left out the sugar?
> And, even it were
> broken, its a common mistake. The code is basically a stripped down
> example of stuff i see in test files A LOT.
Yes, it's common. That doesn't mean it's not broken. I'm not in the habit of
voting whether to redefine wrong behavior as right because it's common.
> And even if i were to concede that my_ok() is broken (which I don't)
> there is still Fergals example of a data driven tests in a loop:
>
> my @tests=(....);
>
> foreach my $test (@tests) {
> is($test->[0],$test->[1]);
> }
>
> how do you propose to get a useful line number out of that? Are you
> going to say that its broken as well?
Not at all. The line number is correct: there are no additional call frames
between the call to the underlying library and the call point of the test.
The line number reported there is the correct line number for the important
point of calling the test.
I didn't say it was useful. I said it was correct.
> Now if it was written
>
> foreach my $test (@tests) {
> is($test->[0],$test->[1],$test->[2]);
> }
>
> Then i can find the test _easily_.
For a value of "easily" defined as "by looking up the definition of the data
structure in @tests and then reading further in the code until I find the
point of the call", yes.
I prefer to have two points of information for debugging this sort of test:
the diagnostic and the line number and source file of the call point.
> And speaking of
> $Test::Builder::Level at me let me ask a question, how many are going
> to read Test::Builder to get the line numbers from tests in Test::More
> right? Experience shows not very many people[1].
People who don't read the documentation and get bad results get approximately
zero sympathy from me.
> Heck the variable isn't even mentioned in Test::More.
Then it should be, though maybe Test::Tutorial is a better place for it.
> And Test::Builder isn't mentioned in Test::Simple at all (presumably because
> it doesnt use it, in which case $Test::Builder::Level isnt going to help.)
That's quite an assumption. Write a test case.
> [1] I might wager that a lot of test authors dont even notice the
> problem. I say this because in my own experience of writing my own
> test suites the line number is something I dont even look at. I know
> what test failed because I wrote the test file and the code its
> testing and I can find the relevent stuff in an instant. Finding the
> code responsible for a failing test is something that IMO is done more
> often by module consumers who for one reason or another are seeing
> things go wrong. The author on the other hand is unlikey to have seen
> the tests fail, and therefore might not even know the line numbers are
> wrong.
Lazy people often write buggy code. I sure do.
-- c
