On Sat, Dec 15, 2001 at 02:19:21PM -0800, Kurt D. Starsinic wrote: > On Dec 15, Dave Rolsky wrote: > > Ok, so that's a bit off the topic of "why use isa_ok()" but I just don't > > see why people seem to object to the use of Test::More in the core Perl > > tests. I can't see how it couldn't help improve the quality of the tests > > while providing a standardized way to do the things that all the tests do > > anyway. > > I, for one, don't object to the use of Test::More in the core Perl > tests. I'm just being responsibly skeptical about the promotion of many > different test harness modules with many different functions. It runs > the risk of _seeming_ like the People in Charge don't know what testing > is for, and of not providing effective guidance to those people writing > their first tests.
We have Test::Simple, Test::Tutorial and eventually Test::FAQ for such guidance. Work on those if you're worried about providing effective guidance. Beyond that, I'm not going to cut down my toolbox to only a hammer just because it looks tidy. (And its PERL! Of course you're going to have 25 different types of bevels in the toolbox.) Its also been "discovered" just recently that its now possible to inter-mix several different Test::* modules in the same program, as long as they all use the Test::Builder backend (and Barrie found a way to cheat with Test.pm). http:[EMAIL PROTECTED]/msg01091.html This means its no longer necessary to write monolithic test modules. You can write special purpose modules for special purpose things and combine them as you please. After the next release, which will complete the interface with cmp_ok() (aka. is_op), unlike() and eval_ok(), Test::More's growth is pretty much over. I'll leave the rest to seperate modules. > I also think that we have a real testing problem, which is that it > isn't _trivially_ easy to run a huge test suite (like Alzabo's, or Perl's) > and to find out at the end which tests failed and what their output was. > 3000 lines and 20 minutes later, a message that 3 out of 10000 tests > failed is not maximally useful. Greater variety of test functions is not > (IMNSHO) nearly as important as addressing that. Test::More outputs its diagnostics to STDERR, so they appear as you run 'make test'. PERL_DL_NONLAZY=1 /usr/local/bin/perl -Iblib/arch -Iblib/lib -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t t/Module-Info....NOK 3# Failed test (t/Module-Info.t at line 15) # got: 'foo' # expected: 'bar' t/Module-Info....ok 73/74# Looks like you planned 74 tests but ran 1 extra. t/Module-Info....dubious Test returned status 2 (wstat 512, 0x200) DIED. FAILED test 3 Failed 1/74 tests, 98.65% okay Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- t/Module-Info.t 2 512 74 1 1.35% 3 Failed 1/1 test scripts, 0.00% okay. 0/74 subtests failed, 100.00% okay. Even in the core: $ ./perl TEST ../lib/foo.t t/../lib/foo....# Failed test (../lib/foo.t at line 10) # got: 'foo' # expected: 'bar' # Looks like you failed 1 tests of 1. FAILED at test 1 Failed 1 test script out of 1, 0.00% okay. Its a little messy, but it works. I'd welcome help making it look nicer. However t/test.pl (the low-fat Test::More substitute which almost everything in t/ uses) puts its diagnostics on STDOUT which gets sucked up by Test::Harness. That could be changed. As for providing further information at the end of a Perl core test run, that requires modifying t/TEST which has its own set of mental health problems. Finally, if you want to see real movement in improving the state of 'make test' output, help out finishing up Test::Harness 2.00 http://www.pobox.com/~schwern/src/Test-Harness-2.00_01.tar.gz and finishing the ExtUtils::* tests so we can work on that safely (the lives of MakeMaker and Test::Harness being intertwined). -- Michael G. Schwern <[EMAIL PROTECTED]> http://www.pobox.com/~schwern/ Perl Quality Assurance <[EMAIL PROTECTED]> Kwalitee Is Job One An official "I want James Earl Jones' cock up my ass" t-shirt. http://www.goats/com/archive/010303.html
