Re: ANNOUNCE: Test::XML
On Mon, Mar 31, 2003 at 12:45:14PM +0100, Dominic Mitchell wrote: I've just uploaded a new module to CPAN[1], Test::XML. It contains a couple of functions for examining XML output of functions in a slightly saner way than is(). http://search.cpan.org/author/SEMANTICO/Test-XML-0.03/ is_xml( 'foo /', 'foo/foo' ); # PASS I like this. -- You see, in this world there's two kinds of people. Those with loaded guns, and those who dig. Dig. -- Blondie, The Good, The Bad And The Ugly
Re: testing with stubs
On Thu, Dec 12, 2002 at 03:27:32PM -0600, Danny Faught wrote:
I have a legacy Perl script, not object-oriented but at least ported to
Perl 5. I want to use a real-world example, rather than new code like
what Kent Beck uses in the book Test-Driven Development. So I thought
I'd implement a unit test for one of the functions in this old script.
I'm using Test::Unit::Procedural.
I'm pulling my hair out trying to stub out the functions that the
function under test is calling. Here's the function (complete with an
error that will be corrected as part of the exercise):
sub abort_handler {
my ($event) = @_;
print STDERR stress_driver: aborted by SIG$event-data()\n;
log (stress_driver: aborted by SIG$event-data());
exit (cleanup);
}
I want to stub out (replace) the log and cleanup functions and the
builtin exit function.
In your test, do something like this:
BEGIN {
# Override with itself so it can be overridden at run-time later.
*CORE::GLOBAL::exit = sub {
CORE::exit @_;
}
}
{
my $exit_code;
no warnings 'redefine';
local *CORE::GLOBAL::exit = sub {
$exit_code = shift;
goto EXIT;
};
my @logs = ();
local *Foo::log = sub {
push @logs, @_;
}
my $cleanup_called = 0;
local *Foo::cleanup = sub {
$cleanup_called++
return something;
}
...do code which calls abort handler...
EXIT: ..tests here...
}
I'll probably also want to redirect STDERR to capture that output as well
That's easy, tie it. Look at t/lib/TieOut.pm in the Test-Simple tarball.
But I can't figure out how to turn unit test mode on and off.
Scripts are hard to test. Libraries are easy. So...
Step 1: pull all the subroutines out of the script and into a seperate
library.
Step 2: test that library.
That leaves a lot less to try and test in the script.
Beyond that the real problem is that scripts must be called as seperate
processes which makes much of the subroutine overriding tricks above
difficult. I have a trick which simulates running a perl script, but all
its really doing is eval'ing the code in the current process. This means
the tricks above will work.
It can be found here:
http://www.pobox.com/~schwern/tmp/esmith-TestUtils.pm
Adapt as you like.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
conway: unit of mind expansion. One Conway == ~20 lines of Perl code
found in $CPAN/authors/id/D/DC/DCONWAY, which gives the sensation
of your brain being wrapped around a brick, with kiwi juice squeezed
on top.
-- Ziggy
Wiki Wiki Working
After shuffling some files around and doing some permissions tricks, the Perl QA Wiki is working again! Everything is now editable. http://www.pobox.com/~schwern/cgi-bin/perl-qa-wiki.cgi -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Schwern Contrary to popular belief, Perl is not developed by a team of independently wealthy mad scientists. autrijus (qw/independent wealthy mad scientist/)[rand(3)]; # choose one autrijus the reason it's not rand(4) is because perl isn't a science :p
Re: Changing T::B behavior
, but test state is shared between the original and the copy my $shared_tb = Test::Builder-create(state = $tb-state); You're right, this one isn't necessary given the *state methods sketched out above. with those plus the push/pop_stack methods you can pick and choose what sort of state sharing you want. If we have explicit access to the state - is there any advantage in having a stack of objects? Localising $Test::Builder::Default, or saving/restoring T::B-default would seem to give you all the functionality necessary. (You could always subclass T::B if you really wanted it :-) Its more convenient. I really don't want to think about how all this will interact with threading :-/ (Famous last words) There shouldn't be a problem. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One It's Tobacco time!
Re: Test::Builder-level
On Tue, Nov 12, 2002 at 04:21:38PM +, Adrian Howard wrote:
At YAPC::Europe there was some discussion about Test::Builder-level,
$Test::Builder::Level and the fact that they don't really work well as
implemented. I know we reached some sort of consensus about how to do it
better, but I've forgotten it.
Yet another suggestion to add to the list of possible options.
Most of the time when I need to mess with level, the code that does
the test is sitting in a different package from the one that is
calling the test. How about having it walk up the call stack until it
find a package that differs from the one it was called in?
No nasty messing with numbers and you don't have to fiddle with it
every time you add/remove a sub in the call chain.
ETOOMAGICAL and it makes it impossible to have something like this:
# Test code
use Test::Foo 'test_func';
test_func($stuff);
# Test/Foo.pm
package Test::Foo;
use Test::Foo::Bar 'real_test_func';
sub test_func {
...
real_test_func(@_);
...
}
# Test/Foo/Bar.pm
package Test::Foo::Bar
use Test::Builder;
my $TB = Test::Builder-new;
sub real_test_func {
...
$TB-ok($test);
...
}
a practical example would be an Exporter/Exporter::Heavy poor-man's
autoloader setup.
Of course, there's nothing stopping you from overriding level() to be
magical, once I implement your ideas about being able to change the default
object and stack them.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
You'll do.
Changing T::B behavior (was Re: And for the mod_perl folks... Test::Builder-reset)
On Tue, Nov 12, 2002 at 11:04:53PM +, Adrian Howard wrote:
O! Just had an idea prompted by reset(). How about having
$builder-push_state;
$builder-pop_state;
that would store and restore the complete state of the builder object
in a stack? Would make it easier to isolate a set of tests from the main
sequence.
I think I'll merge your $Test::Builder::default idea with this. Have a
stack of Test::Builder objects rather than just the states. This way you
can do something like the above. If you add a $tb-copy method which
returns a carbon-copy of $tb then...
$tb-push_state; # calls $tb-copy and pushes it into the stack
# the copy will now be used by default
$tb-pop_state; # pops the stack and tosses it, restoring the
# original.
but what if you want to use your own subclass? Well if you also add a
$tb-state method which return the full internal state it makes it possible
for subclasses to copy each other. So...
$tb-push_state(My::Test::Builder-copy($tb));
...my test code...
$tb-pop_state;
The problem there is the case where you want to override behaviors but still
keep state between the two objects. So things like the test counter and
test details would have to be preserved. I guess this is what chromatic was
talking about when he suggested having only certain parts of the internal
state be a singleton.
So looking at the stuff in reset():
sub reset {
my ($self) = @_;
$Test_Died = 0;
$Have_Plan = 0;
$No_Plan = 0;
$Curr_Test = 0;
$Level = 1;
$Original_Pid = $$;
@Test_Results = ();
@Test_Details = ();
$Exported_To= undef;
$Expected_Tests = 0;
$Skip_All = 0;
$Use_Nums = 1;
($No_Header, $No_Ending) = (0,0);
$self-_dup_stdhandles unless $^C;
return undef;
}
We can seperate them out into Test::Builder configuration and test state.
This is test state and would be shared amongst the T::B objects in the
default stack
$Test_Died
$Have_Plan
$No_Plan
$Curr_Test
$Original_Pid
@Test_Results
@Test_Details
$Expected_Tests
$Skip_All
and these are T::B configuration values and wouldn't be shared, though they
would be initially copied.
$Level
$Exported_To
$Use_Nums
$No_Header
$No_Ending
the output filehandles
So I guess we need several constructors.
real constructor - a normal constructor
singleton constructor - returns a single, default object. This object
is actually a wrapper around the default object
stack.
copy - copies the state of one object to a new one
copy share state- like copy, but test state is shared between the
original and the copy
with those plus the push/pop_stack methods you can pick and choose what sort
of state sharing you want.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
My enormous capacity for love is being WASTED on YOU guys
-- http://www.angryflower.com/497day.gif
Re: Removing CPAN's dependency on Test::More
On Wed, Nov 13, 2002 at 11:32:21AM +1100, Ken Williams wrote: So why does CPAN.pm depend on Test::More anyway? It only uses it during its 'make test' phase, not during runtime operations. And when you can't install modules very easily because you don't have CPAN.pm working properly, it's, well, not very easy to install other modules. ;-) Seems like one of the following would be better: 1) Include a copy of Test::More and Test::Harness in a t/lib/ directory and use that This is easiest. You should only have to throw Test::More/Builder in there and slap a use lib 't/lib' in mirroredby.t and Nox.t. The tests aren't using any features which require a Test::Harness upgrade. 2) Rework the two Test::More-using tests (mirroredby.t and Nox.t) to just use Test.pm Or is this a sneaky way to get more people to install Test::More on their machines? CPAN::MakeMaker? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Ooops, fatal mutation in the test script.
[ANNOUNCE] Test::Simple/More/Builder 0.48_01
http://www.pobox.com/~schwern/src/Test-Simple-0.48_01.tar.gz Since there's a lot of internal and external change going on in this release, I decided to put out an alpha first. The external API changes: * The Test::Harness upgrade is no longer optional. * threads.pm is no longer automatically loaded And new features. * Test::Builder-reset * use_ok() now does proper version checks * 'no_diag' switch for Test::More Did I miss anything? I had a lot of patches to catch up on. I'm punting on Nick's ending callback idea until next release. 0.48_01 Mon Nov 11 02:36:43 EST 2002 - Mention Test::Class in Test::More's SEE ALSO * use_ok() now DWIM for version checks - More problems with ithreads fixed. * Test::Harness upgrade no longer optional. It was causing too many problems when the T::H upgrade didn't work. * Drew Taylor added a 'no_diag' option to Test::More to switch off all diag() statements. * Test::Builder/More no longer automatically loads threads.pm when threads are enabled. The user must now do this manually. * Alex Francis added reset() reset the state of Test::Builder in persistent environments. - David Hand noted that Test::Builder/More exit code behavior was not documented. Only Test::Simple. 0.47 Mon Aug 26 03:54:22 PDT 2002 * Tatsuhiko Miyagawa noticed Test::Builder was accidentally storing objects passed into test functions causing problems with tests relying on object destruction. - Added example of calculating the number of tests to Test::Tutorial - Peter Scott made the ending logic not fire on child processes when forking. * Test::Builder is once again ithread safe. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One kiloconway: unit of extreme mind expansion. Equal to 1024 conways, one kiloconway gives the sensation that all of the quantuum particles in your brain have been spontaneously converted into Mars bars (IN CONSTANT TIME!). The presumed existance of kiloconway scale events has led many to believe that higher forms of intelligence do in fact exist, and are living quite happily in Rural Australia. -- Ziggy
Re: And for the mod_perl folks... Test::Builder-reset
On Mon, Nov 11, 2002 at 09:05:13AM -0800, chromatic wrote:
Also, for those who aren't happy with the fact that Test::Builder is a hard
singleton with its state held in a bunch of file-scoped lexicals (hard to
debug) reset() made me collect together all those state variables in one
point in the source making it much easier someone to convert them into
object data, if they feel so inclined. I'm not yet convinced its a good
idea.
We *could* add a method called really_create_a_new_builder() that doesn't have
the singleton properties, but what problem does that solve? As long as we're
stuck with test numbers, we have to try not to confuse Test::Harness.
Little known fact: test numbers are optional.
I know people have grumbled before about there being no way around the
singleton property, I just can't remember why. I'm doing that a lot lately.
Maybe I should check my hospital receipt, make sure all they took out was my
appendix. What's this scar on the back of my head...?
Ok, here's one from me.
It would make Test::Builder easier to test itself for one. There's lots of
T::B tests which deliberately throw it off into some bizarre state or cause
lots of failures. These results must be pipped off somewhere and analyzed
seperately. For those tests, we're back to using a hand-rolled ok()
function.
With two T::B objects, we could beat on one object while leaving another in
a normal state to perform the tests.
The real reason why I put all the data into lexicals rather than a hash is
because its easier to type $Have_Plan than $self-{Have_Plan}.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
11. Every old idea will be proposed again with a different name and
a different presentation, regardless of whether it works.
-- RFC 1925
Re: And for the mod_perl folks... Test::Builder-reset
On Mon, Nov 11, 2002 at 02:51:00PM -0800, chromatic wrote:
On Monday 11 November 2002 14:40, Michael G Schwern wrote:
We *could* add a method called really_create_a_new_builder() that doesn't
have the singleton properties, but what problem does that solve? As long
as we're stuck with test numbers, we have to try not to confuse
Test::Harness.
Little known fact: test numbers are optional.
With the mandatory Test::Harness upgrade, yes.
No, they've always been optional. Don't try to pin this one on me! :)
$ cat ~/tmp/nonums.t
#!/usr/bin/perl -w
print 'TEST';
1..3
ok
ok
ok
TEST
$ perl5.4.0 -MTest::Harness -e 'runtests @ARGV' nonums.t
nonums..ok
All tests successful.
Files=1, Tests=3, 0 secs ( 0.00 cusr 0.02 csys = 0.02 cpu)
My point is that mixing. Test::Builder outputs is bad juju:
Test::Builder-new would remain as a singleton. We'd just provide an
alternate constructor to provide a second object if someone really wants it.
I know people have grumbled before about there being no way around the
singleton property, I just can't remember why. I'm doing that a lot
lately. Maybe I should check my hospital receipt, make sure all they took
out was my appendix. What's this scar on the back of my head...?
And why are you beeping?
I get Comedy Central if I stand on my head.
Ok, here's one from me.
It would make Test::Builder easier to test itself for one. There's lots of
T::B tests which deliberately throw it off into some bizarre state or cause
lots of failures. These results must be pipped off somewhere and analyzed
seperately. For those tests, we're back to using a hand-rolled ok()
function.
With two T::B objects, we could beat on one object while leaving another in
a normal state to perform the tests.
Simplifying the tests there is a *good* reason. They scare me.
I suppose we could also log only certain kinds of tests, too. That might make
some of my fiendish plans easier. (Log only can_ok(), in this T::B object.)
Hmmm... can_ok() isn't a T::B method. Ooops.
In theory you could write a T::B subclass which calls each method in a
wrapper to change the output. Something like:
package Test::Builder::Log::like;
sub like {
my($self) = shift;
my $old_output_fh = $self-output();
my $old_fail_fh = $self-failure_output();
my $old_todo_fh = $self-todo_output();
$self-output('some_output.log');
$self-failure_output('some_output.log');
$self-todo_ouptut('some_output.log');
local $Test::Builder::Level = $Test::Builder::Level + 1;
my $ret = $self-SUPER::like(@_);
$self-output($old_output_fh);
$self-failure_output($old_fail_fh);
$self-todo_output($old_todo_fh);
return $ret;
}
Something like that, obviously made more generic hand wave.
Hmmm. All those *output calls are really irritating. I think I'll change
it to something like this instead:
package Test::Builder::Log::like;
my %outputs = map { ($_ = 'some_output.log') } qw(test failure todo);
sub like {
my($self) = shift;
# Add outputs() to combine all three *output methods
# together. Also, have it return the former set of
# filehandles.
my %old_fhs = $self-outputs(%outputs);
local $Test::Builder::Level = $Test::Builder::Level + 1;
my $ret = $self-SUPER::like(@_);
$self-outputs(%old_fhs);
return $ret;
}
Maybe we're too coarsely grained on the singletonness. It's only the test
counters and outputs that really need to be single, right? We could make
Test::Builder::Counter and Test::Builder::Output as singletons and let
Test::Builder use those by default. For the tests, we could mock them or
override them, or whatever we find necessary.
I think its more complicated than that. Depends on how its being used.
Right now my use would be to effectively have two independent tests running
in parallel in the same process. One test being the beating which is
testing Test::Builder and the results are trapped, the other test doing the
testing of the first.
In this case, you don't even want the counter and output as singletons. You
want complete seperation. Two tests running in parallel without any
interference.
I can't think of why you'd want two T::B objects which are only partially
sharing state.
Serious suggestion.
The real reason why I put all the data into lexicals rather than a hash is
because its easier to type $Have_Plan than $self-{Have_Plan}.
Hmm, perhaps a source filter that allows:
.$Have_Plan
It's only one character longer...
Is it .$Have_Plan this week? I thought
Re: And for the mod_perl folks... Test::Builder-reset
On Tue, Nov 12, 2002 at 01:31:43AM +, Mark Fowler wrote:
Test::Builder-new would remain as a singleton. We'd just provide an
alternate constructor to provide a second object if someone really wants it.
You know, that would at the very least tidy up Test::Builder::Tester
somewhat. It's currently just a bit of a hack...(sssh, I didn't really
say that)
That's two votes.
With two T::B objects, we could beat on one object while leaving
another in a normal state to perform the tests.
One day (maybe not this rewrite, maybe not the next) I'd like to see you
be able to write layers of tests. That is to say that you have some kind
of Test::Harness like system running actually inside a Test::Builder test
that runs another set of tests and returns ok or not ok if all *its*
tests returned ok or not. And then you could have subtests within that,
down to an arbitary level of abstraction. Does this make sense?
Test::Harness::Straps-analyze* already do that if I understand you
correctly.
Not that I'm one to break OO boundaries, but by allowing of breaking the
OO boundaries just a little bit by skipping the accessors you could just
do this, which means you are less likey to shoot yourself in the foot with
forgeting to reset the outputs back again.
package Test::Builder::Log::like;
my %outputs = map { ($_ = 'some_output.log') } qw(test failure todo);
sub like {
my ($self) = shift;
local $self-{outputs} = \%outputs;
local $Test::Builder::Level = $Test::Builder::Level + 1;
return $self-SUPER::like(@_)
}
LamentOh, to have real end-of-scope conditions./Lament
The real reason why I put all the data into lexicals rather than a hash is
because its easier to type $Have_Plan than $self-{Have_Plan}.
http://magnonel.guild.net/~schwern/talks/Refactoring/slides/slide015.html
;-)
The nice part about refactorings is they're reversable.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
That which stirs me, stirs everything.
-- Squonk Opera, Spoon
Test::Builder-level
At YAPC::Europe there was some discussion about Test::Builder-level, $Test::Builder::Level and the fact that they don't really work well as implemented. I know we reached some sort of consensus about how to do it better, but I've forgotten it. Anyone remember? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One We're talkin' to you, weaselnuts. http://www.goats.com/archive/000831.html
Upcoming change to Test::Simple/Builder/More wrt threads
In 0.48 the behavior of Test::Simple/More/Builder with regard to threads will change. Previously, if you were using a perl = 5.8.0 and have ithreads compiled, loading Test::More would load threads and threads::shared. This was to avoid the problem of a user doing this: use Test::More; use threads; and then wondering why the test output was confused. Test::More must know threads are enabled when it compiles so it can share the appropriate variables. Unfortunately, this makes it impossible to use Test::More or any Test::Builder derived module to explicitly test code with threads off on an ithread enabled Perl. I discovered this problem when I tried to test threads::shared without threads.pm loaded. Couldn't do it. Had to use Test.pm. As the current attempt at DWIM leaves Test::Builder unable to test a certain class of code, it must be changed. It will be necessary to load threads.pm before Test::More or Test::Builder if you wish to use threads in your tests. use threads; use Test::More; -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One GOD made us funky!
Re: callbacks at the end of Test::More ?
On Sat, Oct 26, 2002 at 04:22:49PM +0100, Nicholas Clark wrote:
However, I'd like to be able to cleanly print out my random number seed to
STDERR (or whatever Test::Builder's correct name for this is) if it believes
that any tests have failed, and I can't see a clean way to do this.
In an END block, grab the T::B object and check the summary().
#!/usr/bin/perl -w
use Test::More tests = 1;
pass('foo');
END {
my $failed = grep!$_, Test::More-builder-summary;
diag $failed ? Failed\n : Passed\n;
}
The only trouble there is if the test as a whole fails during the ending,
like because the wrong number of tests were run or the exit code was odd,
the above logic is too simple to know that. You have to go through the
sort of logic _ending() does.
Not sure to add in an ending callback or an is_passing() method. Or both.
BTW How does one get the current srand if you use the one set by Perl?
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
I'm not actually Kevin Lenzo, but I play him on TV.
Re: Some upcoming changes to Test::Builder
On Tue, Oct 15, 2002 at 10:34:26AM +0100, Nicholas Clark wrote:
On Mon, Oct 14, 2002 at 09:00:42PM -0400, Michael G Schwern wrote:
5.8.0's threads are giving me serious headaches. When 5.8.1 comes out I
might drop support for 5.8.0's threads just so I can remove a large volume
of work-around code.
Leaving support for 5.005 threads in? I'm confused.
Test::Builder has never supported 5.005 threads. At least, I haven't done
anything special to make it work.
Or do you mean dropping all automatic threading support?
What I mean is in order to make Test::Builder work with 5.8.0's ithreads I
had to hack around a whole lot of bugs and write some really awful code.
Most of it having to do with automatic array/hash extensions not being
shared. For example:
# 5.8.0 threads bug. Shared arrays will not be auto-extended
# by a slice. Worse, we have to fill in every entry else
# we'll get an Invalid value for shared scalar error
for my $idx ($#Test_Results..$Expected_Tests-1) {
my %empty_result = ();
share(%empty_result);
$Test_Results[$idx] = \%empty_result
unless defined $Test_Results[$idx];
}
That whole block is just a work around for a 5.8.0 ithreads bug. There's
more like it to make Test::Builder work transparently with ithreads. Here's
another:
my %result;
share(%result);
%result = (
'ok' = 1,
actual_ok = 1,
name = '',
type = 'skip',
reason= $why,
);
$Test_Results[$Curr_Test-1] = \%result;
which was originally written:
$Test_Results[$Curr_Test-1] = {
'ok' = 1,
actual_ok = 1,
name = '',
type = 'skip',
reason= $why,
};
and could not be written:
$Test_Results[$Curr_Test-1] = share {
'ok' = 1,
actual_ok = 1,
name = '',
type = 'skip',
reason= $why,
};
because threads::shared::share() is broken when threads are disabled.
Really irritating stuff. Every release since 0.45 has added more ithread
work arounds.
So when 5.8.1 comes out and ithreads work better I plan on dropping support
for 5.8.0's still broken ithreads like a hot potato so I can remove all that
hackery. 5.8.1's less buggy ithreads will be supported, hopefully with a
minimum of hackery. Users will simply have to upgrade, which they really
should do anyway if they're using threads.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
I'm spanking my yacht.
Re: Test::Class - comments wanted
On Sun, Oct 13, 2002 at 04:01:10PM -0700, David Wheeler wrote: On Sunday, October 13, 2002, at 10:05 AM, Tony Bowden wrote: Makes it simpler for people who prefer the 'no_plan' style of testing Maybe this is what I just don't get. I'm not one of those people, so I don't really understand why people might prefer it. Especially here where there's such a natural way to specify them, and you're only counting them per method, rather than over the entire test. I have to agree with Tony. I think it's important to explicitly indicate the number of tests that a given method runs, and to be explicit about saying when you're not sure how many tests there will be. The reason I went with no_plan in Test::Inline was that unlike a dedicated test script, a T::I test is cobbled together from a series of seperated blocks of tests and it's more difficult than usual to count them all and add them up. Adds another step to writing more tests. Test::Class shares that cobbled together nature, but if it has a way to specify them on a per-block basis, that will work without much trouble. OTOH, my thinking recently is that the explicit plan has become obsolescent. [1] The explicit plan protects against: 1. Your test dying. 2. Your test not printing tests to STDOUT 3. Exiting early via exit(). #1 and #2 are protected by other implicit mechanisms in Test::Builder. #3 will be protected against by overriding CORE::exit in a future version. For the negatives to the explicit plan: 1. Makes writing SKIP blocks more difficult 2. Makes writing complex, iterative/looping tests more difficult Of course, no_plan requires Test::Harness to be upgraded which has been causing some problems lately. Something I'm moving to fix by making the Test::Harness upgrade non-optional when you install Test::More. But that's an implementation issue and will eventually fade away. [1] This thinking makes me nervous, so I'm open to someone convincing me otherwise. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Here's hoping you don't become a robot!
Some upcoming changes to Test::Builder
I'm working on Test::Builder 0.48 and here's some relatively major changes I'm putting in. Some advance warning and a chance for folks to convince me otherwise. * Test::Harness will no longer be optional. Currently, the tests do not require a Test::Harness upgrade. They simply skip over anything that requries T::H 2.0. This has caused a lot of problems lately with botched and shadowed T::H installs and people installing Test::More without realizing T::H wasn't really upgraded and tests which have 'no_plan' not working. The real reason I made the T::H upgrade optional was it made it easier to test against older perls. I could just run the test suite against, for example, a clean 5.5.3 without having to dirty the installation by installing T::H 2.0. I've figured a way around this with MakeMaker hackery. A future version of Test::Harness will include a post-install check to make sure it actually got installed properly and isn't being shadowed, so it will help the problem from that end. * threads will no longer be loaded by default. Currently, Test::Builder loads threads.pm if available. It was done this way because otherwise one would have to remember to load threads before Test::More (or other T::B based module) in order for it to work properly. Unfortunately, this makes it impossible to test unthreaded code with Test::More when perl is built with threads as I recently found out while trying to test threads::shared in its disabled state. As I'm going for maximum flexibility, and loading threads.pm by default makes a whole class of tests impossible, threads.pm will no longer be loaded by default. 5.8.0's threads are giving me serious headaches. When 5.8.1 comes out I might drop support for 5.8.0's threads just so I can remove a large volume of work-around code. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One List context isn't dangerous. Misquoting Gibson is dangerous. -- Ziggy
Re: [ Memory ] Re: Thought
On Thu, Oct 03, 2002 at 10:43:43AM +0100, Nicholas Clark wrote: The reason I'm saying it might not be much use to people unfamiliar with the internals of unix is precisely because it does return a precisely defined but not directly useful value - the highest address on the heap. If you read it, call perl code that creates a known perl structure, and read it again, doesn't go up directly and exactly related to the amount of memory that structure needed. Depending on how much other memory there is free, it may not go up at all, or it may go up more if the allocation system grabs a big block of free space. Ok, now I'm really confused. - sbrk() returns, effectively, the size of the heap. - the heap may not go up when you create a perl structure but MemUsed() is shown to be used like this: use Devel::Internals; my @array = (1..1); MemUsed creation of array; my @dup = @array; MemUsed duplication of array; and MemUsed, in scalar context, returns the sbrk value, the size of the heap. So I presume in the above code it simply returns the difference between the sbrk value between the two calls. But Nick just said this isn't actually the memory used, it's just the different in heap size. But the function is called MemUsed! Brane hurt! A lot of the problem here is the expectations from the names. - MemUsed() implies we're going to be able to figure out the amount of memory used by a hunk of Perl code without knowing a lot of magic. Instead, we get something about heap sizes. - sbrk() implies dark memory magic. Most people know it's just some memory thing beyond free() and malloc(). When they see it, they get scared, and rightfully so apparently. The problem is when you put those two next to each other, one promising a friendly interface, one a bare-metal interface, it confuses the intent of the module. Is it for Joe End User or is it for Joe Core Hacker? It would seem this module is for Joe Core Hacker, and it's all about sbrk(). So let's make that clear in the names: NAME Devel::sbrk - Access sbrk values SYNOPSIS use Devel::sbrk; my $end = sbrk (); my @array = (1..1); print The creation of this array used , sbrk () - $end, bytes\n; # Easier use Devel::sbrk; my @array = (1..1); sbrkChange creation of array; my @dup = @array; sbrkChange duplication of array; Ironically, I'm now advocating sbrk() as a name. :) Nick is right, if sbrk() is something special, call it that. With this interface, nobody's going to complain that sbrkChange() didn't actually return the amount of memory used by my @dup = @array; You might want to generalize the concept a bit to a more general heap size module. That way on systems where you don't have sbrk but can query the heap size it will still make sense. I don't know if they are quite the same thing, so I just leave it as a suggestion. NAME Devel::Memory::Heap - Access the size of the memory heap SYNOPSIS use Devel::Memory::Heap; my $end = HeapSize (); my @array = (1..1); print The creation of this array used , HeapSize () - $end, bytes\n; # Easier use Devel::Memory::Heap; my @array = (1..1); HeapChange creation of array; my @dup = @array; HeapChange duplication of array; A more broad Devel::Internals or a more user-friendly Devel::Memory can be left for later, and the functionality of Devel::Memory::Heap can be put into it if it makes sense. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Here's some scholarly-ass opinions...
Re: Test::More
On Tue, Sep 24, 2002 at 05:57:41PM +0200, H.Merijn Brand wrote: 1. use_ok should have an entry in the manual for minimal version use_ok (Test::More, 0.47); This currently doesn't work quite right. Observe... $ perl -MTest::More -wle 'plan tests = 1; use_ok(Text::Soundex, 0.20)' 1..1 not ok 1 - use Text::Soundex; # Failed test (-e at line 1) # Tried to use 'Text::Soundex'. # Error: Import directive '0.2' was not recognized at (eval 1) line 3 # Looks like you failed 1 tests of 1. The trouble is use_ok does this: require Text::Soundex; Text::Soundex-import(0.20); this is different than use Text::Soundex 0.20. In the latter, Perl handles the version check itself. But in the former, Text::Soundex's import routine does the work. Normally, Exporter will emulate the version check but if you're not using Exporter, as Text::Soundex does not, anything could happen. For similar reasons, Test::More can't check itself. $ perl -MTest::More -wle 'plan tests = 1; use_ok(Test::More, 0.40)' 1..1 not ok 1 - use Test::More; # Failed test (-e at line 1) # Tried to use 'Test::More'. # Error: You tried to plan twice! Second plan at (eval 1) line 3 # Looks like you failed 1 tests of 1. schwern@blackrider:~/src/devel/Test-Simple$ I'll have to put in some special case code to make it work right. 2. I'm testing conversions to and from Unicode Unicode... wasn't he the bad guy in the Transformers Movie? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Kids - don't try this at--oh, hell, go ahead, give it a whirl...
Re: Test::More::can_ok when using AutoLoader
On Sat, Aug 31, 2002 at 02:05:53PM +0200, Elizabeth Mattijsen wrote:
$ perl -Mblib -wle 'use AutoExample; print Yes if
AutoExample-can(foo)'
Using /home/schwern/tmp/AutoExample/blib
Yes
Hmmm... I'm doing BEGIN { use_ok( 'Thread::Pool' ) }... Maybe there is a
difference there...
Hmmm. Shouldn't be a difference.
Ok, so it _should_ work. I'll see if I can boil this down and create a
bugreport if I can.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Out of ammunition. God save the King.
Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])
On Mon, Sep 02, 2002 at 08:13:06AM +0100, Nick Ing-Simmons wrote: I understand all that. My point was that while test itself may care where it is run, blib.pm does not mind as much. Also blib.pm's job is to make running an un-installed module easy which is what you want to do for a pre-install test. So if there is boiler-plate stuff to do blib.pm is an appropriate place to do it. Thus is you need to be in t this might suit cd t; perl -Mblib=lib foo/bar.t would work, and I would be more than happy it it looked for a t directory as well. There's a chicken/egg problem here. blib.pm is going to find the lib/ directory so we can load uninstalled modules. So... how do you find blib.pm? I believe the above worked for you because you happen to have an already installed perl in the same spot as the new one is going to go. If you move that out of the way, no good. Other problems include that we can't assume blib.pm works. Don't want the whole basic test suite falling apart because of a module failure. Finally, since you have to run from t/ anyway, blib.pm is overkill. This works just as well: cd t; ./perl -I../lib foo/bar.t Anyhow, t/TestInit.pm *already does this*. schwern@blackrider:/usr/local/src/perl-current$ ./perl -It -MTestInit -wle 'print @INC; use Cwd; print cwd;' .../lib /usr/local/src/perl-current/t While the result is still ugly, it means we can expand and alter the requirements for running a core test. For example, the PERL_CORE environment variable should be set (t/TestInit.pm currently doesn't). So the full command is really something like this: cd t; PERL_CORE=1 ./perl -I../lib path/to/test.t Which could be reduced somewhat to: ./perl -It -MTestInit t/path/to/test.t and perhaps reduced a little further by moving/linking TestInit.pm into the top level directory. ./perl -MTestInit t/path/to/test.t but that will cause trouble when running tests with -T (because . is no longer in the path). When tests are run using t/TEST (ie. make test) are run with TestInit. So strictly speaking the BEGIN block is redundant. t/harness (ie. make test_harness) is currently not using TestInit. There's currently a bug where $Test::Harness::switches is not honored. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One MERV GRIFFIN!
Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])
On Tue, Sep 03, 2002 at 11:26:01AM +1000, Ken Williams wrote: Thus is you need to be in t this might suit cd t; perl -Mblib=lib foo/bar.t I'd be happier if that 'cd t;' happened inside blib.pm as 'chdir t'. It already happens inside TestInit. I'd be happier still if tests didn't run from t/, since that's not how most CPAN modules work. The trouble here is we don't want tests running from the top level. Too much chance of them overwriting something they shouldn't be, or writing a file and forgetting to clean it up. Or touching the wrong directory, etc... So t/ is used as a safer location. It's also used just because that's what all the other tests do, so there's some cargo-cult/needless tradition. So they can't run from the top level directory, and at that point you're already different than CPAN. However, we can chdir for them, that's not hard. There's a chicken/egg problem here. blib.pm is going to find the lib/ directory so we can load uninstalled modules. So... how do you find blib.pm? This doesn't seem like a real problem. The test suite can find it because it knows where it's being run from. It can set up @INC however it wants, to make sure blib.pm is found. Then blib.pm sets things up for the rest of the tests. I thought the whole idea was to run blib.pm to set @INC. If the test suite is setting up @INC we don't need blib. And if we're not using blib to setup @INC there's no point in shoving extra functionality unrelated to finding modules into it. Put it into something else. Like TestInit.pm, which already has all this. Other problems include that we can't assume blib.pm works. Don't want the whole basic test suite falling apart because of a module failure. For core perl, we can indeed assume blib.pm works. If it doesn't, we want to know about it right away. You want to be able to continue with basic testing even in the event of a catastrophic failure. There was a big discussion along these lines about at what point in the tests we can start relying on modules, this was about using things like Test and Test::More in basic core tests. The result was that 'make test' should rely on as little as possible and I agree with that. Worse, blib relies on Cwd.pm and File::Spec. If any of these fail the test suite doesn't work at all. No good. Finally, since you have to run from t/ anyway, blib.pm is overkill. This works just as well: cd t; ./perl -I../lib foo/bar.t I have to wonder why it's preferable to put code on command lines rather than in modules. As mentioned below, it's not. That's why it's already in TestInit.pm (are you sensing a theme here?) I was just spelling out the equivalent command to show blib wasn't buying us anything. Really, it would be nice if you could tell the module about certain situations, like -MTestInit=core or whatever, and that would do whatever's necessary for a core test run (including chdir, @INC adjustments, etc.). TestInit is a module only available and useful in the core for doing the special little things necessary to run a CPAN-style test as a core test. There wouldn't be anything to do otherwise outside the core. Heck: with a smart enough TestInit, it could even adjust @ARGV so that you could specify all *.t paths in Unix format, if that would be helpful. Hmmm. I can't think of a use. and perhaps reduced a little further by moving/linking TestInit.pm into the top level directory. ./perl -MTestInit t/path/to/test.t but that will cause trouble when running tests with -T (because . is no longer in the path). '.' won't necessarily be in the path anyway. Sorry, I ment perl's @INC. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Not king yet
Re: Looking for tool like pydoc to document perl cgi scripts
On Mon, Sep 02, 2002 at 01:01:24PM -0400, Jeff Kowalczyk wrote: Is there an application for perl cgi scripts that will generate formatted HTML documentation similar to what I can get with pydoc? http://web.lfw.org/python/pydoc.html POD perldoc. See the perldoc and perlpod man pages. It can produce HTML and lots of other formats. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Still not king
Re: Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])
On Tue, Sep 03, 2002 at 03:34:39PM +1000, Ken Williams wrote: I thought the whole idea was to run blib.pm to set @INC. If the test suite is setting up @INC we don't need blib. blib.pm only adds to @INC. Seems like the core requires something to strip @INC down to a small set, not add to it. @INC only contains paths to where the libraries will be installed. Until installation happens, perl can't find it's own not-yet-installed libraries without help. We remove all other entries from @INC so that previously installed modules don't interfere with testing the new versions. Anyway, what I meant was that @INC needs to be one value to find blib.pm, and then blib.pm changes @INC to be another value, so there's no chicken/egg problem. There is no other value to be changed to. All you need to do is find lib/ '.' won't necessarily be in the path anyway. Sorry, I ment perl's @INC. That's what I meant too. People can build Perl without '.' in @INC, right? I don't think so. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One If your module passes test You're one of the very best Don't fuck up the MANIFEST Burma-Shave - ignatz
Eliminating the core-test BEGIN block (was Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting])
On Fri, Aug 30, 2002 at 05:54:15PM +1000, Ken Williams wrote: Oh, one big lib/, not several different ones? So then why can't it be run with perl -Mblib=lib t/foo/bar.t ? Because tests all try to run from t/. That's what the chdir 't' is for. t/TEST does this for you and so do the tests themselves which makes it much more forgiving. Otherwise it would be very picky and this: $ cd /path/to/src/perl $ ./perl -Ilib t/foo/bar.t wouldn't work when you ran a test manually. You'd have to do: $ cd /path/to/src/perl/t $ ./perl -I../lib foo/bar.t and always remember to run from t/. I'm just trying to push the decision into a more appropriate place, though I guess in the end it's not such a big deal. This is already effectively done by the TestInit module but nobody trusts it. t/TEST will run each test with -MTestInit. t/harness does not so that will have to be somehow fixed. I usually do exactly that when I run them manually: ./perl -Ilib lib/Foo/whatever.t. but that -Ilib isn't really necessary since the test does it. If we remove the BEGIN block code and place that elsewhere things get more complicated when trying to run an individual test: ./perl -It -MTestInit lib/Foo/whatever.t or ./perl -I. -MTestInit ../lib/Foo/whatever.t What it boils down to is how to you remove the need for the cargo-cult BEGIN block in each test while still making it easy to run individual tests by hand? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Tasty, yet morally ambiguous.
Re: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]
On Fri, Aug 30, 2002 at 06:25:27PM +1000, Ken Williams wrote: This is because I've been frustrated a number of times by modules that died on CPAN when they got integrated into the core, but needed bugs to be fixed, but wouldn't see new versions until perl got maintenance releases... All that really requires is someone to pick up the torch and release new versions to CPAN. And keep the CPAN versions and perl versions in sync. As much as they're supposed to, p5p doesn't treat the CPAN version as the canonical one and always forgets to report patches back to the author. Keeping them in sync with p5p's patches is the hardest part. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One There is a disurbing lack of PASTE ENEMA on the internet.
Re: Thought
On Fri, Aug 30, 2002 at 10:30:22AM +0200, H.Merijn Brand wrote: I was thinking about highjacking a standard function: read () Giving read() semantics completely unrelated to reading a filehandle would be a bad choice of syntax. So would making it a keyword. Could you do it (mostly) via XS? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Do you have a map? Because I keep getting lost in your armpits.
Fwd: [ken@mathforum.org: Re: [Inline 0.43] insecure dependency when tainting]
There's already the beginnings of something like this in t/TestInit.pm, but
it wasn't updated for the PERL_CORE trick. Nor do I know if it can be
entirely trusted.
Also, Nick's example is a little odd. You usually don't want '.' (ie. t/)
in your @INC. It's more like this:
BEGIN {
if($ENV{PERL_CORE}) {
chdir 't';
@INC = '../lib';
}
}
but in some cases you need to include something more. For example,
MakeMaker does this:
BEGIN {
if($ENV{PERL_CORE}) {
chdir 't';
@INC = ('../lib', 'lib');
}
else {
unshift @INC, 't/lib';
}
}
so it can see specialty helper modules in t/lib/.
- Forwarded message from Ken Williams [EMAIL PROTECTED] -
From: Ken Williams [EMAIL PROTECTED]
Date: Thu, 29 Aug 2002 14:08:00 +1000
To: Nicholas Clark [EMAIL PROTECTED]
Subject: Re: [Inline 0.43] insecure dependency when tainting
On Thursday, August 29, 2002, at 06:51 AM, Nicholas Clark wrote:
You'll often see regression tests in the core start like this:
sub BEGIN {
if ($ENV{PERL_CORE}){
chdir('t') if -d 't';
@INC = ('.', '../lib');
} else {
unshift @INC, 't';
}
if $ENV{PERL_CORE} is set, then the test knows that it's being
run as part
of a core build, and so it should force @INC to only find uninstalled
modules in the core tree.
Seems like it would be nicer to set that externally, rather than
in every test file. Maybe some code like this could be added to
blib.pm or something? Then one could do something like this:
perl -Mblib=only t/foo.t
Sorry if I've wasted your time by telling you something you
already knew,
or had inferred from Hugo's reply.
I might have inferred it, but this helped me understand various
implications, actually.
-Ken
- End forwarded message -
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Try explaining that to my brain which is currently ON FIRE!
http://www.goats.com/archive/010704.html
Re: Help spreading Test
On Wed, Aug 28, 2002 at 09:32:10PM -0700, chromatic wrote: Perhaps we are looking at this from the wrong direction. Instead of installing bundled modules, perhaps a shell should be able to specify that certain dependencies are available only for testing. That way, users wouldn't necessarily have to install Test::Builder and Test::Builder::Tester and so forth in the public directories. I think David hit the nail right on the head. Given what the Test::* modules are for, I think that this is common. And since they're small and won't waste much in the way of resources once they've been installed, I personally think it's no big deal to install them. Just install the things. They're small and you only have to do it once. A CPAN shell will automate the process so it's rather moot. And if you're not using a CPAN shell every module install is going to be a chore anyway. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One I blame myself. AND SATAN.
Re: [PATCH] Test::Builder holds object inside
On Thu, Aug 22, 2002 at 10:29:25PM +0900, Tatsuhiko Miyagawa wrote: Here's a patch for Test::(Simple|More) to allow idioms like my $obj = Foo-new; ok($obj, 'object is defined'); Without this patch, passed $obj was referenced in Test::Builder, so REFCNT doesn't come to 0 before global cleanup. Whoa, nice catch! And thanks for catching the missing t/has_plan's in the MANIFEST. :) -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One gigaconway: a hypothetical unit of mind expansion, so disturbing it is likely to change the fundemental nature of the universe. Some contend that gigaconway events, while rare, are much cheaper to produce than antiprotons, nuclear weapons or even XML specifications, and start at US$60,000 each. If you believe gigaconway events are indeed possible, please send your tax deductable contributions to: The Conway Fund, c/o Yet Another Society http://www.yetanother.org/ -- Ziggy
Re: Add Test::Harness 2.x Prerequisite to Test::Simple?
On Sat, Aug 24, 2002 at 03:08:08PM -0700, chromatic wrote: I've been using Test::Exception on a project and am very glad to have it. I ran into a small issue trying to install it, though: it has TODO tests, but those failed as the existing version of Test::Harness (1.26 or so) did not understand them. Since TODO tests are a feature of Test::More, perhaps it would be correct to mark Test::Harness 2.x as a prerequisite for the Test::Simple bundle. I almost sent Adrian a patch doing this for Test::Exception, but it seems that Test::Simple is the best place to handle this. Test::Harness 2.03 is a prerequsite of Test::More. Either it didn't get installed for some reason or your old version is shadowing. T::H somewhere around 1.2x installed itself into site_perl, not the core, so you might have a leftover. Try a make install UNINST=1. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One In my day, we didn't have none of yer new-fangled token-pasting operators! We stuck the tokens together with our BARE HANDS and we LIKED IT. -- Mark-Jason Dominus fondly remembers perl 1.0 in [EMAIL PROTECTED]
Re: [PATCH MakeMaker.pm] Add documentation for ExtUtils::MakeMaker::prompt()
On Thu, Aug 22, 2002 at 06:55:46AM -0500, Dave Rolsky wrote: +If Cprompt() detects that it is not running in interactively (say, +if it is running from a CPAN shell), or if the PERL_MM_USE_DEFAULT +environment variable is set to true, the $default will be used without +prompting. This prevents automated processes from blocking on user +input. In my experience, it still prompts under the CPAN shell, but not CPANPLUS. Ok, I'll just pull out the reference to the CPAN shells and just say if it's not running interactively. Let the CPAN shell authors figure it out. They should probably have a configuration option. I've actually come to rely on the former behavior and was surprised that CPANPLUS didn't work the same way, causing it to be basically imposssible to do a fresh install of Alzabo via CPANPLUS. How about a way to _force_ it to prompt or give up if that's really impossible? A timeout argument to prompt() might be worthwhile. And an accompanying environment variable. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One I've just gone through a lung-crushing breakup with my blender and I don't think I should screw my forehead alone tonight.
[ANNOUNCE] Test::Simple 0.47
http://www.pobox.com/~schwern/src/Test-Simple-0.47.tar.gz Two major bugs were added in 0.46. Objects would accidentally be stored internally causing destructors not to fire when people expected them to. Also ithread support was broken. These have both been fixed. 0.47 Mon Aug 26 03:54:22 PDT 2002 * Tatsuhiko Miyagawa noticed Test::Builder was accidentally storing objects passed into test functions causing problems with tests relying on object destruction. - Added example of calculating the number of tests to Test::Tutorial - Peter Scott made the ending logic not fire on child processes when forking. * Test::Builder is once again ithread safe. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One I do have a cause though. It is obscenity. I'm for it. -- Tom Lehrer Smut
Re: Testing POSIX locale support
On Mon, Aug 26, 2002 at 04:05:26PM -0400, John Peacock wrote: Specifically, in order to test that the locale stuff is working, I need to have two different locales installed to switch between. Currently, I am using en_US and en_GB, but obviously that ignores most of the planet. I was hoping to get some suggestion for how I can go about being sensitive about how I do these tests, so that I get as few misleading failures as possible. One thought I had was to test for a series of locales during the Makefile.PL processing, then update the t/test.t file for some combination of the installed locales. The problem I have already identified is that `locale -a` may report erroneous information (partially installed locales), so I am going to have to poke around some more. Perhaps you can run a small suite of checks against things you find in the locale list to be reasonably sure the locale is valid and fully installed and pick two which seem to be so. In particular, I know I am going to have to skip that testing entirely on platforms without any locale support (like CygWin), but I am still going to have some issues where I may only have a single locale fully installed to test. I need something like the opposite of TODO, so that I can have failures but warn that those failures are acceptable with caveats. I really need 'ok', 'not ok', and 'not ok but not to worry too much' so I can handle all the various possibilities. If you're going to have a test which suggests you simply ignore the failure then just don't bother running the test. A success tells you it's working but a failure doesn't tell you it's broken, so it's not worth much. What you should probably do is emit further diagnostic information on what the test failure might mean like: is($this, $that) || diag(MSG); Please check that your $locale_i_was_testing_with locale is fully installed and try again. Otherwise, contact the author. MSG -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Monkey tennis
Thoughts from TPC: Testing perlguts with Inline::C
More chronicles of brainstorming from TPC. Last year at TPC, Hugo asked for ideas about testing the internal C functions of Perl. Testing perl itself is ok, but if we can test at an even finer grained level of the C API many bugs can be more easily caught. Additionally the documentation will be improved in the process (can't test it if you don't know what it's supposed to do) and the XS mechanism might also be improved as more people have to touch it to test it. At the time, there was nothing to do this except write tests in C, and that sucks. It's hard enough to find people to write tests in Perl. Then Ingy comes along with Inline::C. One of the tricks he pulled off at YAPC::Europe was writing a simple Inline module which exposed all the internal C functions as perl functions. AH HA! Now we can test C functions just like any other Perl function. So, this is a request to place Inline::C into 5.9 to facilitate easy testing of the C API. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One IMHO bugs in Perl 5 shouldn't carry over to Perl 6. (Unless, of course, we *like* the bugs... ;) -- Ken Fox in [EMAIL PROTECTED]
Re: [perl #15479] perl 5.8.0 segfault
On Wed, Jul 31, 2002 at 12:30:12PM +0100, Nicholas Clark wrote: On Wed, Jul 31, 2002 at 01:20:25PM +0200, Rafael Garcia-Suarez wrote: Wasn't there a .t file to run separate perl interpreters and test for core dumps ? We're dissolving that. Use fresh_perl_is() and fresh_perl_like() from t/test.pl and place the test in the appropriate group. So somewhere with other filehandle tests. I keep forgetting that I need to remember to ask this. Is there a FAQ for regression test writing? Well, an guide to so I want to write a regression test, explaining how to do it, how perl5's tests are structured to reduce interdependencies, use Test::More; when Test::More is not appropriate.. Porting/patching.pod About the only thing that's missing is docs for t/test.pl. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One We have returned to claim the pyramids.
Re: [PATCH Test::Builder] Add diagnostic messages to details
On Sat, Jul 27, 2002 at 09:44:03AM -0700, chromatic wrote: This patch captures messages sent through diag() and stores them in the diagnostic array. Now all of the information the tests generate is available for later inspection. Dude, where's my patch? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Me? A robot? That's rediculous! For one thing, that doesn't compute at all!
Re: Help spreading Test
On Sat, Jul 27, 2002 at 09:55:38AM -0700, chromatic wrote: One of the problems I have with using Test::Builder is that I want to distribute packages to systems that do not (necessarily) have a decent version of Test::* installed. Now it is easy to include a copy of a suitable version of Test::Builder with the package (provided it is not too big). Would it be a good idea to add a provision to Test::Builder that can be called, from the Makefile.PL, to display a message like this: The verificiation tests for this package require the Test::Builder package of at least version X.Y. You do not seem to have this installed. I have included a copy of Test::Builder in this distribution that I can use for testing. Do you want me to install this version of Test::Builder as well? This would help spreading the good stuff. I'm not sure of the context here, but at this point of complexity it's probably easier to just make it a normal prerequisite. Why? o If you're not using a CPAN shell, every module installation involving any prerequisites is going to be a chore. Adding one more to that pile isn't really going to make much difference. o At least for Test::Simple/More/Builder, so many modules already have dependencies on them (including CPAN, CPANPLUS and Storable) that you probably already have them installed. There's a slide in the Test::Tutorial talk with a list current as of a month ago. o If you don't already have them installed, you only need to do it once. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One 5. It is always possible to aglutenate multiple separate problems into a single complex interdependent solution. In most cases this is a bad idea. -- RFC 1925
Slides from OSCON
Figured folks might be interested in slides from talks at OSCON: http://www.pobox.com/~schwern/talks/How_To_Be_Lazy_Redux/ http://www.pobox.com/~schwern/talks/Test_Tutorial/ http://www.pobox.com/~schwern/talks/Writing_A_Test_Library/ -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Home of da bomb
Re: use_ok w/version numbers
On Mon, Jul 01, 2002 at 09:57:24AM -0500, Andy Lester wrote: I can do this: use PHP::Session 0.10; to ensure a version number, but I can't do this: use_ok( 'PHP::Session', '0.10' ); because I get this error: alester@flr4[~/tw/Dev]$ more Modules.t #!/usr/bin/perl -w use strict; use Test::More tests = 1; use_ok( 'PHP::Session', '0.10' ); alester@flr4[~/tw/Dev]$ perl Modules.t 1..1 not ok 1 - use PHP::Session; # Failed test (Modules.t at line 6) # Tried to use 'PHP::Session'. # Error: Can't locate object method require_version via package PHP::Session (perhaps you forgot to load PHP::Session?) at /usr/local/lib/perl5/5.6.1/Exporter/Heavy.pm line 105. # Looks like you failed 1 tests of 1. Before I go digging into a patch, is this something that we even want to allow? I know I'd like to allow it, because I'd like to have one .t file that ensures that I have proper module prereqs for my entire source tree. This is due to a quirk in the way use Foo VERSION is handled. use PHP::Session 0.10; this is handled directly by Perl itself and uses UNIVERSAL::VERSION to get $PHP::Session::VERSION. require PHP::Session; PHP::Session-import(0.10); In this case, PHP::Session::import would be used, or an inherited method, usually Exporter::import. But PHP::Session doesn't inherit from Exporter, yet we're winding up in there anyway. The problem is, using PHP::Session is defining a UNIVERSAL::import somehow. DB1 x defined UNIVERSAL::import 0 '' DB2 use PHP::Session DB3 x defined UNIVERSAL::import 0 1 PHP::Session uses UNIVERSAL::require which has to use UNIVERSAL.pm for an obscure ambiguity warning [1]. UNIVERSAL.pm does require Exporter; *import = \Exporter::import; so all classes suddenly have an import method, but they never inherited from Exporter! So Bad Things happen. The fact that UNIVERSAL.pm has an import method is a historical accident. It's only limited to situations when UNIVERSAL.pm is loaded. Since use UNIVERSAL is rather pointless, I'm going to change UNIVERSAL::require so it no longer pre-loads UNIVERSAL.pm and tell the Email::Valid author to remove his use UNIVERSAL line which is what started the problem in the first place. [1] To see this bug, download UNIVERSAL::require 0.02 and do: $ cd UNIVERSAL-exports-0.02 $ perl -Ilib -MUNIVERSAL::require -MUNIVERSAL -wle 1 Ambiguous call resolved as CORE::require(), qualify as such or use at /usr/share/perl/5.6.1/UNIVERSAL.pm line 6. -- This sig file temporarily out of order.
Re: [ANNOUCE] Test::Harness 2.25
On Sun, Jun 16, 2002 at 03:12:17AM -0400, Michael G Schwern wrote: The Test::Harness::Straps part of 16938 has been defered until Pudge gets back to be with some more data. The patch only addresses the symptoms, the real problem is likely elsewhere. http:[EMAIL PROTECTED]/msg00224.html Pudge got back to me. Chris found the real problem: MacPerl doesn't have a working putenv. So even though PERL5LIB is being correctly set the test subprocess won't see it. So I'll put 16938 in. -- This sig file temporarily out of order.
Re: Fwd: Test::Class... comments?
On Sun, Jun 16, 2002 at 08:28:26PM +0100, Adrian Howard wrote: Now, the way I do this at the moment is have num_method_tests() walk up the callstack until it finds a package that also isa Test::Class (in this case Base::Test) and then assume that is the class containing the method whose # tests it should alter. What happens when you've got a subclass of a subclass of Test::Class, and they're each overriding a method and adding a test? While this works, it just seems a trifle evil. When I coded it my thoughts went something like Gosh. That's a greasy hack... must fix that later. When later eventually came around the other solutions I came up with seemed even worse. I just have a sneaking feeling that there's a more elegant (for some definition of elegant) way of handling changing the number of tests for test methods that will cope with inheritance nicely. Make no_plan the default? Works for Test::Inline. -- This sig file temporarily out of order.
Re: Thread-safe Test::Builder
On Sat, Jun 15, 2002 at 06:00:44PM +0100, Mark Fowler wrote: I was, out of curiousity, wondering what happened to these changes as they don't seem be in 5.8.0RC1 My time machine's in the shop awaiting a shipment of sky hooks. (Translation: Arthur's patch came after RC1.) As for why it's not in the CPAN version yet: I'm lazy. -- This sig file temporarily out of order.
Re: Pondering Test::Depend
On Sat, Jun 08, 2002 at 10:45:50AM -0700, chromatic wrote: Taking an average test suite as an example, we could have 'foo.t' and 'bar.t', testing Foo.pm and Bar.pm respectively. Bar depends on Foo, and bar.t mocks a couple of methods of Foo. If bar.t uses Test::Depend, it'll pick up several things: - the module being tested (assumed from a use_ok() or require_ok() call) - the module being mocked (assumed from an entry in %INC) If there's no dependency information, it'll use the current version of Foo.pm, the last modified timestamp of the test file, and the test status of foo.t (if it also uses Test::Depend) to keep track of any changes to Foo.pm. If this dependency information changes, it'll fail a test (or maybe warn) because there's a potential interface change that Bar.pm may need to know. It looks interesting up to this point. Basically, everytime Bar.pm is touched, edited or upgraded, the test will fail. Every doc patch, typo fix and minor upgrade. This will produce orders of magnitude more false negatives than real failures, which will sap the credibility of the test causing it to be ignored. If Perl had strong interfaces, like Java, this could be done a bit more intellegently. But it doesn't. In reality, if Bar.pm changes in an incompatible way Foo.pm's own tests should fail and that should cover you. Like I said earlier, I'm aiming for an 80% solution. There are some assumptions and some fragilities here, but it represents a great improvement over the current system, in my mind. -- This sig file temporarily out of order.
[ANNOUNCE] Test::Harness 2.24
http://www.pobox.com/~schwern/src/Test-Harness-2.24.tar.gz Two bugs. 2.24 Wed May 29 19:02:18 EDT 2002 * Nikola Knezevic found a bug when tests are completely skipped but no reason is given it was considered a failure. * Made Test::Harness::Straps-analyze_file Test::Harness a bit more graceful when the test doesn't exist. 2.23 Wed May 22 12:59:47 EDT 2002 - reason for all skip wasn't being displayed. Broken in 2.20. - Changed the wait status tests to conform with POSIX standards. - Quieted some SYSTEM$ABORT noise leaking out from dying test tests on VMS. -- This sig file temporarily out of order.
What ever happened to Test::Set?
There was a big discussion about writing a module to make it easier to test sets and common complex data structures and superceed the paltry eq_* functions in Test::More. http:[EMAIL PROTECTED]/msg01369.html Anyone working towards that? -- This sig file temporarily out of order.
Test::Harness skip reason report change?
Sarathy's convinced me that we need a better way to format skip reasons so they don't just run off the end of the screen. After a lot of hemming and hawing on my part, we may have found an easy fix. So here's an example of a rather busy report with 2.03. t/sample-tests/descriptive.ok t/sample-tests/with_comments...ok, 2/5 unexpectedly succeeded t/sample-tests/combinedFAILED tests 3, 9 Failed 2/10 tests, 80.00% okay (-1 skipped test: 7 okay, 70.00%) t/sample-tests/skipok, 1/5 skipped: rain delay t/sample-tests/skip_no_msg.ok, 1/1 skipped: [no reason given] t/sample-tests/skip_allskipped: rope t/sample-tests/todook, 1/5 unexpectedly succeeded t/sample-tests/todo_inline.ok, 1/3 unexpectedly succeeded t/sample-tests/simple..ok Failed Test Stat Wstat Total Fail Failed List of Failed --- t/sample-tests/combined 102 20.00% 3 9 (5 subtests UNEXPECTEDLY SUCCEEDED), 1 test and 3 subtests skipped. Failed 1/9 test scripts, 88.89% okay. 2/39 subtests failed, 94.87% okay. and how here's the same battery of tests run with a few simple formatting changes. t/sample-tests/descriptive.ok t/sample-tests/with_comments...ok 2/5 unexpectedly succeeded t/sample-tests/combinedFAILED tests 3, 9 Failed 2/10 tests, 80.00% okay (less 1 skipped test: 7 okay, 70.00%) t/sample-tests/skipok 1/5 skipped: rain delay t/sample-tests/skip_no_msg.ok 1/1 skipped: [no reason given] t/sample-tests/skip_allskipped skipped: rope t/sample-tests/todook 1/5 unexpectedly succeeded t/sample-tests/todo_inline.ok 1/3 unexpectedly succeeded t/sample-tests/simple..ok Failed Test Stat Wstat Total Fail Failed List of Failed --- t/sample-tests/combined 102 20.00% 3 9 (5 subtests UNEXPECTEDLY SUCCEEDED), 1 test and 3 subtests skipped. Failed 1/9 test scripts, 88.89% okay. 2/39 subtests failed, 94.87% okay. It makes a noticable difference when the file path to the test and the skip reason is long. .../ext/I18N/Langinfo/Langinfo.skipped: I18N::Langinfo or POSIX unavail able vs .../ext/I18N/Langinfo/Langinfo.skipped skipped: I18N::Langinfo or POSIX unavailable What do people think? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Kids - don't try this at--oh, hell, go ahead, give it a whirl...
Re: [PATCH] use_ok() with pragmas in 5.005_03
On Tue, Apr 23, 2002 at 03:28:12AM +0900, Tatsuhiko Miyagawa wrote: it seems that calling pragma-import() raises a warning in perl5.005_03. It may be a lexer bug in 5.005_03 ... % perl5.00503 -we 'require base; base-import(CGI)' Unquoted string base may clash with future reserved word at -e line 1. P.S. Mike, the same thing applies with your mixin.pm gotcha -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One let me think it over while Cheese beats you with a baseball bat.
Re: [PATCH] Make Test::More::can_ok() use objects, not just classes
On Sat, Apr 20, 2002 at 11:51:30AM -0700, chromatic wrote: Attached is a patch to make Test::More do the right thing (as I see it) in this case. Previously, it called can() on the class name, which obviously doesn't work here. You're right. I suspect something similar should be done for isa_ok(). Piers already made sure isa_ok() calls isa() directly on the object and doesn't use UNIVERSAL::isa. So that's all good. The included test fails with vanilla 0.43 and passes with the attached patch. Maybe it would be better to bless a scalar, I've used a simplified version of your patch and also tested that isa_ok() honors isa() overrides. Thanks. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One 4 WHEREAS, the siren song of payola issuing from the discordant calliopes of these gambling vessels has led thousands of Kentucky citizens to vast disappointment and woe; -- Kentucky Legislature, HR 256
Re: Starting with a fresh sandbox
On Thu, Apr 11, 2002 at 12:16:27PM -0500, Andy Lester wrote:
I'm using Test::More for doing infrastructure testing throughout my
project. One of the tests I'm doing is making sure that every .pm file is
able to load on its own, since I've had problems where the .pm had
dependencies on other modules, but so long as the calling program included
those modules first, the problem would remain hidden.
One simple way around this is to use SelfTest or Test::Inline and
throw in at the top of each of your modules:
=for testing
use_ok('My::Self');
and then just run the tests in each self-testing module in turn.
So I have a test where I do a use_ok( 'Foo::Bar' ). All fine and good.
What I'd like is a way to do use_ok( 'Foo' ) and a use_ok( 'Bar' ), but
somehow make sure that the order of checking these not matter. Even in an
eval {}, my namespace is going to be polluted after the first use_ok(),
sin't it?
What you want, basically, is the ability to quickly and easily run
perl code in a seperate process. If you're on Unix you can just
fork(). Otherwise you have to make a system() call to Perl.
In the Perl core there's a file t/test.pl. This reimplements most of
Test::More's features, but in a much more conservative way to avoid
using the very features that you're testing. It also contains a
handful of interesting new functions above and beyond Test::More. One
of these is fresh_perl() which runs a given chunk of code in a fresh
Perl process in a cross-platform manner. It's usually used to test
code that will cause a segfault, but's useful for what you want, too.
I've been meaning to make a CPAN version of this code, but I'm out of
tuits.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
11. Every old idea will be proposed again with a different name and
a different presentation, regardless of whether it works.
-- RFC 1925
Re: Starting with a fresh sandbox
On Thu, Apr 11, 2002 at 02:49:30PM -0500, Andy Lester wrote:
One simple way around this is to use SelfTest or Test::Inline and
throw in at the top of each of your modules:
=for testing
use_ok('My::Self');
and then just run the tests in each self-testing module in turn.
The bummer is that I'm not the only programmer, and I've had problems with
.pm files being created that don't compile on their own.
Teach the other folks how to test. :)
I want an automated test to File::Find thru the source tree and load
up each .pm as it gets found, making sure it'll load on its own.
Ok:
sub test {
return unless /\.pm$/;
print # compiling $_\n;
system(qq{$^X -e 'print eval { require $_; 1 } ? ok\n : not ok\n' });
}
use File::Find;
find(\test , shift || '.');
adjust as necessary. Doesn't have to be fancy.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
sun readdened wheat stalks
bowing down in autumn sun
my mind wanders far
-- stimps
Re: Comparing Data Structures Slopply
On Wed, Apr 10, 2002 at 02:03:51PM +0100, Nicholas Clark wrote: On Wed, Apr 10, 2002 at 08:57:30PM +0900, Curt Sampson wrote: On Wed, 10 Apr 2002, Mark Fowler wrote: On Wed, 10 Apr 2002, Curt Sampson wrote: eq_set() is really bag comparison. Well, my point was, it *is* a set comparison if you pass it sets. The problem, in my view, is that perl lets you pass it something which is not a set. Thus, it seems perfectly fair to me for it to produce undefined behaviour under such circumstances. The API doesn't define which side is expected and which side got, does it? eq_set() is not a test function. It doesn't produce ok/not ok and shows no diagnostics. It just returns true if they're equal, false otherwise. So there's no concept of which is expected and which is got anymore than: $got eq $expected $expected eq $got Test::More's eq_set() is just a bad name. Don't fixate on it, write Test module with better set handling. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One It's Flypaper Licking time!
Re: Test::Exception... comments?
On Thu, Apr 11, 2002 at 10:37:10AM +0900, Curt Sampson wrote:
On Tue, 9 Apr 2002, Michael G Schwern wrote:
I'm not sure exactly what the purpose of this is; your test will
still fail if it dies even when not in a lives_ok block, right?
It'll fail and take the whole rest of the test program with it. Some
testing systems like to abort the test script on failure. Perl's
doesn't.
I'm not entirely sure I buy this, since the framework seems perfectly
happy to tell me that something is wrong whether I complete all
the tests in a script or not. But it's hardly a point worth arguing.
Well, yes. But you'd like to see the results of the rest of the rest
of the tests. Not simply up to where your first failure was.
Testing Doctrine says that any tests after the first failure are
suspect because if one interface is failing, anything derived from
that will also be wrong. In practice this is rarely the case and
there's plenty of useful data to be gotten from the full run.
In practice if your test cases are getting so large that any of
this matters you have Other Problems.
Aegis, in particular, pretty much requires tests to always
exit gracefully, passing or not.
As perl does: when it dies, it exits gracefully with error code
255 indicating that the test failed? Or am I misunderstanding how
Aegis works?
Aegis works by reading the exit code of the test. 0 is success, 1 is
failure, anything is something went horribly wrong. The thing to
keep in mind is Aegis was designed by a pre-ANSI C programmer, so any
run-time errors that cause the program to abort are an indication that
something is Very Wrong.
The reason this becomes important is Aegis has two modes of testing.
The first is your normal everything should pass. But the second is
interesting. When your change in Aegis is fixing a bug Aegis requires
that the new tests against the old code *fail*, and that the new tests
against the new code succeed.
It has to fail gracefully, exiting in a predictable manner, in order
to be reasonably sure that new tests will catch the bug. If the test
simply dies you don't know if that was because it caught a SIGINT or
it segfaulted or what.
Which becomes a Royal Pain In The Ass when fixing the bug involves
adding new methods and then you have to do things like:
if( Some::Class-can('this_new_method') ) {
...some tests involving this_new_method()...
}
else {
fail('this_new_method not defined');
}
which becomes Rapidly Tiresome. This is why SKIP and TODO blocks work
the way they do (even though they're not directly applicable) and why
Test::More will exit with the number of tests which failed (makes it
much easier to write a simple Aegis perl testing hook).
XUnity stuff snipped
(And yes, it's not a co-incidence that this looks a lot like Beck
and Gamma's unit test frameworks for Java and Smalltalk.)
Thoughts?
See also Test::Unit and Test::Inline. Test::Unit because it's XUnit
in Perl. Test::Inline because each block of embedded tests runs in
it's own lexical scope.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Obscenity is the last resort of the illiterate, Mother Fucker
-- KAL
Re: Comparing Data Structures Slopply
On Thu, Apr 11, 2002 at 09:54:56AM +0900, Curt Sampson wrote: On Wed, 10 Apr 2002, Nicholas Clark wrote: The API doesn't define which side is expected and which side got, does it? I believe it's defined (though perhaps not explicitly) as the first argument being the got, and the second being the expected. This is how the error messages print it, IIRC. I can tell you quite authoritatively that it is not defined for eq_set, eq_array or eq_hash. You might be confused with is() or is_deeply(). Again, they are *NOT TEST FUNCTIONS*. They produce no diagnostics. They are just normal comparision functions and becoming more and more out of place in Test::More. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One O you fat bastard anus clogged (library paste) you're not laughing now -- Halfjack
Re: Comparing Data Structures Slopply
On Tue, Apr 09, 2002 at 03:26:21PM +0100, Mark Fowler wrote: There's a lot of other problems like that. So I was thinking of writing Test::Sloppy (aka Test::Fuzzy, aka...) What would it do? (I can show you lots of sloppy tests if you like. :) -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Here's some scholarly-ass opinions...
Re: Comparing Data Structures Slopply
On Tue, Apr 09, 2002 at 05:02:32PM +0100, Mark Fowler wrote: On Tue, 9 Apr 2002, Michael G Schwern wrote: On Tue, Apr 09, 2002 at 03:26:21PM +0100, Mark Fowler wrote: There's a lot of other problems like that. So I was thinking of writing Test::Sloppy (aka Test::Fuzzy, aka...) What would it do? (I can show you lots of sloppy tests if you like. :) Sorry, maybe it wasn't clear from the example. sort of like eq_set meets is_deeply. I think Sloppy or Fuzzy is the wrong term for this. It implies you're doing some sort of approximate matching like String::Approx. Sounds like you just want better tools for testing multi-level data structures. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Don't ask me lady, I live in beer.
Re: Test::Exception... comments?
On Tue, Apr 09, 2002 at 10:05:49PM +0100, Adrian Howard wrote:
Hi all,
I've been refactoring a bunch of old tests with Test::More and some
convenience routines for testing exceptions dropped out (along with some
class base testing and mock object modules which still need cleaning up into
something sane.)
dies_ok BLOCK TEST_NAME
Tests to see that BLOCK exits by dying, rather than by exiting
normally. For example:
dies_ok { div(1, 0) } 'divide by zero detected';
You probably want to guarantee that $@ will be how it died so you can do:
dies_ok { div(1,0) } 'div by zero';
like( $@, qr/^Illegal division by zero/ );
Even though you can use throws_ok(), the dies_ok() + $@ combo is more
flexible more processing than just a regex needs to be done on $@.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Death follows me like a wee followey thing.
-- Quakeman
Re: Test::Exception... comments?
On Wed, Apr 10, 2002 at 10:24:32AM +0900, Curt Sampson wrote:
On Tue, 9 Apr 2002, Adrian Howard wrote:
lives_ok BLOCK TEST_NAME
Tests to see that BLOCK exits normally, and doesn't die.
I'm not sure exactly what the purpose of this is; your test will
still fail if it dies even when not in a lives_ok block, right?
It'll fail and take the whole rest of the test program with it. Some
testing systems like to abort the test script on failure. Perl's
doesn't. Aegis, in particular, pretty much requires tests to always
exit gracefully, passing or not.
It's not uncommon to do this:
ok( eval { ...some code...; 1; } );
which would be really all that lives_ok() would be.
Not that I have any real objection to this, but I can't see where
I'd ever use it.
The rest looks great, however. Ideally it should end up in Test::More
at some point, I would think.
Nope. No more functions in Test::More, too much in there already.
I want to encourage the idea of having more than one Test::* module in
use in a test.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
HA HA HA You're all so ridiculous! But thanks for the money!
Re: Test::Builder and Test::More change requests
On Thu, Apr 04, 2002 at 12:01:45PM +0100, Mark Fowler wrote:
Firstly, is there anyway we could get Test::Builder's diag (and thus
Test::More's) diag to return false not true like it currently does.
This would then allow me to write simply
sub is_reversed
{
return $Tester-ok($_[0] eq reverse($_[1]))
or $Tester-diag(The reverse of '$_[0]' is '.reversed($_[0]).
' not '$_[1]' as expected)
}
rather than having to do the horrible 0 at the end to ensure that
the method returns the correct truth (otherwise is_reversed will return
true no matter what is called)
Sensible. Done. If anyone asks, it's your fault.
Secondly, I was wondering about eq_set in Test::More.
ok(eq_set([1,1,1],[1]));
fails, where really it should work as sets don't care about reoccurring
elements. Should this be 'eq_bag' instead?
eq_set() is really just because I got sick of writing:
eq_array([sort @something], [sort @something_else]);
wasn't really thinking about Set Theory at the time. So yes, eq_set()
would probably be better as eq_bag() or something. But we can't
change the name now.
However, the eq_* functions are largely discouraged because they can't
produce intellegent diagnostics. If any new complex data structure
comparison utilites were added I'd rather go the is_deeply() route.
And all that stuff needs to be moved into Test::Builder anyway. Can
'o worms.
As for Test::More, don't think I'll be taking any new functions. The
interface is already too big. But the deep data structure
functionality isn't really complete.
I'd encourage you to write a Test::Builder based module that does more
complete set complex data testing. Either talk to Barrie about
adding it to Test::Differences or start a new module. Test::Set,
Test::Data::Deep, etc...
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Lesbian lovers
return to fourth grade slogans
EAT SOME PASTE, they cry
sobbing lesbian
excluded from the party
allergic to wheat
-- mjd
Re: Test::Harness Conditional Test Execution
On Tue, Mar 26, 2002 at 01:47:38PM +0900, Curt Sampson wrote: On Mon, 25 Mar 2002, Michael G Schwern wrote: 2. Skip test of code where dependencies have been tested and found to be failing. For example, if the test for the database connection module fails, nothing that uses that module should have its test run. (The tests should of course be reported as skipped.) Not really a Test::Harness thing, it's up to your test program to do this. Test::More has facilities to skip blocks of tests or whole test files. They're explained in the Test::Tutorial. Hmm. So you're saying that I should modify my tests to skip themselves when they get information that a module they depend on has failed its tests? How do you propose I get that information to the test? You can't. Not easily. About all you can do is abort the whole suite of tests using the Bail Out feature of Test::Harness should a Really Important Thing fail. print Bail out! Database connection failed\n; and Test::Harness won't run any further test scripts. Anything of a finer granularity rapidly gets difficult. You can use the callback interface, but it's not stable and is subject to change. In fact, it probably will. I still encourage people to use it. Well, since I need it for my current method of working, I'll just use it and be aware that it will change. Make sure to give me any feedback about it to I can expand it according to people's needs. I'll try to at least document it next release. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One GOD made us funky!
Re: Test::Harness Conditional Test Execution
On Mon, Mar 25, 2002 at 04:35:52PM +0900, Curt Sampson wrote: I'm setting a unit testing system for a pile of code right now which is going to expand into hundreds of tests. I've currently got a simple program that finds all of the tests in the system and hands this list to Test::Harness to be run. However, I'd like to be able to do two things: 1. Run the tests in dependency order. That is, if module A uses module B, the tests for module B should be executed before those for module A. Test::Harness will run the files in whatever order you give it to them. So assuming a single module's sanity tests are in a single file you can simply feed them in in a rough dependency order. 2. Skip test of code where dependencies have been tested and found to be failing. For example, if the test for the database connection module fails, nothing that uses that module should have its test run. (The tests should of course be reported as skipped.) Not really a Test::Harness thing, it's up to your test program to do this. Test::More has facilities to skip blocks of tests or whole test files. They're explained in the Test::Tutorial. This should make it easier to fix problems because we'll see right away where we need to start fixing things, and we won't have to go through reams of test failures to find the one low level module failure that caused them all. I'm not very familiar with Test::Harness usage, and I'm wondering if someone could suggest a good way of doing this. I've looked at examples/mini_harness.plx; is using those private methods really the suggested way to go about things? No. It uses an undocumented, experimental callback interface and some private formatting functions from Test::Harness which I'm not quite sure yet how to move into Straps. mini_harness is just a proof-of-concept to see if I could reimplement most of Test::Harness's interface. You can use the callback interface, but it's not stable and is subject to change. In fact, it probably will. I still encourage people to use it. Also, from looking at the code, I get the impression that analyze_file is supposed to run the perl file given to it as an argument. But the docs say otherwise: Like analyze, but it reads from the given $test_file. Well, analyze reads test results, so presumbly the documentation is saying that this is supposed to be a file containing the test results, not the test code. Hmmm. Nope. Your original impression is correct. Docs are just poorly worded. Just touched it up. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One If you have to shoot, shoot! Don't talk. -- Tuco, The Good, The Bad And The Ugly
Re: How to test for the absence of an infinite loop?
On Mon, Mar 18, 2002 at 02:45:33AM -0500, Mark-Jason Dominus wrote:
I've just found a bug in my module. The bug results in an
inappropriate infinite loop.
Before I fix the bug, I want to write up a test case.
I can reproduce the bug OK, but if I put that code into the module
tests, and the bug isn't fixed, or comes back for some reason, then
the test program will go into an infinite loop.
The test harness doesn't seem to notice that. I don't want to go
sticking code into my test files that might cause an infinite loop and
hang up the test harness or the entire system.
If the module were for Unix only, I would stick an 'alarm 10' at the
top of the test program. I'm told this won't work under Win32.
What's the right way to handle this?
Use alarm and skip the test if $Config{d_alarm} is false (see
t/op/alarm.t for an example). If you think the infinite loop is due
to a programming glitch, as opposed to a cross-platform issue, this
will be enough.
The only other thing I can think of is to set up a parent process
which forks off a child and kills it if its still going after 10
seconds. That gets into fork(), which is even less portable than
alarm. Checking to see if Win32 can pseudo-fork is a little more
complicated than alarm (see t/op/fork.t).
If you provide both methods you can cover Unix, Win32 and a bunch of
other platforms.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
What's black and white and makes you duck as it goes by?
A nun with a spear through her head.
How to test that flock works? (was Re: How to test for the absence of an infinite loop?)
On Mon, Mar 18, 2002 at 11:14:59AM -0500, Mark-Jason Dominus wrote:
I just thought of a clever way to do it without alarm!
So clever, it doesn't work!
lock_file($foo);
open(FH, $foo);
ok( !flock(FH, LOCK_NB | LOCK_EX) );
Seriously, on most unix systems, the following:
flock(FH, LOCK_EX);
flock(FH, LOCK_EX|LOCK_NB) or die;
doe *not* die. A process is allowed to ask for (and obtain) an
exclusive lock on a file if it already has a lock on that same file.
There's no deadlock at all.
Oh dear, never knew that.
Ok, how about some variant on this:
BEGIN {
*CORE::GLOBAL::flock = sub (*$) {
my($fh, $flag) = @_;
$Locks{fileno $fh}++;
return flock $fh, $flag;
};
}
I'm going to have to hand-wave a bit at this point. You'll likely
have to tie any handle that goes through flock() so you can check if
it was CLOSEd or DESTROYed (and thus releases the lock).
The idea is you can know when flock() has been called, on what fileno
and with what flags, and also when locks have been implicitly released.
A variant on that might be if you already have internal _lock() and
_unlock() functions.
{
my $orig_lock = \Your::Module::_lock;
my $orig_unlock = \Your::Module::_unlock;
no warnings 'redefine';
sub Your::Module::_lock {
my($file) = @_;
$Locks{$file}++;
goto $orig_lock;
}
sub Your::Module::_unlock {
my($file) = @_;
$Locks{$file}--;
goto $orig_lock;
}
do_something_that_should_result_in_a_lock($file);
is( $Locks{$file}, 1 );
...whatever else...
*Your::Module::lock = $orig_lock;
*Your::Module::unlock = $orig_unlock;
}
Sorry I can't be more specific.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Realize this, sweetheart, I'm squeezing my poodle.
Re: How to test for the absence of an infinite loop?
On Mon, Mar 18, 2002 at 02:14:23PM -0700, Sean M. Burke wrote:
At 03:26 2002-03-18 -0500, Michael G Schwern wrote:
[...]Use alarm and skip the test if $Config{d_alarm} is false (see
t/op/alarm.t for an example). If you think the infinite loop is due
to a programming glitch, as opposed to a cross-platform issue, this
will be enough.
Ya know, for a while I've wished there were a rather tidier to way to ask
if something were implemented. Like perl-can_alarm, via a class perl
that just has a clever AUTOLOAD for these things.
Someday.
*mumble* wild ideas about OS::Spec, File::System::Spec *mumble*
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
my mother once said
never eat paste my darling
would that I heeded
-- stimps
[ANNOUNCE] Test::Harness 2.02
http://www.pobox.com/~schwern/src/Test-Harness-2.02.tar.gz 2.02 Thu Mar 14 18:06:04 EST 2002 * Ken Williams fixed the long standing $^X bug. * Added HARNESS_VERBOSE * Fixed a bug where Test::Harness::Straps was considering a test that is ok but died as passing. - Added the exit and wait codes of the test to the analyze_file() results. A minor release, mostly to fix the Straps bug. Test::Harness::Straps is now in active use thanks to the Good People at Mitel Networks (see also E-Smith) and will be firming up rapidly. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One It's Tobacco time!
[ANNOUNCE] Test::Simple/More/Builder 0.42
http://www.pobox.com/~schwern/src/Test-Simple-0.42.tar.gz Small update. Just fixing a bug in T::B-current_test() to actually make it work. Folks who need to do tests with forking and such will be pleased. 0.42 Wed Mar 6 15:00:24 EST 2002 - Setting Test::Builder-current_test() now works (see what happens when you forget to test things?) - The change in is()'s undef/'' handling in 0.34 was an API change, but I forgot to declare it as such. - The apostrophilic jihad attacks! Philip Newtons patch for grammar mistakes in the doc's. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Beef Coronary
How To Test Test Destructive Things
So here's why I just fixed that bug in Test::Builder
The situation is thus: I've got a little script that's used to
start/restart and stop the ssh daemon. I want to test that it can in
fact stop the ssh daemon. Unfortunately, I ssh'd into that machine!
So stopping it would be Bad. [1]
A TCL test engineer at TPC last year gave me a technique to handle
this. It takes advantage of Perl's ability to replace existing
functions at run-time.
The sshd-stop script basically does its work like so:
use utils;
# this calls /etc/init.d/sshd stop
serviceControl( NAME = 'sshd', ACTION = 'stop' );
# this then kills all existing connections.
killall('HUP', 'sshd');
so the plan is to literally replace serviceControl() and killall()
with stub functions that do nothing but check it got the right
arguments. So write up a little module and run the ssh-stop script
system( $^X, '-It/lib', '-Mreload_overrides', $Original_File );
is( $?, 0, 'ran myself ok' );
the clever thing there is we're loading a little module,
reload_overrides.pm, that plants our stub functions. It looks like
this...
use Test::More 'no_plan';
# Here we tell the test to not use any numbers (because there were
# probably tests output'd before us) and to not do end-of-test
# checks.
my $TB = Test::More-builder;
$TB-use_numbers(0);
$TB-no_ending(1);
first thing that's going on here is we're telling the underlying
Test::Builder object to not use numbers and don't do any extra
processing at the end of the test. This is because it will be run in
a seperate process from the test script itself.
use util;
use util::system;
we load our victims so that when the sshd-stop script goes to load
them it won't blow over our overrides.
package util;
::can_ok('util', 'serviceControl');
no warnings 'redefine';
sub serviceControl {
my(%params) = @_;
::pass('service control called');
::is( $params{NAME}, 'sshd', 'serviceControl NAME == sshd' );
::is( $params{ACTION}, 'stop', ' ACTION == stop' );
return 1;
}
so this is a simple override. First we check to make sure that
serviceControl() is already loaded. Then we tell perl to not warn us
about redefining the function.
The function itself is very simple. It just reads in the arguments
and makes sure its what we expect them to be. We assume that the real
serviceControl() works, so all we care about is how its called.
package util::system;
::can_ok('esmith::util::system', 'killall');
no warnings 'redefine';
sub killall {
my($sig, @commands) = @_;
::pass('killall called');
::is( $sig, 'HUP', ' with a HUP' );
::is( @commands,1, ' one command' );
::is( $commands[0], 'sshd', ' for sshd' );
return 1;
}
and here's the same for killall().
It has to be called with some care, making sure that we take into
account that the subprocess will be running some tests of its own to
make the test counters all work out.
# this will run 9 tests.
my $tb = Test::More-builder;
$tb-current_test($tb-current_test + 9);
system( $^X, '-It/lib', -Mreload_overrides', $Original_File );
is( $?, 0, 'ran myself ok' );
and the result looks something like this:
ok 1 - ran myself ok
ok 2 - sshd lockfile touched
ok - esmith::util-can(serviceControl)
ok - esmith::util::system-can(killall)
ok - service control called
ok - serviceControl NAME == sshd
ok -ACTION == stop
ok - killall called
ok -with a HUP
ok -one command
ok -for sshd
ok 12 - ran myself ok
ok 13 - ran myself ok
ok 14 - sshd lockfile untouched
1..14
the unnumbered tests are coming from the subprocess. They're
surrounded by the rest of the normal test script.
And that's how you can test something that's otherwise destructive.
Questions? :)
[1] Its not just stop. Its stop with extreme prejudice. It kills
all existing connections.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Beer still cheaper than crack!
[ANNOUNCE] Test::Inline 0.14
http://www.pobox.com/~schwern/src/Test-Inline-0.14.tar.gz 0.14 Thu Feb 28 12:38:53 EST 2002 * pod2test now provides an $Original_File * Fixed handling of print STDERR ... in tests * Fixed $_STDERR_ and $_STDOUT_ so they clear themselves between test blocks (bug reported by Wolfgang Weisselberg) - Some point between Test::More 0.30 and 0.33 it became unsafe to redirect STDOUT/STDERR in tests. This broke pod2test. The minimum version of Test::More has been uped (again, thanks Wolfgang) 0.13 Mon Feb 18 16:40:29 EST 2002 * pod2test now exits with 1 if it doesn't find any embedded tests This release of Test::Inline is brought to you by the letters W and M. [1] I'm actually using Test::Inline now so some long standing bugs are being fixed. $_STDERR_ and $_STDOUT_ work much better now. [1] W for Wolfgang Weisselberg and M for Mitel who's paying me to write tests. :) -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Don't worry, baby, my wrath can be pretty groovy. http://www.goats.com/archive/980804.html
Re: More Test::Builder::Test stuff
On Thu, Feb 21, 2002 at 12:52:01PM +, Nicholas Clark wrote: On Sun, Feb 17, 2002 at 02:00:06PM +, Mark Fowler wrote: On Sun, 17 Feb 2002, Mark Fowler wrote: I'd really like to see that it still works on a system that doesn't have Term::ANSIColor installed on it (it should turn colouring into a no-op and skip tests, but I can't test that.) D'Oh! No it shouldn't. As color should work (just without colouring) on systems that don't have Term::ANSIColor installed then the tests shouldn't be skipped at all, they should be carried out to explicitly check that then still work without it. Code updated so that it should do just that. I think that if you have enough disk space you should be able to build yourself a second perl install that doesn't have Term::ANSIColor installed, by configuring perl to use a different prefix and ensuring it doesn't have any of the standard places to find libraries in INC. Simpler thing to do would be to create t/lib/fake/Term/ANSIColor.pm which is something like: package Term::ANSIColor; die, die, die, die, die Can't locate Term/ANSIColor.pm in \@INC\n; and then when you want to simulate not having Term::ANSIColor in the tests you can do: unshift @INC, 't/lib/fake'; ... and any unshielded require/use of Term::ANSIColor will explode convincingly. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Let's face it, said bearded Rusty Simmons, opening a can after the race. This is a good excuse to drink some beer. At 10:30 in the morning? Well, it's past noon in Dublin, said teammate Mike [Joseph] Schwern. It's our duty. -- Sure, and It's a Great Day for Irish Runners Newsday, Sunday, March 20, 1988
Re: More Test::Builder::Test stuff
On Thu, Feb 21, 2002 at 10:05:09AM -0500, Michael G Schwern wrote: Simpler thing to do would be to create t/lib/fake/Term/ANSIColor.pm which is something like: package Term::ANSIColor; die, die, die, die, die Can't locate Term/ANSIColor.pm in \@INC\n; Sorry, that should be: die die kill kill die die die kill die Can't locate Term/ANSIColor.pm in \@INC (\@INC contains: @INC); :) -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Obscenity is the last resort of the illiterate, Mother Fucker -- KAL
Make Schwern poor in just 20 easy steps!
Here's the current list of untested modules. The magic number is 20. B::CC B::Disassembler B::Lint B::Stackobj B::Xref Byteloader CPAN CPAN::FirstTime Dynaloader ExtUtils::MM_NW5 ExtUtils::MM_VMS [exists, but needs some lovin'] ExtUtils::Install ExtUtils::Liblist ExtUtils::Mksymlists Net::Cmd Net::Domain Net::POP3 O Pod::Html Pod::Select -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One purl Hey Schwern! honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk, honk!
Re: VMS status @14710
On Sat, Feb 16, 2002 at 05:05:33PM -0600, Craig A. Berry wrote:
from miniperl during the build before File::Glob exists.
I don't suppose there is a way to invoke at run-time the old-style
glob (which for us is really internal, not external)?
You could instead just change that test so it only runs if File::Glob
is loaded.
if( $INC{'File/Glob.pm'} ) {
...glob('0') test...
}
else {
print ok 8 # skip: File::Glob emulated Unixism\n;
}
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
What about MY need to be generalized and easily dismissed?
http://www.goats.com/archive/000221.html
Re: Interactive tests?
On Thu, Feb 14, 2002 at 03:35:43PM +, Mark Fowler wrote: So..sensible default values, options to change to Makefile.PL for those cases where you need to...and maybe then falling back to it being interactive? There's the entirely undocumented but really, really useful ExtUtils::MakeMaker::prompt() function. use ExtUtils::MakeMaker; my $answer = prompt(Wanna buy a monkey?, Yes); print $answer\n; which does All The Right Things: $ perl ~/tmp/monkey.plx Wanna buy a monkey? [Yes] No No $ perl ~/tmp/monkey.plx Wanna buy a monkey? [Yes] Yes $ perl ~/tmp/monkey.plx /dev/null Wanna buy a monkey? [Yes] Yes Yes That last one is showing prompt() automatically accepting the default when STDIN is not a TTY (ditto for STDOUT). It also honors PERL_MM_USE_DEFAULT environment variable (which, oddly enough, is documented). Documenting it is in my TODO list. Its safe to use. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One They had applied the blinders of steam-grilled hamburgers to my eyes.
[ANNOUNCE] Test 1.20
http://www.pobox.com/~schwern/src/Test-1.20.tar.gz After having it pointed out that all of Test.pm's failure diagnostics were going to STDOUT, and thus not visible via 'make test', I've changed it so that failure diagnostics go to STDERR so they'll always show up. This is similar to the way Test::More behaves (surprise). 2002-02-07 Michael G Schwern [EMAIL PROTECTED] * Release 1.20 * Failure diagnostics now go to STDERR so they show up in 'make test'. - noted in the docs that this module is no longer being developed. 2001-12-17 Michael G Schwern [EMAIL PROTECTED] * Release 1.19 - Resetting globals for mod_perl testing (thanks to Stas Bekman) - License change to be same as perl's -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Some like elmers glue but it needs reapplying. I use super glue. -- tlk
Coming soon to Module::Info...
Just got this working. Next release will have... my @isa = $mod-superclasses; my %methods = $mod-class_method_calls my @methods = $mod-object_method_calls; my @functions= $mod-function_calls; # Check for constructs which make perl hard to predict. my %methods = $mod-dynamic_method_calls; my @lines = $mod-eval_string; my @lines = $mod-gotos; my %controls = $mod-exit_via_loop_control; I'm likely also going to change the interface of packages_inside() and modules_used() to be more like subroutines so I can pack more info into it (like line numbers). Found class method call to binmode via main::STDOUT at /usr/share/perl5/CGI.pm line 182 Found class method call to binmode via main::STDIN at /usr/share/perl5/CGI.pm line 183 Found class method call to binmode via main::STDERR at /usr/share/perl5/CGI.pm line 184 Found object method call to add_parameter at /usr/share/perl5/CGI.pm line 559 Found class method call to request via Apache at /usr/share/perl5/CGI.pm line 269 Found object method call to register_cleanup at /usr/share/perl5/CGI.pm line 270 Found class method call to request via Apache at /usr/share/perl5/CGI.pm line 270 Found object method call to _reset_globals at /usr/share/perl5/CGI.pm line 273 Found object method call to init at /usr/share/perl5/CGI.pm line 274 Found object method call to new at /usr/share/perl5/CGI.pm line 329 Found object method call to _setup_symbols at /usr/share/perl5/CGI.pm line 247 Found object method call to param at /usr/share/perl5/CGI.pm line 540 Found object method call to charset at /usr/share/perl5/CGI.pm line 545 Found object method call to param at /usr/share/perl5/CGI.pm line 369 Found object method call to charset at /usr/share/perl5/CGI.pm line 371 Found object method call to charset at /usr/share/perl5/CGI.pm line 382 Found object method call to cgi_error at /usr/share/perl5/CGI.pm line 388 Found object method call to read_multipart at /usr/share/perl5/CGI.pm line 400 Found object method call to query_string at /usr/share/perl5/CGI.pm line 408 Found object method call to param at /usr/share/perl5/CGI.pm line 413 Found object method call to args at /usr/share/perl5/CGI.pm line 443 Found class method call to request via Apache at /usr/share/perl5/CGI.pm line 443 Found object method call to read_from_client at /usr/share/perl5/CGI.pm line 453 Found object method call to parse_params at /usr/share/perl5/CGI.pm line 472 Found object method call to add_parameter at /usr/share/perl5/CGI.pm line 475 Found object method call to parse_keywordlist at /usr/share/perl5/CGI.pm line 476 Found object method call to param at /usr/share/perl5/CGI.pm line 482 Found object method call to param at /usr/share/perl5/CGI.pm line 488 Found object method call to delete at /usr/share/perl5/CGI.pm line 493 Found object method call to delete at /usr/share/perl5/CGI.pm line 494 Found object method call to save_request at /usr/share/perl5/CGI.pm line 496 Found object method call to all_parameters at /usr/share/perl5/CGI.pm line 295 Found object method call to add_parameter at /usr/share/perl5/CGI.pm line 313 Found object method call to print at /usr/share/perl5/CGI.pm line 519 Found object method call to _make_tag_func at /usr/share/perl5/CGI.pm line 647 Found object method call to _setup_symbols at /usr/share/perl5/CGI.pm line 226 -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One It's Airplane Glue sniffing time!
[ANNOUNCE] Module::Info 0.11
http://www.pobox.com/~schwern/src/Module-Info-0.11.tar.gz Didn't get quite as far as I'd hoped, but I got the hard part's done. The interesting bit is you can now find every function and method call in a program (I just realized I'm not handling function calls with symbolic refs quite right, will fix), the line its on, etc see subroutines_called(). superclasses() is actually a really dirty cheat and I have to spruce it up some. Some users were reporting silent failures in the last few versions. I've made failures much more noisy so we'll see if that can be worked out. 0.11 Sat Feb 2 16:54:23 EST 2002 * Fixed B::Utils-kids * Added subroutines_called() * Added superclasses() * Added dynamic_method_calls() - Documented that it isn't just for modules. - name() can now be used to set - failures inside B::Module::Info now report their errors instead of failing silently * Added a THANKS section. 0.10 Wed Dec 12 05:40:14 EST 2001 * Windows filenames would cause subroutines() and modules_used() to choke. (Thanks to Alessandro Forghieri for the patch) -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Let me check my notes. http://www.sluggy.com
Alternative code review ideas?
I'm revamping my How To Be Lazy tutorial for 2002 with an eye towards making it a collection of techniques which are easy to learn, easy to teach, easy to accept and hard to screw up. I'm stuck on code review alternatives. Essentially, I'm looking for a way to make sure: Stupid mistakes in code get caught Tests get written and have decent coverage Code style isn't attrotious No one programmer gets tunnel vision and cranks out a load of crap Everyone gets some familiarity with the whole code base Experienced programmers work with inexperienced and vice-versa. I know of two good ways to do this, but none meets my criteria. One is Pair Programming, the other is Mandatory, Managed Code Reviews. Pair Programming does all the above to some extent. Its easy to learn, easy to teach (by its very nature) and fairly hard to screw up. Unfortunately, many people find it hard to accept, especially management (You mean you want two programmers doing the job of one?!) so its unsuitable for my needs. Mandatory, Managed Code Reviews is basically where each individual change is reviewed by another member of the team before being committed. Its Mandatory because *all* changes are checked (using the honor system or something like Aegis). Its Managed because people are assigned/take review tasks each day (again, by ad hoc or using something like Aegis). We used it at Blackstar (AFAIK they still do, Tony?) When it worked, it worked great. Problem is if the daily management of the system slipped the whole system became a detriment. Basically, left to ourselves, we would rather code than review. So review tasks piled up, and code couldn't be committed until it passed review. So unless changes were reviewed promptly work would stall. There were other problems. We tended to grab the easy reviews and avoided the hard ones. We tended to use reviews as a crutch to find stupid mistakes and got a bit sloppy when coding. So if the management of the review system gets lax, the whole system comes apart. Therefore it fails the hard to screw up requirement. (You guys solve any of this, Tony?) So I'm looking for something like the above, but that meets my criteria of being easy to learn, teach, accept and hard to screw up. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One gleam comes to my eyes as I combine pure water and triticale. -- mjd
Re: Compiled programs to keep BEGIN blocks?
On Mon, Jan 14, 2002 at 11:13:27AM +0100, Rafael Garcia-Suarez wrote:
On 2002.01.13 22:25 Michael G Schwern wrote:
Why would this:
BEGIN {
push @INC, 'foo';
}
put 'foo' into @INC twice if it were compiled? The compiled program
should not be storing the post-BEGIN value of @INC, it should store
the original value at startup.
The compilation occurs at CHECK-time, that is, after 'foo' has been pushed
into @INC.
I don't know if this is true, but it isn't relevent. Remember, BEGIN,
INIT, CHECK, etc... time is only relevent to the current module being
loaded/run. As this example shows, Bar.pm's code is run before even
Foo.pm's BEGIN block. Replace -MBar with -MO=C and you get the idea.
# ~/tmp/Foo.pm
package Foo;
BEGIN {
push @INC, 'foo';
}
print \@INC as Foo has modified it\n;
print join \n, @INC;
# ~/tmp/Bar.pm
package Bar;
print \@INC as Bar sees it\n;
print join \n, @INC;
$ bleadperl -I/home/schwern/tmp -MBar -wle 'use Foo'
@INC as Bar sees it
/home/schwern/tmp
/usr/local/bleadperl/lib/5.7.2/ppc-linux-64int
/usr/local/bleadperl/lib/5.7.2
/usr/local/bleadperl/lib/site_perl/5.7.2/ppc-linux-64int
/usr/local/bleadperl/lib/site_perl/5.7.2
/usr/local/bleadperl/lib/site_perl
..
@INC as Foo has modified it
/home/schwern/tmp
/usr/local/bleadperl/lib/5.7.2/ppc-linux-64int
/usr/local/bleadperl/lib/5.7.2
/usr/local/bleadperl/lib/site_perl/5.7.2/ppc-linux-64int
/usr/local/bleadperl/lib/site_perl/5.7.2
/usr/local/bleadperl/lib/site_perl
..
foo
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
We're talkin' to you, weaselnuts.
http://www.goats.com/archive/000831.html
Re: Compiled programs to keep BEGIN blocks? (was Re: [RFC] Switch to make Test::Builder output even if $^C ( for B::C ))
On Mon, Jan 14, 2002 at 04:16:49PM +, Piers Cawley wrote: Michael G Schwern [EMAIL PROTECTED] writes: On Mon, Jan 14, 2002 at 10:23:46AM +, Piers Cawley wrote: Um... You're wrong. If you do need 'startup time' initialization then you should do it in an INIT block. If I may quote from the documentation: Like it or not, people put lots of init code into BEGIN blocks, if nothing else for backwards compatiblity with 5.005. perlcc has to compile real world programs. Deferring BEGIN blocks 'til runtime will break rather more realworld program than it fixes I think. Where is deferring involved? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Lord of Kwalitee! Wearer of the CPANTS and Slayer of Pseudo-hashes!
Re: Compiled programs to keep BEGIN blocks?
On Mon, Jan 14, 2002 at 06:45:05PM +0100, Rafael Garcia-Suarez wrote:
# ~/tmp/Bar.pm
package Bar;
print \@INC as Bar sees it\n;
print join \n, @INC;
Nah. You should wrap this code in a CHECK block : otherwise, in
your example, it will be run at BEGIN-time (i.e. when the Bar module
is use'd). That's what O.pm does.
You forget (and I forgot), B::C is encased in the perlcc wrapper
script. So what we really have is this:
print \@INC as perlcc sees it\n;
print join \n, @INC;
system(qq{$^X -cwle 'BEGIN { push \@INC, bar }'});
So perlcc can simply save the state of @INC (and whatever else B::C
might have trouble getting at) and pass that information along.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Maybe they hooked you up with one of those ass-making magazines.
-- brian d. foy as misheard by Michael G Schwern
Compiled programs to keep BEGIN blocks? (was Re: [RFC] Switch to make Test::Builder output even if $^C ( for B::C ))
On Sun, Jan 13, 2002 at 10:04:58PM +0100, Mattia Barbon wrote:
$ bleadperl -MO=-qq,Deparse foo.plx
sub BEGIN {
print foo\n;
}
print bar\n;
If B::Deparse can save BEGIN blocks, B::C can.
I didn't mean that I can't write code to make B::C save BEGIN blocks
( it'd require 10 lines, probably ), I did mean that doing so would
not be a god idea: _I am
saving the state of the program after those blocks were ran_ and
they will be run _another time_ at runtime, and this is not correct:
( for example use lib 'a'; would put 'a' into @INC twice, once
( correctly ) at compile time, and another time ( incorrectly ) at
runtime ). In this
case this is not harmful, but you get the idea.
I don't understand. The compiled program should do exactly what the
original program did. Anything else, as difficult as it may be, is a
bug. Lots of programs and modules do important load-time checks and
logic inside BEGIN blocks.
Why would this:
BEGIN {
push @INC, 'foo';
}
put 'foo' into @INC twice if it were compiled? The compiled program
should not be storing the post-BEGIN value of @INC, it should store
the original value at startup.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Me? A robot? That's rediculous! For one thing, that doesn't compute at all!
Re: [RFC] Switch to make Test::Builder output even if $^C ( for B::C )
On Tue, Jan 08, 2002 at 11:02:27PM +0100, Mattia Barbon wrote:
Code inside the BEGIN blocks has $^C set?? Or are they just not being
run at all? Either of those are bugs. But there's a third
The code inside the begin blocks has $^C set ( as it should ), and they
are being run
( as they should )
The code inside the BEGIN blocks has $^C set *only when compiling with
perlcc* right? If you run the bytecompiled code the BEGIN blocks
should not have $^C set.
Lemme recompile things and try this out.
If you were to take:
BEGIN { print foo }
compile it and run it, would you expect the output to be 'foo'? Or is
the problem that you're getting 'foo' both during the compilation
*and* when its run? As with this:
I'd expect the output of the compiled program to be nothing _normally_
( perlcc foo.pl ),
and foo _if I used the --testsuite switch when calling perlcc_
( perlcc --testsuite foo.pl ).
And in order to not have to introduce hacks in t/TEST, I'd like
that the compiled program resulting from
use Test::Simple tests = 12;
would print nothing normally, and 1..2 _if I passed the --testsuite
switch to perlcc_ .
Now wait a second. Why would the compiled program print nothing
normally? $^C should not be set.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Instant Cross-Platform CGI Programming in Visual Perl 5 with Object Oriented
Design in 7 Days Unleashed for Dummies Special Edition with Exclusive Java
Chapters for Developers. Year 2000 compliant, Enterprise edition and ISO9000-
certified. A Nutshell Handbook Designed For Windows 95/98/NT with a forward by
Larry Bud Melman. Interactive Multimedia CDROM included. 3rd revised editon,
covers Perl5.6.
Of course, it will be refered to by its simple acronym:
ICPCGIPiVP5wOODi7DU4DSEwEJC4DY2KCEedISO9000-cNHD4W9598NTLBMIMCDROM3edP5.6
Re: [PATCH t/run/kill_perl.t t/test.pl] Seperating kill_perl()
On Fri, Jan 11, 2002 at 05:03:46PM +0200, Jarkko Hietaniemi wrote: The name of the function and order of arguments are all up for discussion. Thanks, applied. About the name: I think it's an awful name... doesn't tell me at least at all what it's for (except maybe for doubling as CORE::dump()...). fresh_perl()? kamikaze_perl()? At 5am the imagination is the first to go. I guess the idea is to run a program in a seperate process and compare the output. A cross between runperl() and is(). is_runperl() doesn't seem like a good idea. cmp_runperl? cmp_prog? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One I have this god-awful need to aquire useless crap!!!
[PATCH t/run/kill_perl.t t/test.pl] Seperating kill_perl()
') {
-# some tests will trigger VMS messages that won't be expected
-$results =~ s/\n?%[A-Z]+-[SIWEF]-[A-Z]+,.*//;
-
-# pipes double these sometimes
-$results =~ s/\n\n/\n/g;
-}
-
-$expected =~ s/\n+$//;
-my $ok = $results eq $expected;
-
-unless( $ok ) {
-print STDERR # PROG: $switch\n$prog\n;
-print STDERR # EXPECTED:\n$expected\n;
-print STDERR # GOT:\n$results\n;
-}
-printf %sok %d%s\n, ($ok ? '' : not ), $test,
- length $name ? - $name : $name;
-$test++;
+kill_perl($prog, $expected, { switches = $switch }, $name);
}
__END__
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
It should indeed be said that notwithstanding the fact that I make
ambulatory progress through the umbragious inter-hill mortality slot,
terror sensations will no be initiated in me, due to para-etical phenomena.
[ANNOUNCE] Test::Harness 2.01 now with Test::Harness::Straps example
http://www.pobox.com/~schwern/src/Test-Harness-2.01.tar.gz Test::Harness 2.00 didn't cause any disasters (people are actually using it, right?) so here's 2.01. 2.01 Thu Dec 27 18:54:36 EST 2001 * Added 'passing' to the results to tell you if the test passed * Added Test::Harness::Straps example (examples/mini_harness.plx) * Header-at-end tests were being interpreted as failing sometimes - The 'skip_all' results from analyze* was not being set - analyze_fh() and analyze_file() now work more efficiently, reading line-by-line instead of slurping as before. I'll discuss the Straps example in a seperate post. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One purl Hey, Schwern! THERE IS A HUGE GAZORGANSPLATTEDFARTMONGERING- LIGHTENINGBEASTASAURSOPOD BEHIND YOU! RUN, BEFORE IT GAFLUMMOXES YOUR INNARDLYBITS!
Re: [RFC] Switch to make Test::Builder output even if $^C ( for B::C )
On Sat, Jan 05, 2002 at 10:17:34PM +0100, Mattia Barbon wrote: On Wed, Jan 02, 2002 at 05:37:28PM +0100, Mattia Barbon wrote: I don't care for the variable name, but I'd really like to have this feature. Would it work ok as a Test::Builder accessor method rather than an environment variable? No, unfortunately, because the call order is like this: * init code in the backend * BEGIN blocks of the program ( including use Foo; ) * B::Backend::compile * the coderef returned by B::Backend::compile So I can't call the method from the init code in the backend, because Test::Builder hasn't been loaded yet, and calling it from the ::compile is too late ( output is already lost ); the alternative I see is to use a package variable; so I can set $Test::Builder::i_want_the_output_even_if_dollar_caret_C_is_1 = 1; in initialization code. You could do this: require Test::Builder; my $tb = Test::Builder-new; $tb-output_even_if_compiling(1); in the init code. Since Test::Builder is a singleton your settings should stick. Here's an interesting alternative. Do Clocal $^C = 0 just before running the tests, though that's pretty ugly. But I rwally like the environment variable better, because with the package variable solution I need to set it unconditionally ( because for it to have effect it must be set in the init code, and in the init code I can't look at parameters, because parameters are passed in the call to compile, so I can't set it using a parameter ), and because I was hoping to keep B::C clear from hacks-to-make-the-testsuite-happy. From my PoV, I'm hoping to keep Test::Builder clear from hacks-to-make-perlcc-happy. :) The $^C thing is already a hack for B::Deparse. Could you explain again why you need test output while compiling, I'm not quite following. Assume you had the I_WANT_OUTPUT_DURING_COMPILE environment variable could you show how you'd be using it? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Skrewtape I've heard that semen tastes different depending on diet. Is that true? Skrewtape Hello? Schwern Skrewtape: Hang on, I'm conducting research.
Re: Testing Tests
On Wed, Jan 02, 2002 at 05:10:14PM +, Mark Fowler wrote:
Is there any commonly accepted knowledge about how one should write tests
for Test modules that one might construct using Test::Builder? In
particular, how does one test that one of your new test fails when it
should?
I did have an idea about how nice it would be if you could call a method
in Test::Builder that would cause the next call to ok() to fail if it
should pass and vice versa (and return true where it would have returned
true and false when it would have returned true.)
Since everything winds up using ok() (though that's not officially
documented, I'll do that) you can simply override Test::Builder::ok().
sub My::Builder::ok {
my($self, $test, $name) = @_;
$self-SUPER::ok(!$test, $name);
}
but that only checks that the basic test logic is ok. None
of your diagnostic output is checked.
In order to check diagnostic output...
Redirect the error output to a simple tied filehandle (PRINT is
about all you need) using Test::Builder-failure_output and
todo_output.
Examine the output stored in the tied filehandle to see if its
what you expected.
Test::Simple/More/Builder's t/fail*.t tests do this in varying levels
of contortions. They're a bit brutish since they largely predate
Test::Builder and can't trust it anyway, having to do odd things like
writing its own ok() function.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Do you actually think about what you are saying or is it an improvisational
game of Mad Libs that you play in your head?
Re: [ANNOUNCE] Test::Simple/More/Builder/Tutorial 0.40
On Sat, Dec 22, 2001 at 03:13:12PM -0500, Aaron J Mackey wrote: On Fri, 21 Dec 2001, Michael G Schwern wrote: I can never keep it straight when to put an apostrophe on /it'?s/ It's very simple. If you can replace the its with it is, then there's an apostrophe to denote the contraction. If you need another simple rule, think about this: hers, his, its, not her's, hi's, it's. Thank you. I'll file that away in my brane along with such other important information like the number of times President Ford tripped in public and the words to the Oscar Meyer Bologna song. :P -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One It wasn't false, just differently truthful. -- Abhijit Menon-Sen in [EMAIL PROTECTED]
Re: [ANNOUNCE] Test::Simple/More/Builder/Tutorial 0.40
On Sat, Dec 22, 2001 at 09:48:22AM +0100, Philip Newton wrote: after meditating on this: http://www.angryflower.com/bobsqu.gif That oversimplifies things a bit; the rule possessive things take apostrophe-s doesn't apply to pronouns. Fascinating. Thanks for the patch. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One GuRuThuG make a channel called #Perl, and infest it with joking and fun it doesnt make alot of sense.
Re: is() with arbitrary comparisions
On Tue, Dec 11, 2001 at 01:52:12PM -0500, Kirrily Robert wrote: Are we doing the time warp again, or are the Huskies just tired of pulling the packets across the border? How about: compare($foo, =, $bar) cmp_ok(). Close. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Let's face it, said bearded Rusty Simmons, opening a can after the race. This is a good excuse to drink some beer. At 10:30 in the morning? Well, it's past noon in Dublin, said teammate Mike [Joseph] Schwern. It's our duty. -- Sure, and It's a Great Day for Irish Runners Newsday, Sunday, March 20, 1988
Re: Untested modules update: There's more than we thought
On Mon, Dec 17, 2001 at 08:12:43AM +, Piers Cawley wrote:
What's wrong with
ok ( eval { $foo-isa('Foo') } );
or even:
ok (eval { ref($foo) $foo-isa('Foo') });
As Kurt already pointed out, you can do:
ok( UNIVERSAL::isa($foo, 'Foo') );
but if it fails you have no idea what $foo was.
No you can't. Not if you've overridden isa anywhere. (Which is
perfectly possible.)
Ooooh, hadn't thought about that. I've done it myself a few times (to
make a delegation look like inheritance). Now I have to go fix
Test::More to deal with it. Ya know, I was literally just about to
release 0.40 and I HAD to go read my email first.
Either way, your above ok() solutions still give you no failure
diagnostics, which is the whole point.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
pleasing symmetry
the paste goes in at one end
comes out the other.
-- mjd
Re: HELP: mod_perl and Apache::Cookie
On Sun, Dec 16, 2001 at 04:38:44PM -0500, Philip M. Gollucci wrote: given the following setup: Embedded Perl version v5.6.1 for Apache/1.3.22 (Unix) mod_python/2.7.6 Python/2.1.1 PHP/4.0.6 mod_perl/1.26 process 8458, uname -a FreeBSD xxx.com 4.4-RELEASE FreeBSD 4.4-RELEASE #1: Thu Dec 13 08:25:04 EST 2001 [EMAIL PROTECTED]:/usr/src/sys/compile/PHILIP i386 Someone Please Tell me what I am missing with Apache::Cookie because the folling script works fine if I: replace Apache::Cookie-new($r, with cookie() from CGI snip I think you may have strayed onto the wrong mailing list. This is the list for discussing Perl's QA infrastructure. Figuring out why mod_perl is spitting on you would better be done on one of the mod_perl mailing lists here: http://perl.apache.org/#maillists If anyone one on perl-qa wants to help out, just make sure you don't CC the list. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One There is nothing wrong. We have taken control of this sig file. We will return it to you as soon as you are groovy.
[ANNOUNCE] Test::Simple/More/Builder/Tutorial 0.40
Big release here, folks. Lots of new stuff, knocked off a good chunk of the TODO list. http://www.pobox.com/~schwern/src/Test-Simple-0.40.tar.gz 0.40 Fri Dec 14 15:41:39 EST 2001 * isa_ok() now accepts unblessed references gracefully - Nick Clark found a bug with like() and a regex with % in it. - exit.t was hanging on 5.005_03 VMS perl. Test now skipped. - can_ok() would pass if no methods were given. Now fails. - isnt() diagnostic output format changed * Added some docs about embedding and extending Test::More * Added Test::More-builder * Added cmp_ok() * Added todo_skip() * Added unlike() - Piers pointed out that sometimes people override isa(). isa_ok() now accounts for that. Here's the juicy bits: isa_ok() Handles unblessed refs now - isa_ok([], 'ARRAY'); Better diagnostics. not ok 1 - The object isa Bar # Failed test (-e at line 1) # The object isn't a 'Bar' its a 'Foo' cmp_ok() The is_op() thing I was talking about. Nick, no more need to lie awake at night worrying about testing numbers with is(). cmp_ok($some_horrendous_number, '==', $another_massive_number); # Failed test (-e at line 1) # got: 2420983.14598 # expected: 2420983.145981 In the == case the diagnostics force numeric context, so you can safely do things like: cmp_ok($!, '==', 5); # Failed test (-e at line 1) # got: 2 # expected: 5 cmp_ok($!, 'eq', No such file or martian); # Failed test (-e at line 1) # got: 'No such file or directory' # expected: 'No such file or martian' but it means you also get: cmp_ok(foo, '==', 5); # Failed test (-e at line 1) # got: 0 # expected: 5 which I think is the Right Thing. Thoughts? I've gone through some pains to make the diagnostic output come out just right, but I still think it looks a little ugly in the non 'eq' and '==' cases. cmp_ok(0, , foo)' # Failed test (-e at line 1) # '0' # # 'foo' Let me know what you think. Test::More-builder Used to be to get access to the Test::Builder object underlying Test::Builder you'd just call Test::Builder-new. In order to cover my ass in case I discover that's a bad idea, Test::More-builder is now the official way to get at the underlying object. Though I just realized I left a bunch of Test::Builder-new calls in the tests. unlike() Opposite of like() todo_skip() Combines a todo test with skip's ability to jump whole blocks. Useful for when a todo test can't be run at all because it will die or otherwise cause havoc. Doesn't come up all that often, but I found myself needing it a few times in the core tests. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One GOD made us funky!
Re: [ANNOUNCE] Test::Simple/More/Builder/Tutorial 0.40
On Mon, Dec 17, 2001 at 11:48:34AM +0100, Tels wrote:
What about the problem that can_ok() doesn't increase the testcount number
by the number of methods, but only by one?
Like I said, I tried it that way, didn't like it and changed the
behavior. Found myself having trouble keeping track of the test count
as @methods changed size.
The docs are now explicit about can_ok counting as a single test.
They also offer an alternative if you want the multiple increment
behavior:
No matter how many @methods you check, a single can_ok() call counts
as one test. If you desire otherwise, use:
foreach my $meth (@methods) {
can_ok('Foo', $meth);
}
That'll fail even if @methods is empty, the test count will be wrong.
Contradicts the doc (and the changelog seems to contradict v0.17, since in
v.017 it did increase by 1, too), also makes it hard to see whether can_ok()
really does some looping or simple prints ok $test\n;
The bug with can_ok() succeeding with no @methods has been fixed.
Trust your test library. Or at least trust that things will be fixed
rapidly. :)
Please make can_ok() increase the testcount by the number of methods.
Couldn't now even if I wanted to. Been around too long, would break
too many tests.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
Stupid am I? Stupid like a fox!
[ANNOUNCE] Test 1.19 license change and mod_perl bug fixes
Stas Bekman found a problem with the globals in Test.pm when run under mod_perl (ie. persistently). Test::More probably has the same problem. Also, the license has been changed from Artistic-only to same as Perl. http://www.pobox.com/~schwern/src/Test-1.19.tar.gz 2001-12-17 Michael G Schwern [EMAIL PROTECTED] * Release 1.19 - Resetting globals for mod_perl testing (thanks to Stas Bekman) - License change to be same as perl's -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One I'm spanking my yacht.
[ANNOUNCE] Test::Harness 2.00_05 (RC 3)
http://www.pobox.com/~schwern/src/Test-Harness-2.00_05.tar.gz Things look nice. This release contains only minor fixes, except on VMS where _04 exploded nicely. Should be 100% now, let me know otherwise. I'm trying to get rid of that extra newline in the output. t/00compile ok Its now safe to put -T on your tests in VMS. 2.00_05 Mon Dec 17 22:08:02 EST 2001 * Wasn't filtering @INC properly when a test is run with -T, caused the command line to be too long on VMS. VMS should be 100% now. - Little bug in the skip 'various reasons' logic. - Minor POD nit in 5.004_04 - Little speling mistak -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Hold on while I slip into something a little more naked.
Re: Untested modules update: There's more than we thought
On Sun, Dec 16, 2001 at 02:41:31PM +, Piers Cawley wrote:
Nothing wrong with an adaptor/factory returning something that isn't a
Foo, so long as it has the same interface.
That's why its isa_ok() and not ref_ok().
On the off chance Foo-new is supposed to return something that bears
no relation to Foo at all, then just don't use isa_ok. Or, check that
its that other thing:
my $foo = Foo-new;
isa_ok( $foo, 'Bar' );
The equivalent code without isa_ok() would be:
my $foo = Foo-new;
ok( $foo-isa('Foo') );
except should $foo be unblessed or undef that will explode. You have
to start doing things like:
my $foo = Foo-new;
ok( defined $foo $foo-isa('Foo') );
which rapidly gets tiresome. It also provides you little information
about what value $foo held that caused it to fail.
What's wrong with
ok ( eval { $foo-isa('Foo') } );
or even:
ok (eval { ref($foo) $foo-isa('Foo') });
As Kurt already pointed out, you can do:
ok( UNIVERSAL::isa($foo, 'Foo') );
but if it fails you have no idea what $foo was.
--
Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/
Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One
First day in Prison
Back behind the shower door
Adhesive applied
-- ignatz
Re: lib.t (was Re: Untested modules update: There's more than we thought)
On Sun, Dec 16, 2001 at 07:30:37PM +, Nicholas Clark wrote: I just thought of a better way. Since all we're testing is that lib.pm does the right things to @INC, we can presume that if one of require(), do() or use() works, the rest will work. Can't we just test what @INC now contains by directly reading it? I'm assuming that it's safe for the test to assume that do/require/use work with @INC correctly. Technically, yeah. But Brent's already gone that extra mile. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One Any sufficiently encapsulated hack is no longer a hack.
Re: [ANNOUNCE] Test::Harness 2.00 release candiate 2
On Sun, Dec 16, 2001 at 10:53:14PM +0100, Abe Timmerman wrote: Doh, my bad, I had 2.00_01 hanging around on my system. Works ok on both my regular ActivePerl 5.6.1 (build 628) and freshly compiled bleadperl Yay! Thanks. -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One First day in Prison Back behind the shower door Adhesive applied -- ignatz
Re: Untested modules update: There's more than we thought
On Sun, Dec 16, 2001 at 09:30:18PM -0500, Benjamin Goldberg wrote: Suppose we have RandomThing-new which randomly returns an instance of one of a few dozen different classes, which have no relation at all with each other except a common interface. In such an odd case, don't use isa_ok(). Simp. I think that if all we know about the returned type is that it is supposed to provide some specific interface, it would be more robust to test that the returned thing actually *does* provide the interface. As the ratio of constructors which return objects of random classes to those which produce something a bit more predictable is tiny, I'm not going to start assuming every constructor produces weird objects. Its more robust to do both. Check you got the right type and check the interface. On the off chance that the internals change to start producing randomly typed, yet interface conforming, objects, just change the tests. PS Do you know of anything that does return unpredictably typed objects? -- Michael G. Schwern [EMAIL PROTECTED]http://www.pobox.com/~schwern/ Perl Quality Assurance [EMAIL PROTECTED] Kwalitee Is Job One 36 haikus would scare the shit out of me. Thank goodness for paste! -- Schwern
