Re: [ANNOUNCE] Test::Sims
- Original Message
> From: Michael G Schwern
>
> Ovid wrote:
> > First feature request: automatic Moose support to build random data which
> > conforms to Moose constraints :) (Yes, I know it's much, much harder than
> it sounds).
>
> Hello, what?
package Person;
use Moose;
has name => (is => 'ro', isa => 'Str');
has age => (is => 'ro', isa => 'Int');
...
"name" and "age" must be Str and Int respectively. Each can be easy to break,
but Test::Sims::Moose would be able to:
use Test::Sims::Moose 'random';
Person->new(
name => random('Person.name'),
age => random('Person.age')
);
And that would potentially have issues when it assigns "\t" to $name and -12 to
$age, even those are both valid values for the types in question. It would be
very difficult to find data which automatically fits any random type, but it
could be written for core types and extensible for other types.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: prove is not generating archive when test bails out.
On 29 Jun 2009, at 21:21, Gabor Szabo wrote: When running tests with prove -a file.tar.gz it nicely creates the archive file but if the test bails out the archive file is not created at all. Is this a feature or a bug ? Bug I'd say... -- Andy Armstrong, Hexten
Re: [ANNOUNCE] Test::Sims
Ovid wrote:
> use Test::Sims::Moose 'random';
>
> Person->new(
> name => random('Person.name'),
> age => random('Person.age')
> );
>
> And that would potentially have issues when it assigns "\t" to $name and -12
> to $age
> , even those are both valid values for the types in question.
Then I guess you need better defined types.
Test::Sims isn't so much about generating the random data, that's more
Data::Random is about. So MooseX::Data::Random or something.
--
Robrt: People can't win
Schwern: No, but they can riot after the game.
Calling All Test:: Authors
(Helps if I send this from a subscribed address):
From http://use.perl.org/~Ovid/journal/39193
The latest developer release of Test::More allows subtests. Subtests are great
in that they solve a lot of problems in advanced Perl testing, but they have
required a change in Test::Builder. Previously you could do stuff like this:
package Test::StringReverse;
use base 'Test::Builder::Module';
our @EXPORT = qw(is_reversed);
my $BUILDER = Test::Builder->new;
sub is_reversed ($$;$) {
my ( $have, $want, $name ) = @_;
my $passed = $want eq scalar reverse $name;
$BUILDER->ok($passed, $name);
$BUILDER->diag(<<"END_DIAG") if not $passed;
have: $have
want: $want
END_DIAG
return $passed;
}
1;
And
you've have a simple (untested;) test for whether or not strings are
reversed. The reason that worked is that Test::Builder->new used to
return a singleton. This is no longer true. If someone uses your test library
in a subtest, the above code would break. Instead, you want to do this:
sub is_reversed ($$;$) {
my ( $have, $want, $name ) = @_;
my $passed = $want eq scalar reverse $name;
my $builder = __PACKAGE__->builder;
$builder->ok($passed, $name);
$builder->diag(<<"END_DIAG") if not $passed;
have: $have
want: $want
END_DIAG
return $passed;
}
It's a minor change, it's completely backwards-compatible and it supports
subtests.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Calling All Test:: Authors
>From http://use.perl.org/~Ovid/journal/39193
The latest developer release of Test::More allows subtests. Subtests are great
in that they solve a lot of
problems in advanced Perl testing, but they have required a change in
Test::Builder. Previously you could do stuff like this:
package Test::StringReverse;
use base 'Test::Builder::Module';
our @EXPORT = qw(is_reversed);
my $BUILDER = Test::Builder->new;
sub is_reversed ($$;$) {
my ( $have, $want, $name ) = @_;
my $passed = $want eq scalar reverse $name;
$BUILDER->ok($passed, $name);
$BUILDER->diag(<<"END_DIAG") if not $passed;
have: $have
want: $want
END_DIAG
return $passed;
}
1;
And you've have a simple (untested;) test for whether or not strings are
reversed. The reason that worked is that Test::Builder->new used to return a
singleton. This is no longer true. If someone uses
your test library in a subtest, the above code would break. Instead,
you want to do this:
sub is_reversed ($$;$) {
my ( $have, $want, $name ) = @_;
my $passed = $want eq scalar reverse $name;
my $builder = __PACKAGE__->builder;
$builder->ok($passed, $name);
$builder->diag(<<"END_DIAG") if not $passed;
have: $have
want: $want
END_DIAG
return $passed;
}
It's a minor change, it's completely backwards-compatible and it supports
subtests.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: Calling All Test:: Authors
* Ovid [2009-06-30T10:21:24]
> The latest developer release of Test::More allows subtests. Subtests are
> great in that they solve a lot of problems in advanced Perl testing, but they
> have required a change in Test::Builder. Previously you could do stuff like
> this:
I updated my Test:: libraries to Test::Builder->new in their test routines,
instead, as that's what I thought the original wisdom was. Is that still
okay? (I did not add subtest-specific tests.)
That is, I turned:
my $TEST = Test::Builder->new;
sub is_baloney { $TEST->ok('delicious') }
into:
sub is_baloney {
my $TEST = Test::Builder->new;
$TEST->ok('delicious');
}
--
rjbs
Re: Calling All Test:: Authors
On Tue, Jun 30, 2009 at 10:21 AM, Ovid wrote: > my $BUILDER = Test::Builder->new; I'm running visitcpan to generate a list of offenders now. Results posted soon. -- David
Re: Calling All Test:: Authors
On Tue, Jun 30, 2009 at 11:01 AM, David Golden wrote: > On Tue, Jun 30, 2009 at 10:21 AM, Ovid wrote: >> my $BUILDER = Test::Builder->new; > > I'm running visitcpan to generate a list of offenders now. Results posted > soon. And here we go: http://echo.dagolden.com/~xdg/2009-06-30-test-builder-new.txt Some are certainly false positives, e.g. Test-Simple itself. -- David
Re: Calling All Test:: Authors
- Original Message
> From: Ricardo SIGNES
>
> I updated my Test:: libraries to Test::Builder->new in their test routines,
> instead, as that's what I thought the original wisdom was. Is that still
> okay? (I did not add subtest-specific tests.)
>
> That is, I turned:
>
> my $TEST = Test::Builder->new;
> sub is_baloney { $TEST->ok('delicious') }
>
> into:
>
> sub is_baloney {
> my $TEST = Test::Builder->new;
> $TEST->ok('delicious');
> }
Currently this should be fine as all the builder() method does is call
Test::Builder->new (and that's what I did in Test::JSON, to be honest). I
can't say whether or not this behavior will change in the future. I just used
the information passed to me by Schwern and by the Test::Builder::Module
documentation.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: [ANNOUNCE] Test::Sims
On Jun 30, 2009, at 12:13 AM, Ovid wrote: And that would potentially have issues when it assigns "\t" to $name and -12 to $age, even those are both valid values for the types in question. It would be very difficult to find data which automatically fits any random type, but it could be written for core types and extensible for other types. "\t" might be valid in a name (you never know, people change their names to all sorts of crazy shit). But the age data type should have a constraint on it. Best, David
"Fluent" tests?
Part of the problem with have with 'Test::More' and friends is that there's not
an easy way to attach diagnostics to any test. This is largely because the
interface is set in stone. So I was thinking about a rewritten test interface
which allows something like this (this code actually works, by the way):
use Test::Fluent 'no_plan';
my ( $have, $want ) = ( 1, 1 );
have $have, want $want, reason 'Some test name';
have [ 3, 4 ], want [ 4, 5 ], reason 'cuz I said so'; # fails
true 3, reason '3 had better be true';
false 3, reason '3 had better still better be true';# fails
At the end of each of those, you could append a 'diagnostics' tag which is
required to take a hashref:
my $armageddon = $world->destruction;
false $armaggedon,
reason "World->destruction must never return true",
diagnostic { world => $world };
I don't know that this is a brilliant interface, but just playing around with
it works (the have/want pair automatically falls back to Test::Differences if
$have or $want is a reference).
It's a lot more explicit that Test::More and friends. That means it's a lot
more verbose. However, it's something which could be played with.
Is this a bad idea? Are there other suggestions people want? If we had a dot
operator, it would be clean to write this:
have($have).want($want).reason($some_reason);
But we don't, so we'd have to fall back on the ugly:
have($have)->want($want)->reason($some_reason);
Though I could easily overload the dot operator to get the first syntax.
Or if we needed delayed evaluation for some reason:
have { $have }, want { $want }, reason $some_reason);
That's also ugly and requires the (&) prototype (and friends).
Thoughts? Am I totally smoking crack here? If there's a clean way to shoehorn
diagnostics on the Test::More-style interface, I guess that would be ok.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: "Fluent" tests?
On Tue, Jun 30, 2009 at 1:17 PM, Ovid wrote: > Thoughts? Am I totally smoking crack here? If there's a clean way to > shoehorn diagnostics on the Test::More-style interface, I guess that would be > ok. Doesn't Test::Builder2 address this? I'd rather see more energy directed at getting that done. - David
Re: "Fluent" tests?
On Tue, Jun 30, 2009 at 6:17 PM, Ovid wrote:
>
> Part of the problem with have with 'Test::More' and friends is that there's
> not an easy way to attach diagnostics to any test. This is largely because
> the interface is set in stone. So I was thinking about a rewritten test
> interface which allows something like this (this code actually works, by the
> way):
>
> use Test::Fluent 'no_plan';
>
> my ( $have, $want ) = ( 1, 1 );
> have $have, want $want, reason 'Some test name';
> have [ 3, 4 ], want [ 4, 5 ], reason 'cuz I said so'; # fails
> true 3, reason '3 had better be true';
> false 3, reason '3 had better still better be true'; # fails
>
>
> At the end of each of those, you could append a 'diagnostics' tag which is
> required to take a hashref:
>
> my $armageddon = $world->destruction;
> false $armaggedon,
> reason "World->destruction must never return true",
> diagnostic { world => $world };
>
> I don't know that this is a brilliant interface, but just playing around with
> it works (the have/want pair automatically falls back to Test::Differences if
> $have or $want is a reference).
>
> It's a lot more explicit that Test::More and friends. That means it's a lot
> more verbose. However, it's something which could be played with.
>
> Is this a bad idea? Are there other suggestions people want? If we had a
> dot operator, it would be clean to write this:
>
> have($have).want($want).reason($some_reason);
>
> But we don't, so we'd have to fall back on the ugly:
>
> have($have)->want($want)->reason($some_reason);
>
> Though I could easily overload the dot operator to get the first syntax.
>
> Or if we needed delayed evaluation for some reason:
>
> have { $have }, want { $want }, reason $some_reason);
>
> That's also ugly and requires the (&) prototype (and friends).
>
> Thoughts? Am I totally smoking crack here? If there's a clean way to
> shoehorn diagnostics on the Test::More-style interface, I guess that would be
> ok.
>
> Cheers,
> Ovid
> --
> Buy the book - http://www.oreilly.com/catalog/perlhks/
> Tech blog - http://use.perl.org/~Ovid/journal/
> Twitter - http://twitter.com/OvidPerl
> Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
>
>
At the hackathon Schwern and I had a go at implementing the ugly ->
type interface for Test::Builder2 based on an idea from a PDX.pm
meeting.
The idea being that it would work something like this,
ok($func('a'), 'some test')->diag('some cheap diagnostics');
or for delayed diagnostics,
{
my $result = ok $func('a'), 'A test';
if(!$result) {
$result->diag("expensive diagnostics we'd rather not run");
}
}
The idea being that these result objects would then be usable by
modules like Test::More and you would be able to easily attach
diagnostics in the right place.
Note that the block in the second example is a necessary evil for the
delayed diagnostics because of the way we make sure we don't produce
the output before we know what diagnostics come with the test.
I did have a go at writing this up at
http://colinnewell.wordpress.com/2009/03/31/perl-qa-hackathon-testbuilder2/.
Colin.
Re: "Fluent" tests?
- Original Message > From: David Golden > > On Tue, Jun 30, 2009 at 1:17 PM, Ovidwrote: > > Thoughts? Am I totally smoking crack here? If there's a clean way to > shoehorn diagnostics on the Test::More-style interface, I guess that would be > ok. > > Doesn't Test::Builder2 address this? I'd rather see more energy > directed at getting that done. Well, I see this as conflating two issues. First, we've waited years for TB2 and we might have to continue waiting years for it. Second, can we create a better interface? I don't think people should sit around and have to wait for other authors. Schwern and Colin have done good work, but there's nothing there yet and I know Schwern is as bad as I am about pushing out new code :) Also, I think playing around with more fluent interfaces is a good idea. If my interface is great, why not? If it's bad, what would people *love* to see in a test interface which allows them to naturally write tests? Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog- http://use.perl.org/~Ovid/journal/ Twitter - http://twitter.com/OvidPerl Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Fw: "Fluent" tests?
I forgot to hit 'reply all' :)
Also, I had considered this:
have $some_value, assuming { shift > 7 }, reason "Argument must be greater
than 7";
And that would allow us to naturally put complex constraints onto the values.
Cheers,
Ovid
--
Buy the book - http://www.oreilly.com/catalog/perlhks/
Tech blog- http://use.perl.org/~Ovid/journal/
Twitter - http://twitter.com/OvidPerl
Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
- Forwarded Message
> From: Ovid
> To: Colin Newell
> Sent: Tuesday, 30 June, 2009 19:17:54
> Subject: Re: "Fluent" tests?
>
> - Original Message
>
> > From: Colin Newell
> >
> > > my $armageddon = $world->destruction;
> > > false $armaggedon,
> > > reason "World->destruction must never return true",
> > > diagnostic { world => $world };
> > >
> > > I don't know that this is a brilliant interface, but just playing around
> with
> > it works (the have/want pair automatically falls back to Test::Differences
> > if
> > $have or $want is a reference).
> > >
> > > It's a lot more explicit that Test::More and friends. That means it's a
> > > lot
>
> > more verbose. However, it's something which could be played with.
> >
> > At the hackathon Schwern and I had a go at implementing the ugly ->
> > type interface for Test::Builder2 based on an idea from a PDX.pm
> > meeting.
> >
> > The idea being that it would work something like this,
> >
> > ok($func('a'), 'some test')->diag('some cheap diagnostics');
> >
> > or for delayed diagnostics,
> >
> > {
> > my $result = ok $func('a'), 'A test';
> > if(!$result) {
> > $result->diag("expensive diagnostics we'd rather not run");
> > }
> > }
> >
> > The idea being that these result objects would then be usable by
> > modules like Test::More and you would be able to easily attach
> > diagnostics in the right place.
> >
> > Note that the block in the second example is a necessary evil for the
> > delayed diagnostics because of the way we make sure we don't produce
> > the output before we know what diagnostics come with the test.
> >
> > I did have a go at writing this up at
> > http://colinnewell.wordpress.com/2009/03/31/perl-qa-hackathon-testbuilder2/.
>
> I appreciate the work you've done and I think it would be an improvement over
> what we have. However, rather than settle for "good", I'm wondering what
> would
> be "great"? I certainly don't know that I have the answer here, so I would
> love
> to see what people think is an INCREDIBLE interface for writing tests.
>
>
> Cheers,
> Ovid
> --
> Buy the book - http://www.oreilly.com/catalog/perlhks/
> Tech blog- http://use.perl.org/~Ovid/journal/
> Twitter - http://twitter.com/OvidPerl
> Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: "Fluent" tests?
Ovid wrote:
use Test::Fluent 'no_plan';
my ( $have, $want ) = ( 1, 1 );
have $have, want $want, reason 'Some test name';
have [ 3, 4 ], want [ 4, 5 ], reason 'cuz I said so'; # fails
true 3, reason '3 had better be true';
false 3, reason '3 had better still better be true';# fails
I would much rather see something like
cmp_ok(
have => 1,
want => 2,
reason => 'Some test name',
diagnostic => { world => $world },
);
Much more Perlish. I've always disliked some APIs where it's not immediately
clear what's a function call and what's an argument to that function: is
reason() an argument to want() or have()?. It also seems more obvious for doing
things like data-driven tests where you just have a large data structure that
tests are run against.
--
Michael Peters
Plus Three, LP
Re: "Fluent" tests?
On Tue, Jun 30, 2009 at 2:15 PM, Ovid wrote:
> Also, I think playing around with more fluent interfaces is a good idea. If
> my interface is great, why not? If it's bad, what would people *love* to see
> in a test interface which allows them to naturally write tests?
Well, if you're doing interface design, one of the first things that
comes to mind is that the name of the test should come first, not
last.
I've often taken to writing tests like this:
is( $have, $want
"label goes here"
);
This means I can skim down the screen and all of the test labels line
up nicely. So I'm in favor of something along the lines of
test "label goes here" => is( $have, $want ) => diag $diag;
Which could work nicely for nesting, too
test "a subtest label" => nest {
# sub tests here
};
It would be test( $label, @objects ), where objects are executed in
some sensible order (e.g. diag objects only called if assertion object
returns false).
It could get quite involved, if one really wanted
test "label" => is( $have, $want ) => setup \&setup => teardown
\&teardown => diag \&diag => note \¬e;
Then creating new tests is just a matter of creating assertion
classes. (Assertion classes could return diagnostic objects as well,
perhaps).
-- David
Re: "Fluent" tests?
On Tue, Jun 30, 2009 at 2:46 PM, David Golden wrote: > On Tue, Jun 30, 2009 at 2:15 PM, Ovid wrote: >> Also, I think playing around with more fluent interfaces is a good idea. If >> my interface is great, why not? If it's bad, what would people *love* to >> see in a test interface which allows them to naturally write tests? > > Well, if you're doing interface design, one of the first things that > comes to mind is that the name of the test should come first, not > last. > > I've often taken to writing tests like this: > > is( $have, $want > "label goes here" > ); > how about fn_label_goes_here_is ($have, $want, "maybe a supplememtary msg") with AUTOLOAD catching it, recognize the _is suffix, convert the fn-label-goes-here into a more readable message. + suffix carries exactly current meaning (no grey area) + test label and implementing function are ALWAYS clearly associated. (no more counting $i++'s) + name prefix can be guaranteed unique (by erroring on reused prefixes) or encouraged (with warning) or tacitly done (with _N_ infix) or some more subtle combo - limited mostly by need for a clear description. - AUTOLOAD is its own kind of ugliness.. - dont really want such an enormous dependence on a not-quite-basic feature.
