Re: Test::Builder feature request

[Nadim Khemir]

Hi,

On Friday 12 January 2007 01:49, [EMAIL PROTECTED] wrote:
> You don’t have to use objects to get the same end effects as mocking
> objects.

Right!

Now, as my "devil's advocate"  signature tried to show, this thread is for the 
fun of the discussion. I'm sure all of use, and Ovid more than the rest of 
us, knows what good quality code should look like. I was just wondering if we 
should be flexible, which could be a quality, or not flexible and be 
more "right" though that could mean loosing some of the perlishness we all 
like.

Cheers, Nadim.

Module::Build and the Test Harness Report in the CPAN Testers

[Shlomi Fish]

Hi all!

In this message to Israel.PM:

http://perl.org.il/pipermail/perl/2006-September/008165.html

Yuval Kogman claimed that Module::Build generates CPAN testers reports with no 
output from the test harness. Now, I released Math::GrahamFunction, which is 
based on Module::Build, a few days ago and today received this failure error 
report:

http://nntp.x.perl.org/group/perl.cpan.testers/397523

And indeed it does not contain the output of the test harness.

Is this a known problem with Module::Build? Is it intentional, or are people 
otherwise working on resolving it? Is there anything I can do about it?

Regards,

Shlomi Fish

-
Shlomi Fish  [EMAIL PROTECTED]
Homepage:http://www.shlomifish.org/

Chuck Norris wrote a complete Perl 6 implementation in a day but then
destroyed all evidence with his bare hands, so no one will know his secrets.

Re: Module::Build and the Test Harness Report in the CPAN Testers

[David Golden]

On 1/12/07, Shlomi Fish <[EMAIL PROTECTED]> wrote:

Yuval Kogman claimed that Module::Build generates CPAN testers reports with no
output from the test harness. Now, I released Math::GrahamFunction, which is
based on Module::Build, a few days ago and today received this failure error
report:

http://nntp.x.perl.org/group/perl.cpan.testers/397523

And indeed it does not contain the output of the test harness.

Is this a known problem with Module::Build? Is it intentional, or are people
otherwise working on resolving it? Is there anything I can do about it?


It's a known problem with CPANPLUS support for Module::Build and
CPANPLUS is used by the smoke tester.

CPAN.pm and CPAN::Reporter capture output from Module::Build based
distributions just fine.

Regards,
David

Re: Module::Build and the Test Harness Report in the CPAN Testers

[Sébastien Aperghis-Tramoni]

Shlomi Fish wrote:


Hi all!


Hello Shlomi,

Yuval Kogman claimed that Module::Build generates CPAN testers  
reports with no
output from the test harness. Now, I released Math::GrahamFunction,  
which is
based on Module::Build, a few days ago and today received this  
failure error

report:

http://nntp.x.perl.org/group/perl.cpan.testers/397523

And indeed it does not contain the output of the test harness.

Is this a known problem with Module::Build? Is it intentional, or  
are people
otherwise working on resolving it? Is there anything I can do about  
it?


This is a known issue. See ticket CPAN-RT#9793

  » http://rt.cpan.org/Public/Bug/Display.html?id=9793

--
Sébastien Aperghis-Tramoni

Close the world, txEn eht nepO.

Test Coverage vs. Refactoring

[Shlomi Fish]

As I've been increasing the test coverage of XML::RSS, something has occured 
to me. Let's suppose we have the following pseaudo-code:

<<
sub func1
{
my ($self, @params) = @_;

.
.
.
if (COND)
{
$self->do_something(...);
}
.
.
.
}

sub func2
{
my ($self, @params) = @_;

.
.
.
if (COND)
{
$self->do_something(...);
}
.
.
.
}
>

Now, in order to achieve full test coverage we need to cover both (identical) 
conditional in func1 as well as func2. However, after we refactor the code to 
extract the conditional, we get the following code:

<<

sub handle_cond
{
my ($self, ...) = @_;
if (COND)
{
$self->do_something(...);
}
}

sub func1
{
my ($self, @params) = @_;

.
.
.
$self->handle_cond(...);
.
.
.
}

sub func2
{
my ($self, @params) = @_;

.
.
.
$self->handle_cond(...);
.
.
.
}

>>

Now we only have to cover the outcomes of the condition in only one function. 
(or two different outcomes in different functions). So our test coverage is 
not as comprehensive as the previous one.

This is naturally a limitation of test coverage in general which only checks 
coverage for individual code atoms, and not for all the different paths of 
execution (their Cartesian products).

Right now an ad-hoc solution is to make sure that one has a good test coverage 
before one refactors the code. But I think the philosophical problem is also 
interesting.

Regards,

Shlomi Fish


-
Shlomi Fish  [EMAIL PROTECTED]
Homepage:http://www.shlomifish.org/

Chuck Norris wrote a complete Perl 6 implementation in a day but then
destroyed all evidence with his bare hands, so no one will know his secrets.

Re: Test Coverage vs. Refactoring

[Ovid]

--- Shlomi Fish <[EMAIL PROTECTED]> wrote:

> This is naturally a limitation of test coverage in general which only
> checks 
> coverage for individual code atoms, and not for all the different
> paths of 
> execution (their Cartesian products).

Regrettably, the while code coverage is generally fairly easy to
achieve, path coverage is very difficult.  It's a common problem.  I
know of one gentleman who described his problem of having a Chain of
Responsibility pattern where every step of the chain, properties stored
in a database could alter the outcome.  There were, at last count,
50,000 properties in the database.  Code coverage was the easy bit.

Cheers,
Ovid

--

Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/

Re: Test::Builder feature request

[Michael G Schwern]

chromatic wrote:
> (I know; it's not exactly what you were asking.  I just wanted to get that in 
> a public mailing list so I could call that the "Star Trek: Generations" 
> fallacy.  You steal a spaceship, which flies through space, to fly through 
> space to a planet, flying through space, where a temporal anomaly, which also 
> flies through space, deflected by a supernova, which you flew through space 
> in your spaceship which flies through space, passes close enough to the 
> planet, because both of them fly through space, that you can jump off a 
> bridge into it.)

Its all tachyons, man.


> I'm sympathetic to the argument that your test file is going to be ugly when 
> the code you're testing is ugly, but it seems to me that containing that 
> ugliness within the specific test file until you can refactor the test file 
> is much better than bolting another ad-hoc feature onto a testing system 
> which already makes way to many underspecified assumptions and would be 
> fairly difficult to replace with something nicer, someday.

I agree.  There's a simple, existing solution to this problem and its not one 
we want to encourage anyway.  The situation would be different if the same 
people didn't control the tests and the code being tested, and thus required 
some sort of coordination, but they do control both sides.

And it won't work anyway.

Having $ENV{RUNNING_TESTS} = 1 should presumably indicate that we're running in 
a testing environment.  Ok, but how does Test::Builder know that?  Answer: it 
doesn't.  Loading Test::Builder does not imply tests are being run.  An example 
of this, off the top of my head, is Jifty which always loads Test::Builder (for 
hacky reasons, I admit).  I'm sure with a little thought one could find more, 
but even if not I do not want to the loading of Test::Builder to imply we're 
running in a test environment.  It limits Test::Builder.

Ok, what about setting it when a plan is initiated?  Surely that means tests 
are being run?  Consider Test::AtRuntime, a Test::Builder derived module 
designed to embed tests in the code to be executed during normal operations to 
help catch and diagnose errors.  Basically an extension of the run-time 
assert() concept.  Test::Builder is loaded.  There's a plan.  Test functions 
are being executed.  But the system is operating normally.  A similar problem 
would occur if I ever got around to moving Carp::Assert over to using 
Test::Builder for its guts.

This might seem like nit-picking for a corner case, but Test::Builder is 
designed to be generic.  The more automatic "convenience" features which get 
built into it the less generic it is.  The less generic it is the less neato 
testing modules can be spawned from it.  So I push back at them and encourage 
them to be built in one level up on top of Test::Builder.

Re: Test Coverage vs. Refactoring

[A. Pagaltzis]

* Ovid <[EMAIL PROTECTED]> [2007-01-13 00:25]:
> I know of one gentleman who described his problem of having a
> Chain of Responsibility pattern where every step of the chain,
> properties stored in a database could alter the outcome. There
> were, at last count, 50,000 properties in the database. Code
> coverage was the easy bit.

First thought: Test::LectroTest. Still not easy, but makes the
problem more tractable.

Regards,
-- 
Aristotle Pagaltzis //