Re: Summarizing the pod tests thread

[A. Pagaltzis]

* Adam Kennedy <[EMAIL PROTECTED]> [2007-08-01 05:25]:
> David Golden wrote:
> >* Alternatives -- Eric Wilhelm suggests putting these tests in
> >at/ and running them with prove.  Likewise, Module::Build and
> >ExtUtils::MakeMaker could have targets added that run tests in
> >an "at/" directory as another way to encourage the practice.
> >(Module authors who want such targets can upgrade without ever
> >affecting end-users)  Lobby Ken Williams and Schwern.
> 
> A small nit, if we end up converging around the use of a
> relatively standard directory name, can we at least use
> something relatively self-evident? For example "author/"?

I’d suggest `aut_t`. Shorter than any alternative and still more
self-explanatory than most.

-- 
*AUTOLOAD=*_;sub _{s/(.*)::(.*)/print$2,(",$\/"," ")[defined wantarray]/e;$1}
&Just->another->Perl->hack;
#Aristotle

Re: Fixing the damage caused by has_test_pod

[A. Pagaltzis]

* Andy Lester <[EMAIL PROTECTED]> [2007-07-30 22:30]:
> On Jul 30, 2007, at 2:46 PM, A. Pagaltzis wrote:
> >
> >>Certainly, the intent is to have boilerplate.t NOT get
> >>shipped with  the module.
> >
> >Disagree. The tarball should contain the full source necessary
> >to recreate everything the author did. Stuff that’s not
> >relevant to end-user installation of the module should simply
> >be kept out of the way.
> 
> You can't disagree.  It's a statement of fact that RJBS and I
> did not  intend for boilerplate.t to actually get shipped with
> the module.

I can’t disagree with the fact that this is what you intended,
but I can and do disagree with what you intended.

If the original author feels the boilerplate test is no longer
necessary, then not shipping the file is an automatic consequence
of deleting the file from his own source tree. If the original
author feels the boilerplate test might be useful again in the
future, and hence does not delete it from his own source tree,
then it should ship with the distro as an author test.

There is no good reason to special-case the boilerplate test.

Regards,
-- 
Aristotle Pagaltzis // 

Re: Summarizing the pod tests thread

[A. Pagaltzis]

* A. Pagaltzis <[EMAIL PROTECTED]> [2007-08-03 09:08]:
> I’d suggest `aut_t`.

Err, `auth_t`.

Regards,
-- 
Aristotle Pagaltzis // 

Re: Revising the Perl Testing quick reference card

[Adrian Howard]

On 2 Aug 2007, at 19:05, nadim khemir wrote:
[snip]

Test::Block

[snip]

I (personally) don't think T::B is generally useful enough. These  
days I'd push things like Test::Group / Test::Class instead.


Cheers,

Adrian

Re: Tests involving regexes

[Fergal Daly]

Not a solution to your problem but I just want to point out that the
correct way to write regex tests is

is($re, qr/what I expected/);

and that this will never be upset by changes like demerphq is describing,

F

On 03/08/07, demerphq <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I thought id bring a tricky subject that ive encountered in Perl core
> development up here so you testing mavens can have a look at the
> problem and maybe come up some suggestions for me.
>
> The problem is this: one of the main obstacles for adding new regex
> flags to the perl regex engine are the oodles of tests that expect
> stringified regular expressions to look like:
>
> /\A\(\?[msix-]*:.*\)\z/s
>
> There are also lots of tests out there that do thing like:
>
> is("$qr","(?ms-ix:...)",'Got back the regex we expected');
>
> all of these break if I add a new modifier or change the flag layout
> in any way (it was surprising how many test failures were caused by
> reordering the flags). With Perl 5.9.4 or so there is a new modifier
> 'p' (for 'preserve') but various special properties of what it
> represents mean I was able to bypass the problem as it only shows up
> in the list if its on, there is no 'off' form of it. However for a
> true boolean flag this wont work.
>
> So assuming we want to add new modifiers (we kinda do) how should this
> be addressed? Both going forward and dealing with legacy code.
>
> BTW I recall that a notable area in the core which does this type of
> naughty testing is actually the Test:: modules themselves. :-)
>
> Im open to suggestions that dont involve hunting down all of these
> naughty tests and fixing them. Including stuff that doesnt strictly
> involve test modules.
>
> Cheers,
> Yves
>
>
> --
> perl -Mre=debug -e "/just|another|perl|hacker/"
>

Re: running author tests

[David Golden]

On 8/2/07, Andy Armstrong <[EMAIL PROTECTED]> wrote:
> On 2 Aug 2007, at 23:53, Eric Wilhelm wrote:
> > I think a new toplevel (e.g. ./author_t or ./author_test) directory is
> > going to be most compatible.
>
> +1

Agree, +1 for "author_t" -- using "t" rather than "test" is a better
parallel to the existing usage of "t".

David

Re: Summarizing the pod tests thread

[David Cantrell]

Salve J Nilsen wrote:


Bug #1: The module build output text is too verbose. (Hiding the detailed
output would be useful.)


It certainly is, especially when there's XS things being built.  The 
vast majority of the time all you're interested in is failures, no-one 
cares that yet another file compiled just fine, or what weird 
incantation was needed to make that happen.  Nor do normal people care 
about what files were extracted from the tarball, where they were copied 
to, and so on.  In fact, all I usually care about is:

  what dependencies have been found;
  the test results

Bug #4: There isn't a sufficiently clear test output summary telling Joe 
which
module broke the dependency chain - so he can't look into it 
himself.
(Visualizing the dependencies and show where it broke may help. 
Maybe

displaying the relevant dependencies in a way like tree(1) does?)


That would be very useful, both for non-perl people installing stuff, 
and also for authors, especially if the CPAN testers were to send that 
stuff to people whose modules don't get tested because of pre-requisites 
that failed.


I hope you don't mind, but I've just posted to cpan-testers about this, 
quoting you.


Bug #5: There's no simple way available Joe to report/post the failed 
test to someone who cares. (It may help asking if the test failures 
should be reported, possibly resulting in the installation of 
Test::Reporter and it picking up the previous Build.log files)


CPAN::Reporter makes this easy. [author CCed]

But for this to work, one has to lower the "barrier of entry" for giving 
feedback, and up the quality of the base level of feedback offered.


Are you suggesting that some kind of test reporting mechanism should be 
core in 5.10?  It's a nice idea - it's one of very *very* few things 
that I'd not object to having added to the core.


Additionally, it's fallacious to assume that authors are either 
helpful, proactive, competant, or even (unfortunately in some cases) 
alive.
This is true, and I'm actually assuming that the authors ARE all these 
things (or at the very least _wants_ to be these things.)


Many authors *are* helpful, proactive, competent and alive.  That 
includes most of the authors of the most important modules that I've 
reported failures in.  Sadly, a lot of authors seem to be indifferent 
based on how I keep seeing the same failures again and again and again 
while smoke-testing CPAN.  Only a tiny minority of authors are 
deliberately unhelpful, or are incompetent or dead.


Would it make sense to find a way to make it easier to "take over" 
projects that are obviosly standing still?


It's already easy, and there were some mumblings on another perly list 
recently (module-authors, IIRC) about documenting it better so it stops 
appearing to be hard.


--
David Cantrell

Tests involving regexes

[demerphq]

Hi,

I thought id bring a tricky subject that ive encountered in Perl core
development up here so you testing mavens can have a look at the
problem and maybe come up some suggestions for me.

The problem is this: one of the main obstacles for adding new regex
flags to the perl regex engine are the oodles of tests that expect
stringified regular expressions to look like:

/\A\(\?[msix-]*:.*\)\z/s

There are also lots of tests out there that do thing like:

is("$qr","(?ms-ix:...)",'Got back the regex we expected');

all of these break if I add a new modifier or change the flag layout
in any way (it was surprising how many test failures were caused by
reordering the flags). With Perl 5.9.4 or so there is a new modifier
'p' (for 'preserve') but various special properties of what it
represents mean I was able to bypass the problem as it only shows up
in the list if its on, there is no 'off' form of it. However for a
true boolean flag this wont work.

So assuming we want to add new modifiers (we kinda do) how should this
be addressed? Both going forward and dealing with legacy code.

BTW I recall that a notable area in the core which does this type of
naughty testing is actually the Test:: modules themselves. :-)

Im open to suggestions that dont involve hunting down all of these
naughty tests and fixing them. Including stuff that doesnt strictly
involve test modules.

Cheers,
Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Failures in pre-requisites

[David Cantrell]

I wrote:
> Salve J Nilsen wrote:
> > Bug #4: There isn't a sufficiently clear test output summary telling
> > Joe which module broke the dependency chain - so he can't look into
> > it himself.  Visualizing the dependencies and show where it broke
> > may help.  Maybe displaying the relevant dependencies in a way like
> > tree(1) does?
> That would be very useful, both for non-perl people installing stuff, 
> and also for authors

David Golden thought that this might be more useful as a web thing than
done by email by the CPAN testers, so I hacked something up:
  http://cpandeps.cantrell.org.uk/

It's a foul, disgusting hack.  If anyone wants to make it less sucky (eg
to work off local databases of test results instead of just naively
web-scraping) then the source is available too.

Try it with something like Catalyst to see the test results for half of
the CPAN.

-- 
David Cantrell | Official London Perl Mongers Bad Influence

  There once a a tramp with "enable"
  Whose router used proprietary cables.
When he got pissed on meths,
He thought "screw IOS",
  "Let's apt-get install iptables"

Re: Tests involving regexes

[Michael G Schwern]

demerphq wrote:
> So assuming we want to add new modifiers (we kinda do) how should this
> be addressed? Both going forward and dealing with legacy code.

My first inclination is that what you want is a way to get at the underlying
structure of the regex so you can query what flags there are on it directly
rather than relying on the string.  I wave my hands in the direction of XS.

Another option is to write a parser for qr strings to do the same thing.

Or you can get sneaky.  Since what you really want is a way to say "All the
usual flags and things, plus $foo" you can compile a very simple regex and use
that to divine the default flags and their order for the version of perl being
run.  Then you can do, a subtractive diff *observe the hands waving like so*
to remove the defaults and leave only what you expect.

Re: Failures in pre-requisites

[Michael G Schwern]

David Cantrell wrote:
> David Golden thought that this might be more useful as a web thing than
> done by email by the CPAN testers, so I hacked something up:
>   http://cpandeps.cantrell.org.uk/
> 
> It's a foul, disgusting hack.  If anyone wants to make it less sucky (eg
> to work off local databases of test results instead of just naively
> web-scraping) then the source is available too.
> 
> Try it with something like Catalyst to see the test results for half of
> the CPAN.

Pretty!

Re: Failures in pre-requisites

[Ricardo SIGNES]

* David Cantrell <[EMAIL PROTECTED]> [2007-08-03T16:21:44]
> David Golden thought that this might be more useful as a web thing than
> done by email by the CPAN testers, so I hacked something up:
>   http://cpandeps.cantrell.org.uk/

That's pretty cool, even if it is a foul, disgusting hack.

-- 
rjbs