Re: cpantesters - why exit(0)?
Christopher H. Laco wrote: chromatic wrote: On Tuesday 02 September 2008 11:01:44 David Golden wrote: You encourage what you measure, In theory, yes. In practice, that hasn't been the experience to date. Testers over 70K: 1 587018 Chris Williams (BINGOS) 2 318527 Andreas J. König (ANDK) 3 188392 David Golden (DAGOLDEN) 4 151457 David Cantrell (DCANTRELL) 5 148505 Slaven Rezić (SREZIC) 6 73425 Jost Krieger (JOST) 7 73104 Yi Ma Mao (IMACAT) Do you think this group couldn't game the stats if all they wanted was a high score? Being snide about peoples volunteer efforts isn't particularly constructive. Someone in that top seven has sent plenty of useless reports. (Hi, I'm from CPAN Testers! I have my client configured not to install required dependencies! Your distribution doesn't work! Hope that helps!) If you think that people should be rewarded (acknowledged?) for useful reports, start defining useful and the heuristics you'd use to identify them. * Does the report identify an actual failure for the common use case of CPAN installation or does it identify a failure in configuring the CPAN Testers client? * Does the report identify a known failure already reported elsewhere with the same characteristics? * Does the report identify a success on a previously unknown platform/configuration combination? * Does the platform combination include a supported version of Perl? My criteria for usefulness suggest answers of Yes. No. No. Yes. Yes. I realize that the third question is more difficult to answer in the presence of XS components, but most of the distributions on the CPAN are pure Perl. -- c * Does the report actually include the error at all? useful: A CPAN testers FAIL report that actually includes the failure it signifies. http://www.nntp.perl.org/group/perl.cpan.testers/2008/08/msg2060496.html http://www.nntp.perl.org/group/perl.cpan.testers/2008/08/msg2060470.html Yes, the second one does error in the middle of the output...barely. Had the errors been after the 50k, the report would be doubly useless: [Output truncated after 50K] does no good when the sole purpose of such reports is to r1eport errors from systems you aren't looking at, or even have locally to test with.
Re: cpantesters - why exit(0)?
David Golden wrote: On Tue, Sep 2, 2008 at 2:23 PM, Christopher H. Laco [EMAIL PROTECTED] wrote: Yes, the second one does error in the middle of the output...barely. Had the errors been after the 50k, the report would be doubly useless: [Output truncated after 50K] does no good when the sole purpose of such reports is to r1eport errors from systems you aren't looking at, or even have locally to test with. The report truncation will go away when we get CPAN Testers off of email. There was some discussion of what percentage of reports are over 50K and it's pretty darn small. Sorry you were (almost) on the unhappy end of that. -- David That's normal. I have this knack for being the person that falls into the ditch rather than just walking safely over the bridge. :-) -=Chris
Re: cpantesters - why exit(0)?
Aristotle Pagaltzis wrote: * Ovid [EMAIL PROTECTED] [2008-09-02 22:40]: For all of the bogus reports I get, I'd rather get the bogus ones along with the good ones than nothing at all. I'd much prefer that I find out immediately if there's a disaster rather than have someone email me and say this broke our software! ++ Regards, Hell, I'd be happy if someone just *used* my stuff. :-)
Re: Test::Harness Output Change
Andy Lester wrote: On Aug 21, 2008, at 4:36 AM, Ovid wrote: Why? If we want other extensions, stripping them is probably bad. We definitely want other extensions. I have a pending project that relies on running .t and .phpt next to each other. xoa I've got one at home now that also has .rb files... Why .phpt instead of .php?
Re: The spewing problem.
Ovid wrote:
--- Sam Vilain [EMAIL PROTECTED] wrote:
However I
am yet to see anything other than a monkeypatch on Ovid's journal,
and an incomplete patch linked earlier on the thread.
Care to explain the term 'monkeypatch'?
I've always heard of it as injecting code into someone elses [broken]
module to get things fixed or changed when subclassing is a no go:
package MyStuff;
use Someone::Elses::Junk;
*Someone::Elses::Junk::foo = sub {
};
sub foo {
my $obj = Someone::Elses::Junk-new;
$obj-foo(...);
};
Give or take my bad example thereof.
signature.asc
Description: OpenPGP digital signature
Re: Proc::Fork on Windows
A. Pagaltzis wrote: * David Golden [EMAIL PROTECTED] [2007-12-19 17:30]: It looks like it does (as long as you remove the -T from the shebang in 01.real.t. Anyone with Windows who cares to, please bang on http://plasmasturm.org/attic/Proc-Fork-0.5.tar.gz before I send it to the CPAN. Thanks again for tracking this down, David. Regards, All tests pass: Strawberry Perl 5.8.8/XP -=Chris signature.asc Description: OpenPGP digital signature
Re: Running CPAN as a regular user, installing as root
Matisse Enzer wrote: I was following the comments in http://use.perl.org/~Alias/journal/34680 and came across something that was new to me, and seemed worthy of repeating here - if only one other person learns this it'll be worth it: In comment http://use.perl.org/comments.pl?sid=37249cid=58370 brian d foy points out that these days there is a really easy way to run CPAN as a normal user, and install as root. Here's how to set your CPAn config to do this: From your shell prompt: sudo cpan Then from the cpan shell: o conf make_install_make_command 'sudo make' o conf mbuild_install_build_command 'sudo ./Build' o conf commit quit From now on, run cpan as a regular user: cpan Some::New::Module That will build as you, and install as root. You may also need to change the permissions on your cpan_home directory so that your regular user can write to it. Brian points out that Normal first-time configuration these days offers that, but if you are like me and have been upgrading CPAN on an old configuration you may have overlooked this very useful addition. This public service announcement brought to you while I should be working for The Man. -M Thanks for posting this and reminding me about it. I'll stand up as at least one other person realizing this so I can clean up my new perl install on OSX/Ubuntu this evening. -=Chris signature.asc Description: OpenPGP digital signature
Re: a standard place for extra tests
chromatic wrote: On Thursday 16 August 2007 20:14:31 Eric Wilhelm wrote: I am certain that more than one 'extra tests' directory is needed, thus the thought to make them into subdirectories (objections?) (They cannot live under 't/' due to compatibility issues.) Which compatibility issues? I've used t/author/ for quite a while without any compatibility issues. -- c Right, accept when your t/author tests are tests for you, the author and my t/author tests are for my Author class/classes/pages/etc. -=Chris signature.asc Description: OpenPGP digital signature
Re: a standard place for extra tests
chromatic wrote: On Friday 17 August 2007 09:31:50 Christopher H. Laco wrote: chromatic wrote: I've used t/author/ for quite a while without any compatibility issues. Right, accept when your t/author tests are tests for you, the author and my t/author tests are for my Author class/classes/pages/etc. Fair enough, but I'm not sure that this is a place where any heuristic will work perfectly. At some point, I suspect that someone will have to encode some information about which tests are functional and which tests are bonus, nice-to-know fun. -- c I'm always a fan of meta info. I'd have to say, I'd much rather there be a way to specify which tests are author tests within the t files themselves. Then let the files roam where they may, and let the tools, or the test runners figure out when to run the author tests rather than looking for a directory. -=Chris signature.asc Description: OpenPGP digital signature
Re: a standard place for extra tests
Adriano Ferreira wrote: On 8/17/07, Christopher H. Laco [EMAIL PROTECTED] wrote: chromatic wrote: On Thursday 16 August 2007 20:14:31 Eric Wilhelm wrote: I am certain that more than one 'extra tests' directory is needed, thus the thought to make them into subdirectories (objections?) (They cannot live under 't/' due to compatibility issues.) Which compatibility issues? I've used t/author/ for quite a while without any compatibility issues. -- c Right, accept when your t/author tests are tests for you, the author and my t/author tests are for my Author class/classes/pages/etc. Testing some Author stuff would be rarer than having author tests. So maybe we could standardize on something like t/author and when other value is desirable, a key/value pair may be specified in META.yml (and in Makefile.PL/Build.PL). # META.yml --- # the default location for author tests (not needed) author_tests: t/author # when something different is wanted author_tests: t/developer That convinces me even more that me other email was spot on imho. Looking for a directory and assuming what's in it are an authors' tests is pure guesswork. It's rude that you assume whatever directory I have isn't used for other purposes and will ignore those tests on make test. *.t files need a way to declare themselves as an authors' test. Let the runner/harness inspect/ask each *.t file if it's an author test or not, and run it or not...regardless of what directory it's in. signature.asc Description: OpenPGP digital signature
Re: a standard place for extra tests
chromatic wrote: On Friday 17 August 2007 11:17:10 Christopher H. Laco wrote: To whit: Who knows better whether foo.t is an author test or not [in the absence of the actual human author]... The harness, or foo.t? The author. The harness asking the t files will always be more accurate than the harness just find t files somewhere. It doesn't matter where the metadata is, as long as someone who knows (the author) enters that metadata in a way that makes sense to the tools... ... sort of like I've been doing for at least one and probably two years now with Module::Build and my custom disttest action, which I know I've mentioned a few times now, which leads me to wonder why people so conveniently forget it every time this discussion comes up. *cough* selective memory *cough*...myself included. :-) signature.asc Description: OpenPGP digital signature
Re: running author tests
Adriano Ferreira wrote: On 8/2/07, David Golden [EMAIL PROTECTED] wrote: Having an actual pod/pod-coverage.t gives a handy place to put those customizations. Yes, some of that could be put in pod_coverage_options in a config or a ACTION_testpod method, but to me, that introduces extra complexity that I don't want cluttering Build.PL. Agreed in all manners. IMHO, Build.PL and Makefile.PL should be kept as dumb as possible. And easy to be read (and understood). Having an authortest/ directory that gets run with a harness is simple and consistent. There is already a convention that tests go under the t/ directory and also that they usually match the glob t/*.t. I cannot think of anything simpler and that can possibly works, than to stick the extra tests into subdirectories of t/ like chromatic does ( t/author/*.t or t/developer/*.t ). The behavior could be triggered by updating M::B and EUMM to make them run the extended glob t/**/*.t when PERL_AUTHOR_TESTING or AUTOMATED_TESTING is setted. Even programmers unaware of such conventions that are trying with the distribution, will make $ prove t/author/pod.t naturally as he/she runs any other tests. Regards, David I must say, I like t/author more than t/authortests... However, my main beef overall is now we essentially have a 'reserved name' directory that may interfere with how I may choose to group tests for larger distributions. For example, I have a boatload of tests to test http server code in t/live, and tests to test providers in t/providers and tests to test catalyst in...shockingly...t/catalyst, etc... If I have a boatload of tests for a 'author' object, I might choose to group then in t/author Sure, it's rare. Sure, I could name all those tests t/author_*.t... Either way, I think just assuming a directory name is rude. Now, if there were a way to tell the harness which directory to consider as 'author' tests.. then I guess I don't care. -=Chris signature.asc Description: OpenPGP digital signature
Re: Revising the Perl Testing quick reference card
Andrew Ford wrote: I have corresponded with Ian Langworth and he has agreed that I should revise the Perl Testing quick reference. I shall be converting it to LaTeX and expanding it to be a two-sided card, which implies doubling the amount of information on the card. I intend to publish it under the same Creative Commons license and will also publish the refcard.cls LaTeX class file that it uses. I shall go through the Test::* modules to see which others to cover, but I wondered whether anyone had feelings about which they would like to see. I have also started playing with WWW::Selenium and feel a quick reference card coming on there -- to go with my (as-yet-only-paritally-written) WWW::Mechanize card. Regards Andrew Well, I hope this doesn't stir the pot again, but a list of sanctioned ENV variables and their meaning would be nice.. TEST_AUTHOR, PERL_AUTOMATED_TESTING, etc. -=Chris signature.asc Description: OpenPGP digital signature
Re: Summarizing the pod tests thread
chromatic wrote: On Tuesday 31 July 2007 12:35:06 David Golden wrote: On 7/31/07, chromatic [EMAIL PROTECTED] wrote: 1) POD can possibly behave any differently on my machine versus anyone else's machine, being non-executed text and not executed code What version of Pod::Simple do you have? What version does everyone else have? Will POD parsed on your machine always parse the same everywhere? POD being a mostly-human readable markup language, the possible problem is that the POD renders incorrectly. In this case, the documentation is still available, whether by reading around code incorrectly appearing in the documentation or reading the documentation directly, not through a POD translator/transliterator. This is not a functional failure; it has nothing to do with whether the *code* will behave correctly on a user's system. Unless that render failure also means that --help/pod2usage in a script doesn't work. Then it's a functional failure. signature.asc Description: OpenPGP digital signature
Re: Fixing the damage caused by has_test_pod
Eric Wilhelm wrote:
# from David Golden
# on Monday 30 July 2007 05:34 am:
The issue at hand is really *release* testing (i.e. did I bump the
version, did I test my Pod, do I use good style, etc.) being mixed
with *functional* testing -- and the corresponding push for release
testing modules being included as requirements.
Yes. We want to know that the author tested the code and went through
the checklist before the release. We would like to be able to verify
it ourselves, but we don't want all of that to get in the way of
installation.
Ok, I can agree with that.
For that, I blame -- among other things -- Module::Starter for
including pod.t and pod-coverage.t and with a default setting to run
tests. Better would have been to skip tests unless
$ENV{AUTHOR_TESTING} or some similar flag was set.
Again, yes. Though I'm going to stick with delete them.
I think the important bit is that `make test` only runs tests which
verify the module's functionality.
Anything else needs to be a separate test target or out-of-band tool.
I don't agree. What runs when I do 'make test' is up to me, and if I
want to litter it up with 'author' tests, then that's my business; right
or wrong. Don't like it, then don't use my modules. (I still think all
author tests should not run by default...)
What needs to be unencumbered is:
perl Makefile.PL (should not reference non-functional modules in prereq)
make install (just install it please)
Any notion that make test needs to be altered, or should or shouldn't
run some and not other 'tests' is silly imho.
Putting Test::Foo in PREREQ=Bad. Fine. I understand.
Bitching about how many and what tests get run during 'make test' is absurd.
Of course, I could be missing the point of it all.
signature.asc
Description: OpenPGP digital signature
Re: Fixing the damage caused by has_test_pod
Eric Wilhelm wrote: # from Christopher H. Laco # on Monday 30 July 2007 11:14 am: I don't agree. What runs when I do 'make test' is up to me, and if I want to litter it up with 'author' tests, then that's my business; right or wrong. Don't like it, then don't use my modules. (I still think all author tests should not run by default...) This is not about what happens when *you* do `make test`, it's about what happens when the end-user does `make test`. The default module-starter setup creates this t/pod.t #!perl -T use Test::More; eval use Test::Pod 1.14; plan skip_all = Test::Pod 1.14 required for testing POD if $@; all_pod_files_ok(); If *you* don't have Test::Pod, and *I* do, *I* cannot install your module if the pod doesn't pass. Huh? In that code, no Test::POD = skip all tests. How does that equate to an install failure for anyone...either the author, the dist maint, or the end user? signature.asc Description: OpenPGP digital signature
Re: Fixing the damage caused by has_test_pod
Eric Wilhelm wrote: # from Christopher H. Laco # on Monday 30 July 2007 11:14 am: I don't agree. What runs when I do 'make test' is up to me, and if I want to litter it up with 'author' tests, then that's my business; right or wrong. Don't like it, then don't use my modules. (I still think all author tests should not run by default...) This is not about what happens when *you* do `make test`, it's about what happens when the end-user does `make test`. The default module-starter setup creates this t/pod.t #!perl -T use Test::More; eval use Test::Pod 1.14; plan skip_all = Test::Pod 1.14 required for testing POD if $@; all_pod_files_ok(); If *you* don't have Test::Pod, and *I* do, *I* cannot install your module if the pod doesn't pass. Oh, duh. *smacks forehead*. I get it. You have it installed, the tests run, and fail, then you can't install. I say that's fine. If it fails and you can't install it, then don't. Arguments about whether the tests should or shouldn't be run [or included at all] is irrelevant. Tests failed. Don't install. File RT. If the author made a choice to have them run always and piss people off, or restrict the user base, then that's the authors prerogative. Now, I won't argue that it would be better if the author put in a skip check for TEST_AUTHOR or other variants, but that's up to the author and not something that should be enforced or needs to be regulated. If I ship a module and test coverage fails, that's still a bug. The end user still has no documentation on what the hell the 'naked' method is supposed to do. Sure, Test::Pod::Coverage could fail for some reason other than it's main purpose of checking coverage and finding a naked method, but so then can Test::More and any other test module. Failed Tests = Don't Install. Any arguments over what tests failed and why, or if they should even be run or shipped is moot in my book. signature.asc Description: OpenPGP digital signature
Re: Fixing the damage caused by has_test_pod
Adam Kennedy wrote: For background on this email, see the following entry in my journal. http://use.perl.org/~Alias/journal/33893 Lately, I've noticed a series of modules that are cargo-culting the use of test_pod and test_pod_coverage in their tests. Not only are we seeing spurious failures for users, but worse some authors are putting Test::Pod and Test::Pod::Coverage as dependencies, and in extreme cases, Perl::Critic as well. Now, one argument is that it's the author's fault for not appropriately skipping these tests if the Test::Pod module is not available. However, in most of these cases the author would not have made these errors AT ALL if not for the fact that Kwalitee says they should have these tests. The CPANTS Game just makes this pressure worse, by making it competive. That said, I do see the value of these tests EXISTING in the distribution, as their provide a guarentee that any new maintainers for the modules will continue to comply with the principles those tests implement. BUT for most of these tests, it is BAD that the end-user EVER run them. Thus, I would like to propose the following. 1. That the running of POD-related tests by end users is considered harmful. I can never agree with this. A failing pod test is the first sign pod2* generation is broken. It's rare, but I've been in the situation where for some reason (mistmatching dist requirements, failed upgrades, bad dist packages, broken troff, etc) the creation of man pages/html from the dist pod fails outright. Pod tests assure the user that they will be able to perldoc/man the thing they are installing. That's doubly true if you're using pod2usage to expose help in scripts. Should they be run automatically, all the time? Maybe. Maybe not. Should they skip if the test modules aren't installed? Absolutely. Should they require the Test* modules in prereq? OF course not. But I don't think saying pod tests are something the end user should never run is wrong. Then again, I'm never on the popular side of these discussions. Personally, part of me wants to say: stop worrying about what tests I decide to ship and enable with my dists. -=Chris signature.asc Description: OpenPGP digital signature
Re: Paying for TAP 2.0
Paul Beckingham wrote: I'm wanting sparse output: 1..100 sparse 12 not ok 83 not ok Which is three lines of output, instead of 97, but contains the same information as: 1..100 1 ok 2 ok ... 12 not ok ... 83 not ok 84 ok ... 100 ok The essence of this is that with the plan of 1..100 sparse, missing ok N lines are assumed to exist, if they are not specifically output. Because I need to retain the output of all tests, and those files get large, but mostly because of the sheer redundancy. But how do you know 23 ok if you were never told that it ran ok? signature.asc Description: OpenPGP digital signature
Re: CPANTS?
Thomas Klausner wrote: Hi! On Tue, Mar 06, 2007 at 02:34:16PM +, Andy Armstrong wrote: Does anyone apart from Thomas Klausner know anything about the status of CPANTS? It's been down for about five days now. I'm trying to offer him free hosting for it but he doesn't seem to be getting his mail just now either. I got the mail but was/am busy (work and YAPC...) I also have troubles contacting the sysadmin who's currently hosting CPANTS, so I think I'll accept your generous offer. I'd suggest we settle the rest in private mail. It's the CPANTS shuffle! Isn't this like, server/host 3 or so? :-) -=Chris signature.asc Description: OpenPGP digital signature
Re: CPANTS?
Thomas Klausner wrote: Hi! On Tue, Mar 06, 2007 at 10:01:09AM -0500, Christopher H. Laco wrote: It's the CPANTS shuffle! Isn't this like, server/host 3 or so? :-) It is, which is my punishment for beeing to lazy to set up my own server :-) No, lazy is good. Don't let anyone fool on that one. I'm still trying to get rid of servers in my basement after 10 years, not get more of them. :-) signature.asc Description: OpenPGP digital signature
Re: a safer way to use no_plan?
chromatic wrote: On Saturday 03 March 2007 18:18, Andy Lester wrote: Good Lord do I get frustrated at the handwringing over test counting. Look, it's simple. You write your tests. You run it through prove. You see how many tests it reports. You add it at the top of the file. Voila! But Andy, what if I modify the test file? I don't want to have to modify the test file to tell my test harness that I've modified the test file! If only we had something that flies through space -- c Can't we use UNIVERSAL to work around these edge cases and throw lots of warnings? :-P -=Chris signature.asc Description: OpenPGP digital signature
Re: UNIVERSAL::ref might make ref( $mocked_obj ) sane
Yuval Kogman wrote: On Sun, Feb 25, 2007 at 22:34:15 -0800, chromatic wrote: Why not, I already have half of the other stuff in UNIVERSAL Just don't tell Adam. Or me. I have a personal hate relationship with MockObject in this way. I love MockObject. I hate getting warnings about 3 party modules use of can in my test suite. -=Chris signature.asc Description: OpenPGP digital signature
Re: ExtUtils::MakeMaker, and t/ sub-directories
Nik Clayton wrote:
Paul Johnson wrote:
On Thu, Feb 08, 2007 at 09:26:01AM +, Nik Clayton wrote:
[ I vaguely recall a discussion about this, but my search-fu is weak,
and I can't find it ]
Is there a standard way/idiom to get ExtUtils::MakeMaker to support
tests in subdirectories of t/?
I've got a bunch of tests, and rather than client-ls.t, client-add.t,
client-commit.t, etc, I'd like t/client/ls.t, t/client/add.t,
t/client/commit.t, and so on.
I have this in one of my Makefile.PLs, which seems to be just about
what you
are looking for:
WriteMakefile
(
...
test = { TESTS = t/*/*.t },
...
);
Ah. My mistake for not being clear enough. I want to run t/*.t and
t/*/*.t.
Of course, I tried
test = { TESTS = [ t/*.t, t/*/*.t ] },
and it doesn't work. It's just occurred to me that I'm trying to be too
clever.
test = { TESTS = t/*.t t/*/*.t },
works perfectly.
N
I offer this word of warning. If you have too many tests, or many
longly-named tests, win32 will in all likelyhood barf with a command
line too long error.
I had this problem in Handel (too many tests..it's a good problem to
have really), and even after updating EU::MM and all of my other Test::
bits, I still had the problems.
Some people do. Some people don't. It seems to be a crapshoot. But the
only consistant fix for me was to keep all of my tests in t/, and to not
use the test = {TESTS} idium
I also had the same issue when using M::I w/Makefile.PL, which is just
EU::MM underneath I assume.
-=Chris
signature.asc
Description: OpenPGP digital signature
Re: symphony and TAP
Christopher H. Laco wrote: While strolling through PlanetMVC, I noticed a free chapter about unit testing using it's own testing library call Lime. Low and behold, my eyes see TAP and some props to Test::More. Kewl. :-) http://www.symfony-project.com/weblog/2007/01/17/book-preview-read-the-unit-tests-chapter.html http://www.symfony-project.com/uploads/assets/The_Definitive_Guide_to_Symfony_Chapter_15.pdf -=Chris Bah. Probably should have gone to qa instead. :-/ -=Chris signature.asc Description: OpenPGP digital signature
Re: P::C or PPI bug?
Chris Dolan wrote: On Nov 7, 2006, at 6:15 PM, Christopher H. Laco wrote: I just wanted to get some thoughts on this before I filed a bug report with either PPI or Perl::Critic: I'm going through and testing all of my *.t files for RequireTestLabels. I was humming right along when I ran into an oddity. The newest Perl::Critic 0.21 + PPI 1.118 complains that the tests don't have labels, yet they clearly do. After some tinkering, I stumbled across the culprits: This causes a RequireTestLabel failure: is_deeply([$storage-primary_columns], [qw/id otherpk/], 'return DBIC primary keys from schema'); other is_deeply tests pass, like the one before it: is($storage-_primary_columns, undef, 'no primary columns defined'); I looked through RT, and I don't see anything that really points to anything on this one in either dist. Ideas? This is fixed in the latest PPI developer release v1.199_xx. The array constructor stuff should be perfect. The hash constructor parsing still needs some work to further disambiguate it from anonymous blocks. (like map, grep, eval, sort, etc) It never hit RT because I just fixed it myself instead of reporting. Chris I gave the latest PPI + Perl::Critic SVN a shot late last night. That problem is indeed fixed for me, but another on cropped up with UseStrict, even through I am, and it only happens in 2 of 30 files. As soon as I get ouf of my meetings this morning, I'll update on the local box and have another go and post the actual error. -=TheOtherChris signature.asc Description: OpenPGP digital signature
P::C or PPI bug?
I just wanted to get some thoughts on this before I filed a bug report with either PPI or Perl::Critic: I'm going through and testing all of my *.t files for RequireTestLabels. I was humming right along when I ran into an oddity. The newest Perl::Critic 0.21 + PPI 1.118 complains that the tests don't have labels, yet they clearly do. After some tinkering, I stumbled across the culprits: This causes a RequireTestLabel failure: is_deeply([$storage-primary_columns], [qw/id otherpk/], 'return DBIC primary keys from schema'); other is_deeply tests pass, like the one before it: is($storage-_primary_columns, undef, 'no primary columns defined'); I looked through RT, and I don't see anything that really points to anything on this one in either dist. Ideas? -=Chris signature.asc Description: OpenPGP digital signature
Devel::Cover eval oddity
Anyone have any ideas on this blip? http://handelframework.com/coverage/blib-lib-Handel-Base-pm.html line #171 Lord knows, it doesn't really matter since that's the only piece left, but I'm kinda of curious. This is under Devel::Cover 0.59 under 5.8.4, 5.8.6, and 5.8.8 -=Chris signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Chris Dolan wrote: On Nov 2, 2006, at 11:01 PM, Thomas Klausner wrote: metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? No. The CPANTS game is a tool for change. We might as well encourage people to change to the current spec. Chris I don't agree... unless all of the various make dist commands within M::B, EU::MM, etc actually always make the META.yml up the latest version of the spec. If a META.yml has declared itself as 1.0 version, then it need only to be 1.0 compliant, and not 1.2 compliant. There no need to force an upgrade from 1.0 to 1.2. The author has done the right thing and need not be punished. Now, for all META.yml files that don't declare a version at all... that's a different story. signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Christopher H. Laco wrote: Thomas Klausner wrote: Hi! I had some time recently and added some first META.yml checking to CPANTS (with the help of Gabor Szabo): metayml_is_parsable metayml_has_license metayml_conforms_spec metayml_has_license now indictes whether there's a computer readable license in META.yml, while (the also new metric) has_humanreadable_license does some where basic guessing if there's a human-readable license (LICENSE file or pod-section). metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? Well, files that declare themselves as nothing: --- are a free for all... Files that declare: --- #YAML:1.0 should pass the 1.0 spec Files that declare: --- #YAML:1.1 should pass the 1.2 spec err... Files that declare: --- #YAML:1.2 should pass the 1.2 spec I know what I meant. :-) signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Christopher H. Laco wrote: Chris Dolan wrote: On Nov 2, 2006, at 11:01 PM, Thomas Klausner wrote: metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? No. The CPANTS game is a tool for change. We might as well encourage people to change to the current spec. Chris --Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/) I'm ok with has the latest and greatest 1.2 META spec being a metric, but I think it should be one of the above 100.00 bonus points metrics like the has_demo metric. and yes, I'm biased. I was #2. :-) -=Chris I'm not sure why, but about every other post I make isn't showing up on the list... hmm -=Chris signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Thomas Klausner wrote: Hi! I had some time recently and added some first META.yml checking to CPANTS (with the help of Gabor Szabo): metayml_is_parsable metayml_has_license metayml_conforms_spec metayml_has_license now indictes whether there's a computer readable license in META.yml, while (the also new metric) has_humanreadable_license does some where basic guessing if there's a human-readable license (LICENSE file or pod-section). metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? Well, files that declare themselves as nothing: --- are a free for all... Files that declare: --- #YAML:1.0 should pass the 1.0 spec Files that declare: --- #YAML:1.1 should pass the 1.2 spec If matchines versiones to their specs can't be done with the available parsers, Id say go back to what you were going before...which I'll assume was a simple throws error on load or doesn't throw error on load check -=Chris signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Chris Dolan wrote: On Nov 2, 2006, at 11:01 PM, Thomas Klausner wrote: metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? No. The CPANTS game is a tool for change. We might as well encourage people to change to the current spec. Chris --Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/) I'm ok with has the latest and greatest 1.2 META spec being a metric, but I think it should be one of the above 100.00 bonus points metrics like the has_demo metric. and yes, I'm biased. I was #2. :-) -=Chris signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Michael G Schwern wrote: Christopher H. Laco wrote: Files that declare: --- #YAML:1.0 should pass the 1.0 spec Files that declare: --- #YAML:1.1 should pass the 1.2 spec err... Files that declare: --- #YAML:1.2 should pass the 1.2 spec I know what I meant. :-) I thought that was the version of YAML they're conforming to. meta-spec is the META version. meta-spec: url: http://module-build.sourceforge.net/META-spec-v1.2.html version: 1.2 PS If someone would check the metafile generated by MakeMaker 6.31 for 1.2 spec conformance I'd appreciate that. Right. meta-spec version != yaml version. /me smacks forehead. signature.asc Description: OpenPGP digital signature
Re: CPANTS and META.yml
Thomas Klausner wrote: Hi! I had some time recently and added some first META.yml checking to CPANTS (with the help of Gabor Szabo): metayml_is_parsable metayml_has_license metayml_conforms_spec metayml_has_license now indictes whether there's a computer readable license in META.yml, while (the also new metric) has_humanreadable_license does some where basic guessing if there's a human-readable license (LICENSE file or pod-section). metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? Just for giggles, what's the code behind that? Is it different than the current CPAN method: metayml_has_required_fields Is there a good way to check for 1.2 in a test, other than a these fields are required and YAML parses it type of strategy? -=Chris signature.asc Description: OpenPGP digital signature
Re: Failing test on Windows
Nicholas Clark wrote: On Tue, Oct 24, 2006 at 08:08:45PM -0400, Christopher H. Laco wrote: With most modules, I agree. But with utility modules like Module::Pluggable, File::Find::Recursive, etc, not working under taint I'd be surprised if the author of Module::Pluggable wasn't open to patches to fix this. Nicholas Clark :-) 2005-07-09 - 2.9 More Tainting fixes Patches from Christopher H. Laco and Joe McMahon to do more taint fixing I used that module as an example. I agree with jrockway in that some things can't be pre-untainted; things like CGI/HTTP request params. But modules that don't untaint .* should provide something like File::Finds untaint_pattern instead. I'm a firm believer that everyone should run tests under -T. What use is the vast reusable of CPAN if nothing runs under -T and the users script/server runs under -T? People are forced to either a) not reuse CPAN modules or b) write their own or c) run local patched versionall defeating the purpose. I think planning and testing your modules under -T is just being a good CPANizen; just like warnings/strict and writing pod. -=Chris signature.asc Description: OpenPGP digital signature
Re: Sparse Test Output
Paul Beckingham wrote: I'm with Adrian. Printing out ok 100,000 times shouldn't be a big deal unless you're reading the TAP via some sort of IP over clay tablets protocol. But... My test estimate is two orders of magnitude larger, so it actually is a big deal to capture and store those results. But I would like to point out that there is a very low information density in 100,000 ok messages. I was looking to see what folks thought of an *optional* feature that doesn't bother outputting the ok messages, just the interesting ones. Hence the sparse. The basic idea was that instead of: 1..10_000_000 ok 1 ok 2 ... not ok 123 ... ok 10_000_000 The output could be collapsed to the following, with no loss of important information: 1..10_000_000 sparse not ok 123 Yes, I could achieve this with grep, sed, Perl etc, but I thought it might make a nice addition to TAP. Paul. I'm in the same boat. Recently, I've started testing my environment when things go wrong. (I blame Andy). I have one test alone that has a test count of 500,000+. That's a lot of oks to be processed, when I only want the ones that didn't pass. Now, add in a few more tests that easily go into the 50,000 range in addition to 'the bi one', and waiting for the test output/TAP to be parsed takes quite a while. Just my $0.02 worth. -=Chris signature.asc Description: OpenPGP digital signature
Re: Sparse Test Output
Michael G Schwern wrote: Christopher H. Laco wrote: I'm in the same boat. Recently, I've started testing my environment when things go wrong. (I blame Andy). I have one test alone that has a test count of 500,000+. That's a lot of oks to be processed, when I only want the ones that didn't pass. Now, add in a few more tests that easily go into the 50,000 range in addition to 'the bi one', and waiting for the test output/TAP to be parsed takes quite a while. Just my $0.02 worth. I am totally unconvinced. I would love to see some profiling on this. I would hope that the act of running 50,000 tests would swamp the cost of parsing 50,000 tests. Depends on what the tests are. If it's simply verifying that 500,000 pieces of data confirm to a certain format, then those kinds of tests aren't very expensive, or long running. Just for sake of curiosity, I'll profile one of the tests tomorrow just to see what happens. -=Chris signature.asc Description: OpenPGP digital signature
Re: Failing test on Windows
Michael G Schwern wrote: Christopher H. Laco wrote: Shouldn't all tests be set to use tainting? And by extension, shouldn't all modules run under -T? (Just curious) That would be nice, but tainting is a pain in the ass and not something I'd be willing to universally inflict on all module authors. True. On the flip side, there's nothing more irritating that flipping PerlTaintCheck On only to find out a crapload of modules don't run under taint. And that's assuming you have control over whether the flag is set of not. :-/ Damned if you do. Damned if you don't. signature.asc Description: OpenPGP digital signature
Re: Failing test on Windows
Jonathan Rockway wrote:
True. On the flip side, there's nothing more irritating that flipping
PerlTaintCheck On only to find out a crapload of modules don't run under
taint. And that's assuming you have control over whether the flag is set
of not. :-/
Maybe an example would help me understand this, but doesn't the application
*itself* know how best to untaint data? If a module were to check everything
for taint, it would be roughly equivalent to $var =~ /^(.*)$/; $var = $1;
which serves no security purpose. I think that if you're running with taint
on, you (the application developer) should be responsible for properly
cleansing input. If you miss something, you'll want your app to die inside
the module so you know what your app is forgetting to check. If you don't
want your app to die, then don't use taint mode, right?
I would certainly be upset if a module, in the interest of taint safety, set
$ENV{PATH} to qw(/bin /usr/bin) when I wanted to exec a binary
in /usr/libexec/bin instead. It's up to me to know the specific details of
my environment, not a generic module.
Regards,
Jonathan Rockway
With most modules, I agree. But with utility modules like
Module::Pluggable, File::Find::Recursive, etc, not working under taint
is a big bummer because they die under -T internally...in code of my
immediate control. (Yeah, I know. Patches Welcome)
I just found out yesterday that some tests that use Module::Find don't
run under -T and the tests die in Module::Find, no in my .t file.
So, yes, some modules can't untaint things in a sensical manner. But
somethings, like Module::Find shouldn't just go boom under -T either.
Like I said, damned if you do. Damned if you don't.
-=Chris
signature.asc
Description: OpenPGP digital signature
Re: Failing test on Windows
Adam Kennedy wrote: WORKSFORME on Strawberry alpha 2. I'm betting it's something to do with the -T, because the only difference I can think of is that the build is running inside some sort of alternative harness that does something differently... Out of curiosity, why are your tests set to use tainting? Adam K Shouldn't all tests be set to use tainting? And by extension, shouldn't all modules run under -T? (Just curious) -=Chris signature.asc Description: OpenPGP digital signature
Test::MockObject: What am I missing?
I'm having a idiot moment...
I'm trying to mock out some config reading tests for reading from
MP1/MP2 even though I don't have either installed. so I thought
Test::MockObjext is the answer!.
The following code goes boom with a 'Can't locate object method request
via package Apache':
use strict;
use warnings;
use Test::MockObject;
my $mock = Test::MockObject-new;
$mock-fake_module('Apache');
$mock-set_series(undef, $mockrequest);
warn Apache-request;
If I change the fake_module call to
Test::MockObject-fake_module('Apache'), the code gets happt, but it
appears I can then install subs into that Apache using any of the set_
helpers.
Or am I just missing something?
The pod mentions that fake_module is a class and object method, but
doesn't really extol the virtues on when to use one over the other.
Someone cluestick me please.
-=Chris
signature.asc
Description: OpenPGP digital signature
Re: Test::MockObject: What am I missing?
chromatic wrote:
On Thursday 05 October 2006 12:25, Christopher H. Laco wrote:
I'm having a idiot moment...
I'm trying to mock out some config reading tests for reading from
MP1/MP2 even though I don't have either installed. so I thought
Test::MockObjext is the answer!.
The following code goes boom with a 'Can't locate object method request
via package Apache':
use strict;
use warnings;
use Test::MockObject;
my $mock = Test::MockObject-new;
$mock-fake_module('Apache');
$mock-set_series(undef, $mockrequest);
warn Apache-request;
If I change the fake_module call to
Test::MockObject-fake_module('Apache'), the code gets happt, but it
appears I can then install subs into that Apache using any of the set_
helpers.
Or am I just missing something?
The pod mentions that fake_module is a class and object method, but
doesn't really extol the virtues on when to use one over the other.
Someone cluestick me please.
Test::MockObject primarily returns you an object on which you can mock and
call mocked methods. fake_module() lets you install new methods into the
mocked class by passing name and subref pairs, but that's really not the
point of the module.
-- c
I won't pretend not be be confused at the moment then, because I have
other tests that do:
T::MO-new-fake_module('Foo'), and classes that 'use Foo' and do
'Foo-new' just work with the mocked verson. At least, I thought they
did, and they don't throw the same error.
So, all of that aside, what is everyone else using in this type of
scenario of mocking classes used by all rather than passing mocked
object around?
Test::MockModule/MockClass looks like it will do.
-=Chris
signature.asc
Description: OpenPGP digital signature
Re: Test::MockObject: What am I missing?
chromatic wrote:
On Thursday 05 October 2006 12:25, Christopher H. Laco wrote:
I'm having a idiot moment...
I'm trying to mock out some config reading tests for reading from
MP1/MP2 even though I don't have either installed. so I thought
Test::MockObjext is the answer!.
The following code goes boom with a 'Can't locate object method request
via package Apache':
use strict;
use warnings;
use Test::MockObject;
my $mock = Test::MockObject-new;
$mock-fake_module('Apache');
$mock-set_series(undef, $mockrequest);
warn Apache-request;
If I change the fake_module call to
Test::MockObject-fake_module('Apache'), the code gets happt, but it
appears I can then install subs into that Apache using any of the set_
helpers.
Or am I just missing something?
The pod mentions that fake_module is a class and object method, but
doesn't really extol the virtues on when to use one over the other.
Someone cluestick me please.
Test::MockObject primarily returns you an object on which you can mock and
call mocked methods. fake_module() lets you install new methods into the
mocked class by passing name and subref pairs, but that's really not the
point of the module.
-- c
How is above any different than this: s/GEO::IP/Apache/ ?
http://www.perl.com/pub/a/2005/04/07/mockobject_kata.html?page=2
Obviously, my brain is not working right today.
signature.asc
Description: OpenPGP digital signature
Re: Pod Spelling
Ovid wrote: --- Ivan Tubert-Brohman [EMAIL PROTECTED] wrote: Have you tried Pod::Spell and Test::Spelling? I think they already do that. I've tried Test::Spelling only to discover that there are so many exceptions which it doesn't recognize that it was very painful to try and include in a test suite. The list of stopwords for large systems just became very unwieldy. The idea behind the module is great, though. Cheers, Ovid -- Buy the book -- http://www.oreilly.com/catalog/perlhks/ Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/ I hadd tzat problme to, butt Im susch a bAd typistt tht I preszed on nyways. :-) -=Chris signature.asc Description: OpenPGP digital signature
Re: Don't 'rm -fr /' when testing
Ovid wrote: [snip] It would be nice if I could just write 'use My::Test::More' in my test scripts and have that provide what I need, but I'm not sure if trying to re-export all of the test functions from Test::More (kind of like subclassing which isn't a class) is a bright idea, but it's the cleanest I can think of. Am I missing something really obvious here? Cheers, Ovid I'm thinking about doing the same thing. Before Chris Dolan wrote a Testing::RequireTestLabels policy for me (thanks!), I was going to subclass Test::More and expose the usual methods and tack on my argument checking. I still may do it just to merge my Handel::Test (SQLite schema init/populate) and my use of Test::More into one interface. -=Chris signature.asc Description: OpenPGP digital signature
Re: Don't 'rm -fr /' when testing
Ovid wrote:
From: Christopher H. Laco [EMAIL PROTECTED]
I'm thinking about doing the same thing. Before Chris Dolan wrote a
Testing::RequireTestLabels policy for me (thanks!), I was going to
subclass Test::More and expose the usual methods and tack on my argument
checking.
Just threw this together and it seems to do the trick.
package My::Test::More;
use Test::Builder::Module;
@ISA= qw(Test::Builder::Module);
use Test::More;
@EXPORT = @Test::More::EXPORT;
# add whatever you need here:
$ENV{WE_BE_TESTING} = 1;
1;
And in your regular code, use it like normal:
use My::Test::More tests = 23;
Cheers,
Ovid
Time to show my ignorance. Why use Builder::Module directly instead of:
@ISA= qw(Test::More);
@EXPORT = @Test::More::EXPORT;
-=Chris
signature.asc
Description: OpenPGP digital signature
Re: Testing for test labels
Chris Dolan wrote: On Sep 24, 2006, at 5:42 PM, Christopher H. Laco wrote: Ok, I'll play your game. :-) http://perlcritic.tigris.org/svn/perlcritic/trunk/Perl-Critic/lib/Perl/Critic/Policy/Testing/ [Assuming I'm not silly] Empty! Rev. 667 -=Chris D'oh! SVN commits work better if you svn add first... Fixed, should work now. :-) Chris -- Chris Dolan, Software Developer, http://www.chrisdolan.net/ Public key: http://www.chrisdolan.net/public.key vCard: http://www.chrisdolan.net/ChrisDolan.vcf Been there. Done That. I wonder if there's a config option anywhere to have svn commit bark loudly if there are non-added files. signature.asc Description: OpenPGP digital signature
prove vs. inc
While trying to install the latest PPI from svn, I noticed that running 'dmake test' passed, and 'prove -b t/' failed. As more and more people use Module::Install, it seems more and more transient build/test related modules are being stuffed in the inc/ directory. Given the talk about installers being installers (dmake) and testers being testers (prove), what is the correct behavior in this situation? Should the tests be skipping when these modules aren't found, and stop assuming I'm doing a 'make test' instead of a 'prove'? Should prove -l or -b look for an inc/ directory as well? And who should get this nice shiny new RT of mine? :-) -=Chris C:\PPI-1.118prove -b t/ t\01_compile.ok t\02_api.Can't locate Test/ClassAPI.pm in @INC (@INC contains: blib\arch blib\lib C:/strawberry- perl/perl/lib C:/strawberry-perl/perl/site/lib .) at t\02_api.t line 19. BEGIN failed--compilation aborted at t\02_api.t line 19. t\02_api.dubious Test returned status 2 (wstat 512, 0x200) DIED. FAILED tests 1-2192 Failed 2192/2192 tests, 0.00% okay t\03_empiric.ok t\04_element.ok t\05_lexer_practical.ok t\06_round_trip..ok t\07_token...ok t\08_regression..ok t\09_normal..ok t\10_statement...ok t\11_utilok t\12_locationok t\13_dataok t\14_charsetsok 11/11 skipped: Unicode-incompatible locale in use t\15_transform...ok t\16_xml_compatibility...ok t\17_storableok t\18_cache...Can't locate File/Remove.pm in @INC (@INC contains: blib\arch blib\lib C:/strawberry-pe rl/perl/lib C:/strawberry-perl/perl/site/lib .) at t\18_cache.t line 14. BEGIN failed--compilation aborted at t\18_cache.t line 14. t\18_cache...dubious Test returned status 2 (wstat 512, 0x200) t\19_selftesting.Can't locate Class/Inspector.pm in @INC (@INC contains: blib\arch blib\lib C:/strawberr y-perl/perl/lib C:/strawberry-perl/perl/site/lib .) at t\19_selftesting.t line 16. BEGIN failed--compilation aborted at t\19_selftesting.t line 16. t\19_selftesting.dubious Test returned status 2 (wstat 512, 0x200) t\20_tokenizer_regressionok t\21_exhaustive..ok t\22_readonlyok t\23_fileok t\99_author..skipped all skipped: Author tests not required for installation t\99_pod.skipped all skipped: Test::Pod 1.00 required for testing POD t\ppi_elementok t\ppi_node...ok t\ppi_normal.ok t\ppi_statement_variable.ok t\ppi_token__quoteengine_fullok t\ppi_token_quoteok t\ppi_token_quote_double.ok t\ppi_token_quote_interpolateok t\ppi_token_quote_literalok t\ppi_token_quote_single.ok Failed TestStat Wstat Total Fail Failed List of Failed --- t\02_api.t2 512 2192 4384 200.00% 1-2192 t\18_cache.t 2 512?? ?? % ?? t\19_selftesting.t2 512?? ?? % ?? 2 tests and 11 subtests skipped. Failed 3/35 test scripts, 91.43% okay. 2192/5924 subtests failed, 63.00% okay. C:\PPI-1.118 signature.asc Description: OpenPGP digital signature
Re: Testing for test labels
Chris Dolan wrote:
On Sep 24, 2006, at 5:42 PM, Christopher H. Laco wrote:
Ok, I'll play your game. :-)
http://perlcritic.tigris.org/svn/perlcritic/trunk/Perl-Critic/lib/Perl/Critic/Policy/Testing/
[Assuming I'm not silly] Empty! Rev. 667
-=Chris
D'oh! SVN commits work better if you svn add first... Fixed, should
work now. :-)
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Now, for part 2. :-)
What's the sanest way to test t/*.t files without also testing
everything in t/lib ?
If I'm already using all_critic_ok(), I can't just call critic_ok() on
all the t/*.t files because a plan has already been set.
Would it be possible to have all_critic_ok also take globs:
all_critic_ok('blib', 't/*.t')
-=Chris
signature.asc
Description: OpenPGP digital signature
Testing for test labels
I've got a crap loads of tests in Handel at the moment; 1+ checkpoints, and enough files that I hit the dreaded command line too long under win32 [now fixed]. That's not saying all the tests are glorious and not repetitive. :-) I've still got a ways to go before I'm happy without the dist enough to declare the 1.0 version done. Among things on my list are cleaning up the current test code. A big offender is that not all of my tests have labels. I could've swore that sometime in the last month I saw some chatter about a Test::TestDescriptions module, or a ::TestsHaveDescriptions type thing for Perl::Critic. Assuming I'm not just making that up, has anyone seen mention of such a thing? If no such a beast exists thus far, what would be the easiest way to get such functionality? I was thinking that I could just run a subclass of Test::More in my tests that dies whenever no label argument is supplied. That's not a terrible thing since I already have a Handel::Test module that deals with SQLite setup and teardown in most tests. Thoughts? -=Chris signature.asc Description: OpenPGP digital signature
Re: Testing for test labels
Chris Dolan wrote: [snip] Done. I created Perl::Critic::Policy::Testing::RequireTestLabels and added it to the Perl::Critic SVN at http://perlcritic.tigris.org/svn/perlcritic/trunk/Perl-Critic (username: guest, password: ) It was pretty simple to write. Below are the important bits of the code. I encourage others to write Test::More-specific policies for Perl::Critic! Chris [snip] Ok, I'll play your game. :-) http://perlcritic.tigris.org/svn/perlcritic/trunk/Perl-Critic/lib/Perl/Critic/Policy/Testing/ [Assuming I'm not silly] Empty! Rev. 667 -=Chris signature.asc Description: OpenPGP digital signature
Re: [Module::Build] [RFC] author tests
A. Pagaltzis wrote: * Chris Dolan [EMAIL PROTECTED] [2006-02-02 16:55]: On Feb 1, 2006, at 10:35 PM, Tyler MacDonald wrote: I really like this idea. But as you pointed out, it's not just authors that need to worry about running these tests, it's packagers (ppm/deb/etc), automated testers (cpants/testers.cpan.org/etc), and hackers. No, I disagree. I'm specifically talking about author tests, NOT packager tests. Things like Test::Spelling are pointless and difficult for packagers to execute because Test::Spelling relies on an external aspell or ispell program *and* performs differently in the presence of an author's custom dictionary (mine has Dolan; does yours?) These specifically are not exhaustive tests but spit-and-polish tests. I was just gonna say. It’s pointless for anyone but the author to check POD or test coverage. Only under the assumption that the author was negligent and shipped a distribution without running the POD tests does it make any sense for a packager to run them. And then it still doesn’t make sense for *every* packager to run them. Similarly for Devel::Cover – what’s the packager to do, Normally I'd agree, but that's not 100% set in stone. I work on Linux/Win32 and run the usual pod syntax/coverage tests on those platforms. I also have FreeBSD servers for which package could/would be created. On more than one occasion, I've had pod2html/man (troff) errors under FreeBSD that were only found by running the author tests there, even though the pod syntax/coverage was perfectly fine on two other platforms. It's these sorts of problems where the packager running those tests are quite beneficial...Especially on platforms where the packagers/dists may be adding patches to the core dist. -=Chris signature.asc Description: OpenPGP digital signature
Re: RT Permissions
Andy Lester wrote: On Fri, Dec 02, 2005 at 12:10:24PM -0800, Ovid ([EMAIL PROTECTED]) wrote: I've wondered about this myself. I've taken over Class::Trait but I can't take ownership of the RT requests. RT should do it automagically. Email Jesse directly if not. xoxo, Andy For which, first-come, or do all of the co-maints have full RT access as well? -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: RT Permissions
Andy Lester wrote: On Fri, Dec 02, 2005 at 12:10:24PM -0800, Ovid ([EMAIL PROTECTED]) wrote: I've wondered about this myself. I've taken over Class::Trait but I can't take ownership of the RT requests. RT should do it automagically. Email Jesse directly if not. xoxo, Andy Which Jesse... Vincent? -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: CPAN Testers results
Ovid wrote: Hi all, I've noticed that http://search.cpan.org/~ovid/HOP-Parser-0.01/, amongst other modules, has no CPAN test results appearing even though CPAN tester reports are coming in. I've seen this for other modules, too. Is there an announced reason for this I missed or is something down? Cheers, Ovid I've often wondered this myself. It seems like it builds up some sort of delta or count of reports, then does a build; as to avoid rebuilding stats for every report email that comes in. Or, somethings just broken. :-) -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test Suite Slowing Down My Development
Ovid wrote: The code is designed well enough that adding new features is quick and easy. Unfortunately, whenever I need to change my code I fire up a Web server and view the results in the browser and then write the tests after I've written the code (this is closely related to When test-driven development just won't do). This is because XML and XHTML are just text. I need to see the output. I've been finding more and more that small changes in my code are making huge changes in the output and trying to continuously update the tests to exactly match the XML, XSLT and XHTML using Test::XML and XML::XPath has led to a serious productivity drop. I'm considering just using Test::WWW::Mechanize to do integration testing through a Web server I run in the tests. This will be much faster and allow me to get my development speed back up. However, I'd be skipping the unit testing of the output. I'll catch the bugs but it will likely take me longer to track them down. I feel your pain. The test suite for Handel has xml/tt output tests for its AxKit and Template Toolkit plugins. I've got oodles of template pages using the components whos output I compare to static .out files that contain the expected output. Everytime I write a new plugin, or a new tag in the plugin, I waste tons of time just writing the tests for them. So far, I've been good about writing the tests before I write the code, but it takes forever and I rarely get the tests right the first time. I'm curious to see what comes out of your question. I'm in the same boat. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: New kwalitee test, has_changes
Adam Kennedy wrote: Rather than do any additional exploding, I'd like to propose the additional kwalitee test has_changes. I've noticed that a percentage (5-10%) of dists don't have a changes file, so it can be hard to know whether it's worth upgrading, or more importantly which version to add dependencies for. Adam K Would this look for Change OR ChangeLog? Both seem to be popular on CPAN. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: rantTesting module madness
Andy Lester wrote: Usually, Test::* modules are only used for the test phase. I really don't understand the idea of only used for the test phase, as if the tests don't matter, or if there are levels of failure. Either they install OK on the target system, and you can use them with confidence, and they've done their job, or you're going to ignore the tests completely and then who needs 'em? It's like if I'm installing a washing machine, and I don't have a level. I can say Ah, I only need it for the installation, and it looks pretty level, so I don't need the level, or I can say I'm not using this appliance until I've proven to myself that the machine is level and won't cause me any problems in the future because of an imbalance. I've always thought tests passing should be a requisite of make by default. Make fails if tests fail. There should of course be a -skiptests to opt out of testing for those who insist on not doing it, but for the most part, if tests are so important like we the perl community say they are, then they should be run as part of make. Period. Not a popular opinion, but there it is. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Why are we adding more kwalitee tests?
Andy Lester wrote: On Tue, Sep 06, 2005 at 09:12:43AM -0400, Christopher H. Laco ([EMAIL PROTECTED]) wrote: If it serves no purpose for you, ignore it and go on with life; as apposed to spending email list cycles on a CPANTS-is-bad-why-are-we-doing-this diatribe. It's not as simple as just ignore it if the result of your actions are that people stop uploading to CPAN, or new authors are steered away, for fear of scorn and ridicule. Why would they stop uploading? How would they, the new uploaders, even know about CPANTS? It's not like uploaded files automatically return a scathing email and an html response page that says your module sucks; failed CPANTS kwalatee. Go away. This is no different than CPAN Testers. I can upload dists till the cows come home. If my module failes every single cpan testers report, who cares? That doesn't stop or disuade people from uploading does it? Of course not, so why would CPANTS kwalitee be any different? -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Devel::cover bug?
Kevin Scaldeferri wrote: On Jun 3, 2005, at 1:40 PM, Paul Johnson wrote: Certainly. Of course, it's always possible and quite likely that there is a bug in my code somewhere. But there is also a chance that I am conflating two ops, since I have yet to come up with a way to uniquely identify an op (suggestions welcome). You're not running on 5.6.x are you? No, 5.8.x -kevin As I recall [I may be wrong], some of your snippets were under /5.8.0/... isn't 5.8.2 considered squirrelly (technical term) under Devel::Cover? Perl 5.8.0 and 5.8.1 will give slightly different results to more recent versions due to changes in the op tree. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Kwalitee and has_test_*
Tony Bowden wrote: On Thu, Apr 07, 2005 at 08:56:26AM -0400, Christopher H. Laco wrote: I would go as for to say that checking the authors development intentions via checks like Test::Pod::Coverage, Test::Strict, Test::Distribution, etc is just as important, if not more, than just checkong syntax and that all tests pass. CPANTS can't check that for me, as I don't ship those tests. They're part of my development environment, not part of my release tree. Tony That is true. But if you don't ship them, how do I know you bothered to check those things in the first place? [I don't think there is a right answer to that question by the way.] I'm just saying that the presence of those types of tests bumps up some level of kwalittee, and they should be left alone within CPANTS. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Kwalitee and has_test_*
Tony Bowden wrote: On Thu, Apr 07, 2005 at 12:32:31PM -0400, Christopher H. Laco wrote: CPANTS can't check that for me, as I don't ship those tests. They're part of my development environment, not part of my release tree. That is true. But if you don't ship them, how do I know you bothered to check those things in the first place? Why do you care? What's the difference to you between me shipping a a .t file that uses Pod::Coverage, or by having an internal system that uses Devel::Cover in a mode that makes sure I have 100% coverage on everything, including POD, or even if I hire a team of Benedictine Monks to peruse my code and look for problems? The only thing that should matter to you is whether the Pod coverage is adequate, not how that happens. I think you just answered youre own question, assuming you just agreed that I should care about whether your pod coverage is adequate. How as a module consumer would I find out that the Pod coverage is adequate again? Why the [unshipped] .t file in this case. The only other way to tell is to a) write my own pod_coverage.t test for someone elses module at install time, or b) hand review all of the pod vs. code. Or CPANTS. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: TestSimple/More/Builder in JavaScript
David Wheeler wrote: Greetings fellow Perlers, I'm pleased to announce the first alpha release of my port of TestSimple/More/Builder to JavaScript. You can download it from: http://www.justatheory.com/downloads/TestBuilder-0.01.tar.gz Very cool. Very sick. :-) OK, now whos gonna build JPANTS? :-) -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: [Module::Build] Re: Test::META
Ken Williams wrote: Since the 'build', 'test', and 'install' actions are considered the critical path for installing a module, I think it makes sense to warn (not die) during perl Build.PL when one of their required/recommended/conflict dependencies aren't met. Thereafter, only die/warn when running an action and its required/recommended dependencies aren't met. -Ken I'll show my lack of historical knowledge here, and this is swaying just a little off topic. If build, test, and install are considered the critical path, why was Build/make never changed to simple run test always as part of the builds success or failure? Just curious. In a way, I'd be much happier if 'perl Build' or 'make' outright failed if the tests didn't pass, just like if there was a c/linking error. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test::META
Michael G Schwern wrote: [snip] Sticking with ExtUtils::MakeMaker. :-) [But where's the fun in that.] I know you're joking, but you've flipped my rant switch. I was. But at some level, I'm not. If after changing one dist to use M::B I have more issues than I started with [which was just checking the syntax of my manually edited META.yml], then there's no reason to move all of my dists; even if there are only 6 of them. However, the same could be said about my META.yml files period. They weren't broken; just incompleted. I'm the type of idiot that gets the urge to put everything pertinent I can into my META.yml files, even if E::M and M::B don't currently have the means to do exactly what I want. [snip] But it hasn't. And I only see a small number of people patching Module::Build. And there's tons of low hanging fruit available. What's going on here? One thing I see going on is that people are holding Module::Build up to rediculously high standards. Much, much higher than MakeMaker ever was. Anything Module::Build tries to do people still nit-pick it to death, and here's the horrible part, they don't generate much patches. I would think the same is true of any 'replacement' dist. I wonder if CPAN/CPANPLUS don't suffer from the same issue. Take dependency resolution. MakeMaker has one way to specify a dependency. MB has a whole spectrum. And yet people still want to fall back to MM's low resolution dep system because MB's isn't quite high enough. Take create_makefile_pl. Module::Build bends over backwards to be compatible with MakeMaker. It offers not one but THREE different methods of providing that. Hell, it'll even generate a Makefile.PL that will download MB for you! And yet when people encounter small problems with it the response isn't Here's a patch or even I'll just work around that for now. No, its I'm going back to MakeMaker where they'll likely have to do more work and more work arounds to achieve the same effect. Guilty as charged. See top comments. It's not 'going back', it's 'sticking with what already is in place'. I'd be all to willing to take a stab at patching test_requires and the ability to choose whether create_makefile_pl adds recommends: to PREREQ_PM or not during create_makefile_pl. But the former meant getting the META.yml spec updated as well, which didn't seam like something that would happen anytime soon. Maybe that's a bad assumption. [snip] The point is this. * Give MB a chance. * When you encounter a problem in MB, try to patch it. * Do not expect Ken and Randy to do all the work for you. * Do not immediately run back to the warm, familiar, utterly flawed embrace of MakeMaker. Thank you. This has been a rant. So back to M::B I shall go, and I'll make it do my bidding come hell or high patch water. smime.p7s Description: S/MIME Cryptographic Signature
Re: Test::META
Michael G Schwern wrote: On Mon, Mar 28, 2005 at 08:42:50AM -0500, Christopher H. Laco wrote: That's another gripe of mine about M::B and create_makefile_pl. It puts the requires AND build_requires in the PREREQ_PM in the Makefile.PL, which I won't want; nor do I think it right for everyone. There is no build_requires or recommends equivalent in MakeMaker, nor will there be, so putting it into PREREQ_PM is the best thing to do. That's what every MakeMaker-based module on CPAN does after all. Its better than just dropping it and having the build fail. That's a matter of opinion; one I think should be left up to the person makeing Build.PL. Take Test::More for example. It's usually a build_requires and the other Test* things like Test::Strict, Apache::Test, etc are in recommends. Test probably won't run with Test::More, but skipping a few subtests based on recommends is ok. But I don't think build_requires should be a PREREQ_PM requirement at all. *scratch head* but if you don't have the modules necessary to BUILD the module (as opposed to those necessary to run it)... how are you going to build it? See definition below from the docs on what build_requires [ambiguously] means. Maybe there's some confusion here as to what build_requires means? Perhaps you're confusing it with the (possibly mythological) test_requires and test_recommends? Absolutely it's confusion. http://module-build.sourceforge.net/META-spec-new.html A YAML mapping indicating the Perl modules required for building and/or testing of this distribution. These dependencies are not required after the module is installed. So, to me this means something like Test::More. It absolutely has to be in place to some *.t files to ever work, or even load. Now granted the test may end up just skipping because Test::Strict isn't instealled. In that situation, I see Test::More just as required as strict.pm. Maybe this is my issue. To me, building and testing are two different things. I don't have to test. Ever. By I do have to build the module. build_requires is a bad place for test requirements, but recommends isn't right either. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test::META
Randy W. Sims wrote: There is my unpublished CPAN::Metadata at: svn co http://svn.versiondude.net/randys/CPAN-Metadata/trunk/ CPAN-Metadata It reads, writes, and validates metadata according to the spec. It still needs a bit of work, and updates to the actual spec need to be formalized before it will be usefull. Thanks... Along the lines of converting to Module::Build...it went well until I started doing tests...things that worked now break, probably do to how they're now run... t\xsp.Use of uninitialized value in transliteration (tr///) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 99. Use of uninitialized value in pattern match (m//) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 101. Use of uninitialized value in transliteration (tr///) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 99. Use of uninitialized value in pattern match (m//) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 101. The path '' is not an absolute path. Please specify an absolute path The path '' is not an absolute path. Please specify an absolute path The path '' is not an absolute path. Please specify an absolute path So, I've sticking with ExtUtils::MakefMaker for the moment. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test::META
Michael G Schwern wrote: On Sun, Mar 27, 2005 at 08:32:59PM -0500, Christopher H. Laco wrote: Along the lines of converting to Module::Build...it went well until I started doing tests...things that worked now break, probably do to how they're now run... t\xsp.Use of uninitialized value in transliteration (tr///) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 99. Use of uninitialized value in pattern match (m//) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 101. Use of uninitialized value in transliteration (tr///) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 99. Use of uninitialized value in pattern match (m//) at C:/Development/Perl/584/lib/File/Spec/Win32.pm line 101. The path '' is not an absolute path. Please specify an absolute path The path '' is not an absolute path. Please specify an absolute path The path '' is not an absolute path. Please specify an absolute path That's odd, its not where I'd have expected breakage. Want to post your MakeMaker and MB versions so we can poke at them? I rolled back. I'm have to reconsicrate my Build.PL tomorrow. I want to tinker a bit more tomorrow to rule out user stupidity. My bet is that the shebang -w in my *.t files are kicking in where they weren't before; pointing to a difference in how they're invoked between the two envorinments. Along those lines, CTRL-C, CTRL-D, and Terminate batch job.. Y never got me out of the loop: The path '' is not an absolute path. Please specify an absolute path My only recourse was to terminate the CMD window. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Test::META
Is anyone aware of any existing code (aside from YAML) for grocking META.yml? I've got an itch. Aside from user side software tests, I'm also somewhat addicted to developer tests (Test::Strict, Test::Pod, etc) to make sure I'm not making stupid typo mistakes every time I upload a new version of a module. More than once now I've fubared my META.yml just enough that CPAN/Kobesearch won't touch it. I'd like to remedy that problem with a simple Test::META to check my syntax every time I make changes. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test::META
Andy Lester wrote: On Mar 26, 2005, at 6:29 PM, Christopher H. Laco wrote: Is anyone aware of any existing code (aside from YAML) for grocking META.yml? Why are you changing it manually? Well, unless I missed something [likely], to add things after the module is created or updated. Changing requirements, recommends, and build_requires for starters. Sometimes no_index when new modules are added to dists. Changing from the default one create with: # http://module-build.sourceforge.net/META-spec.html #XXX This is a prototype!!! It will change in the future!!! X# to the more clean/proper YAML 1.0... -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: Test::META
Christopher H. Laco wrote: Michael G Schwern wrote: On Sat, Mar 26, 2005 at 07:52:45PM -0500, Christopher H. Laco wrote: Well, unless I missed something [likely], to add things after the module is created or updated. Changing requirements, recommends, and build_requires for starters. Sometimes no_index when new modules are added to dists. Changing from the default one create with: # http://module-build.sourceforge.net/META-spec.html #XXX This is a prototype!!! It will change in the future!!! X# to the more clean/proper YAML 1.0... MakeMaker has very rudimentary META.yml support and is unlikely to improve in the future. Module::Build, OTOH, can handle all this for you automatically. build_requires, recommends, etc... so if you like META.yml, switch to Module::Build. Switch to Module::Build anyway. Reading...reading...reading...reading..done. Still doesn't help with the no_index problem, but it looks interesting. Can dispatch($action, %args) be used to simulate dist = PREP to autogen the pod2text README? -=Chris I missed create_readme. Sorry bout that. Thanks for the push in direction. Time to try converting some dists. -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: TAP/Test::Builder in JS
David Wheeler wrote: Hi All, Is anyone aware of an implementation of Test::Builder/Simple/More and Test::Harness in JavaScript? The testing scene in JS appears pretty sad, but I don't want to do much in JavaScript without a nice testing framework. And Test::More would be my preferred way to go. (Yes, I know that there are JSUnit implementations, but that seems a bit heavy to me...). If not, would anyone like to help me work on one? Regards, David When you say test JavaScript, what kinds of files are we testing? Server side web scripting using JavaScript? Shell scripts files? -=Chris smime.p7s Description: S/MIME Cryptographic Signature
Re: [RFC] Test::CPANTS
Sébastien Aperghis-Tramoni wrote: Christopher H. Laco wrote: I don't know if that answer your needs but Test::Distribution already performs several kwalitee tests on the modules and other files of a distribution. http://search.cpan.org/dist/Test-Distribution/ Sébastien Aperghis-Tramoni -- - --- -- - -- - --- -- - --- -- - --[ http://maddingue.org ] Close the world, txEn eht nepO Yeah, I had looked at that, but there are two things keeping me from using Test::Distribution: prereq Checks whether all use()d modules that aren't in the perl core are also mentioned in Makefile.PL's PREREQ_PM. I have a few modules that are optional; not declared in PREREQ_PM. Probably not a problem since they're evaled in. However, there are plenty of cases where I'm usin-ing one module that is a child of another parent, and the parent is in PRERQ, but the child wouldn't be. versions Checks that all packages define $VERSION strings. Onlt the main module package has a version; not all of the other packages. Maybe that's a bad thing? -=Chris smime.p7s Description: S/MIME Cryptographic Signature
