Re: How to force tests to issue "NA" reports?
On Apr 6, 2005, at 7:13 AM, Robert Rothenberg wrote: Is there a way tests to determine that a module cannot be installed on a platform so that CPANPLUS or CPAN::YACSmoke can issue an "NA" (Not Applicable) report? CPANPLUS relies on module names (e.g. "Solaris::" or "Win32::") but that is not always appropriate in cases where a module runs on many platforms except some that do not have the capability. There's also a separate issue of whether "NA" reports should be issued if a library is missing. (Usually these come out as failures.) Regards, Rob (Co-author of CPAN::YACSmoke) It won't help you *today* but there is a proposed META.yml field called "excludes_os" which fits your request perfectly. http://module-build.sourceforge.net/META-spec-new.html#excludes_os See also "optional_features.foo.excludes_os" http://module-build.sourceforge.net/META-spec-new.html#recommends Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/) PGP.sig Description: This is a digitally signed message part
Re: Kwalitee and Perl::Critic
On Oct 4, 2005, at 1:04 PM, Nik Clayton wrote: I don't have strong opinions about this yet, but has anyone else looked at the Perl::Critic suite of modules on CPAN? It occurs to me that: a) Kwalitee metrics could quite easily be implemented as Perl::Critic plugins. b) The plugins that it ships with (based on Perl Best Practices) may form good kwalitee indicators. As a framework, yes, it has potential for being useful for kwalitee metrics. The PPI foundation that Perl::Critic uses is a very powerful tool for parsing Perl without actually running Perl. Some of the plugins would be good for kwalitee, but many of them are just very well-reasoned opinions. I'm a fan of most of PBP, but many of the ideas are too strict to impose on all of CPAN. Things that PPI/Perl::Critic could judge that might lead to quantitative, non-controversial metrics: * are the variables/functions consistently named? (i.e. all of them in_underscore_style or allInCamelCase) * what's the ratio of globals to subroutines? (smaller is better) * is there a SYNOPSIS section and is it valid Perl? * do OO-style modules bless into a hardcoded package or a passed package name? (former kills subclassing) * are there any undeclared, non-core dependencies? * does the module use Perl 5.6.x+ constructs without declaring "use 5.006"? * does the module use platform directory syntax or portable tools? (e.g. File::Spec, Path::Class) All of these would be very hard to judge without parsing the module. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Kwalitee and Perl::Critic
On Oct 5, 2005, at 6:18 AM, A. Pagaltzis wrote: * what's the ratio of globals to subroutines? (smaller is better) Does that include file-scoped lexicals? ’Cause in that case I disagree – I’m just overhauling a module, in which process I’m also moving it to inside-out object style, and so I’ve got about 20 file-scoped lexical hashes. But that doesn’t at all mean the code is really full of globals. My fault for sloppy terminology -- I did not mean to include lexicals. I meant 'use vars qw(...)' and 'our ...' variables specifically. Chris -- Chris Dolan, Software Developer, www.chrisdolan.net Public key: http://www.chrisdolan.net/public.key
Proposed Kwalitee tests: has_license and/or has_meta_yml_license
In the last year as a Fink maintainer (Mac OS X debian-like package manager), I've come across a couple CPAN modules that have no license information at all. It's very frustrating. I've submitted RT bugs, but one of them has been fixed (thanks Ken Williams). To encourage authors to correct this oversight, I propose a new pair of Kwalitee tests. Both would be nice, but if either of them were implemented, I'd be thrilled. I'd prefer that someone else implement the test (lack of tuits), but if there is approval for the idea without a motivated implementer I will take a hack at it. 1) has_license -- check for the presence of a file named something like LICENSE or COPYING or COPYLEFT or GPL or ... (each test case insensitive, with or without .txt extensions). Alternatively, the test can be more liberal by looking for the string "copyright" in README, *pm and *.pod. 2) has_meta_yml_license -- check for a META.yml field named "license". Module::Build supports this. These tests should not care which license is claimed, just that there is a license present. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Devel::Cover not DWIMming on upgraded Perl -- but problem solved
James,
The explanation is that D::C saves a copy of your @INC at build time
in a .pm and subsequently uses that list to filter out the code perl
libraries. That @INC probably had some /5.8.4/ subdirs in it. With
the new Perl, the @INC now has /5.8.7/ subdirs and those aren't being
filtered.
When I first discovered that detail, I thought "Why can't D::C just
look at @INC at runtime?" The answer, of course, is that you've
probably changed @INC with -Mblib or -Ilib or 'use lib qw(.);' or
something. So, there has to be a pristine @INC to reference.
I haven't fully thought this out, but it seems to me that some
combination of @INC from %Config and $ENV{PERL5LIB} should cover
pretty much all of core, so perhaps D::C can do away with the hard-
coded @INC filter list in the future?
Chris
On Nov 1, 2005, at 7:25 PM, James E Keenan wrote:
When I began to write this posting, it was to get an answer to a
question. But I figured out a workaround halfway through, so now
I'm posting an answer.
I have happily been using Devel::Cover for more than a year on Perl
5.8.4 on Darwin (Mac OS X 10.3). Recently I upgraded to Perl
5.8.7. Tonight I went to use Devel::Cover (v0.55, as previously)
for the first time since that Perl upgrade. At first, it no longer
DWIMmed. Specifically:
1. When I ran 'make test HARNESS_PERL_SWITCHES=MDevel::Cover', I
got this message:
t/01_test..
This version of Devel::Cover was built with Perl version 5.008004.
It is now being run with Perl version 5.008007.
Attempting to make adjustments, but you may find that some of
your modules do
not have coverage data collected. You may need to alter the +-
inc, +-ignore
and +-select options.
t/01_test..ok
I'd never had to set these options before; previously, D::C just
worked. Also, D::C took much longer to run.
2. I was testing coverage of a new module I'm developing called
File::Save::Home. When I called 'cover', I got a report on this
module, but also on every core module as well:
[snip]
-- -- -- -- --
-- --
File stmt bran condsubpod
time total
-- -- -- -- --
-- --
...perl5/5.8.7/AutoLoader.pm0.00.00.00.00.0
n/a0.0
...l/lib/perl5/5.8.7/Carp.pm0.00.00.00.00.0
n/a0.0
...erl5/5.8.7/Digest/base.pm0.00.0n/a0.00.0
n/a0.0
...b/perl5/5.8.7/Exporter.pm 50.0 57.1 44.7 33.30.0
0.3 44.8
...5/5.8.7/Exporter/Heavy.pm 10.48.8 12.5 11.10.0
0.19.8
...l5/5.8.7/File/Basename.pm 28.2 25.87.7 50.0 100.0
0.1 27.8
[snip]
8.7/warnings/register.pm 100.0 50.0n/a 100.00.0
0.1 89.5
blib/lib/File/Save/Home.pm 77.1 44.4n/a 100.0 100.0
0.3 73.4
Total 10.86.24.3 12.3 14.0
100.09.0
-- -- -- -- --
-- --
Writing HTML output to
/Users/jimk/work/fsh/File-Save-Home/cover_db/coverage.html ...
done.
This problem of excessive output is the same one I typically
experience using D::C (v0.47, I believe) on Windows. This is much
more output than I want or need.
I hypothesized that if I were to upgrade to a new version of D::C,
it would be compiled against Perl 5.8.7 and these problems might go
away. But 'cpan' did nothing because it detected that I was using
the most recent version of D::C.
So I downloaded the latest version of Devel::Cover from CPAN and
installed it manually. (I did call 'make install UNINSTALL=1'.)
This solved the problem. The message described above went away,
and 'cover' reported only the results for the module under
development.
Jim Keenan
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Proposed Kwalitee tests: has_license and/or has_meta_yml_license
On Nov 2, 2005, at 10:19 AM, David Landgren wrote: Chris Dolan wrote: In the last year as a Fink maintainer (Mac OS X debian-like package manager), I've come across a couple CPAN modules that have no license information at all. It's very frustrating. I've submitted RT bugs, but one of them has been fixed (thanks Ken Williams). To encourage authors to correct this oversight, I propose a new pair of Kwalitee tests. Both would be nice, but if either of them were implemented, I'd be thrilled. I'd prefer that someone else implement the test (lack of tuits), but if there is approval for the idea without a motivated implementer I will take a hack at it. 1) has_license -- check for the presence of a file named something like LICENSE or COPYING or COPYLEFT or GPL or ... (each test case insensitive, with or without .txt extensions). Alternatively, the test can be more liberal by looking for the string "copyright" in README, *pm and *.pod. 2) has_meta_yml_license -- check for a META.yml field named "license". Module::Build supports this. That would suck, you may as well propose a Kwalitee bit for modules that use Module::Build. I know that the current alpha of ExtUtils::MakeMaker supports this, but until it is released as stable *and* module authors have the time to upgrade EU::MM *and* release a new version of their module (s), those authors will be penalised through no fault of their own. David What penalty? The whole point of Kwalitee is not to reward authors who jump through hoops, but to encourage authors to live up to community expectations. That includes good packaging, good POD and, I say emphatically, clear licensing. Anything we can do to encourage authors to more clearly state their license is a good thing. If that in turn means encouraging them to 1) use Module::Build, 2) upgrade EU::MM or 3) hand-edit META.yml, then I think that's a burden worth bearing. You're complaining that its too big a burden to clearly state your module's license? To me that's just crazy. To some people, the license is actually more important than the module (e.g. if I can only redistribute Artistically license code). After all, Kwalitee is not an entrance barrier to CPAN. It's a tool to let authors know what is important to the rest of us. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Proposed Kwalitee tests: has_license and/or has_meta_yml_license
On Nov 2, 2005, at 12:13 PM, David Landgren wrote: No. I'm complaining that there's no need for two different Kwalitee points for this, that's all. I think one is sufficient (and a very worthy one I should add, in case I wasn't being clear, which I probably wasn't). Ahh, that was unclear from your message. I thought you were protesting the addition of a new Kwalitee test in general, hence my annoyance. Apologies for the misunderstanding. I originally advocated for one of the two methods of determining the license, not necessarily for both. Chris P.S. On a marginally-related note, I released Module::License::Report yesterday. Unfortunately, it's not immediately useful for Kwalitee because it eval()s code from Makefile.PL and/or Build.PL, and makes use of Module::Depends::Intrusive which does more of the same. -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Private tests
Hello all, I've just published an article about public vs. private regression tests. I've defined private tests as t/*.t files that are for the author only and don't go in MANIFEST. Naturally, those don't get as much publicity as tests included in CPAN distributions. In the article I advocate that some tests should be private. In particular, 1) those that test non-critical aspects of a module (like documentation and coding style) 2) those that are too expensive to run often 3) those that require special software or customization In my conclusion I describe a possible system where authors publish the results of private tests with their distributions as a trust- based kwalitee system. That is, authors assert kwalitee rather than be judged for it. http://www.chrisdolan.net/talk/index.php/2005/11/14/private- regression-tests/ Both positive and negative feedback is very welcome! Chris -- Chris Dolan, Software Developer, www.chrisdolan.net Public key: http://www.chrisdolan.net/public.key
Private tests
Hello all, I've just published an article about public vs. private regression tests. Private ones are t/*.t files that are for the author only and don't go in MANIFEST. Naturally, those don't get as much publicity as tests included in CPAN distributions. In the article I advocate that some tests should be private. In particular, 1) those that test non-critical aspects of a module (like documentation and coding style) 2) those that are too expensive to run often 3) those that require special software or customization In my conclusion I describe a possible system where authors publish the results of private tests with their distributions as a trust- based kwalitee system. http://www.chrisdolan.net/talk/index.php/2005/11/14/private- regression-tests/ Both positive and negative feedback is very welcome! Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Private tests
Tels, I believe you have misunderstood my intentions. I was not advocating that any algorithmic tests be non-public. The only tests that should be private are ones that satisfy one or more of the following restrictions: 1. require special additional software that’s difficult or expensive to acquire, 2. require special configuration to run properly, 3. don’t affect the quality of the final software, or 4. take too long to run. That is, tests like spell-checking, pod-checking, Perl::Critic and ones that take tens of minutes to run. Naturally, any test whose results may vary from machine to machine should be public wherever feasible. After reading some of the insightful comments posted on my blog, I've been convinced that the private tests should be included in the CPAN distribution, but disabled in some way (perhaps via a file extension other than .t?). That way, resourceful or interested users can run the tests but average users don't need to. The example I included at the end of the article stating a failed test under Windows was only tangentially related and perhaps was a distraction. I was saying that perhaps it would be useful if authors could assert things about their personal testing experience that would be machine readable and useful to others. In that example I was suggesting that the author could announce that the hypothetical module was known to fail under Windows and, as a TODO test, was an implicit request for help from other developers. Chris On Nov 15, 2005, at 4:12 AM, Tels wrote: -BEGIN PGP SIGNED MESSAGE- Moin, On Monday 14 November 2005 18:21, Chris Dolan wrote: Hello all, I've just published an article about public vs. private regression tests. I've defined private tests as t/*.t files that are for the author only and don't go in MANIFEST. Naturally, those don't get as much publicity as tests included in CPAN distributions. In the article I advocate that some tests should be private. In particular, 1) those that test non-critical aspects of a module (like documentation and coding style) 2) those that are too expensive to run often 3) those that require special software or customization In my conclusion I describe a possible system where authors publish the results of private tests with their distributions as a trust- based kwalitee system. That is, authors assert kwalitee rather than be judged for it. http://www.chrisdolan.net/talk/index.php/2005/11/14/private- regression-tests/ Both positive and negative feedback is very welcome! Private tests will only be run by the author, meaning they will be only run on a very small subset of all systems the modules can be used on. This limits their usefullness quite a bit. Case ein point: I can test my modules on linux, 32 bit, unthreaded, under unicode, and under perl 5.8.x. Thats about it, everything else gets really really complicated for me to set up and maintain/test. So, no win32, no mac, now solaris, no irix, no perl 5.6.x, no iso-something, no EBDIC (or however it is spelled), no threading, no 64 bit, no SMP system. As for 1), these things should matter (the "broken window analogy") and you would be surprised to know how these tests can pass on your system, and still fail on other systesm (forget to include the .pod file in MANIFEST is the most obvious one). As for 2), random testing should be employed (Math::BigInt does this, it runs 256 or so tests with random number patterns (and thus known results like "2 * A - A == A"). The tests are quite fast, but they cover only a small subset of potential values. However, since each system and user runs a new, different random set, you end up with a really huge testing number being run. (Yes, this has found some bugs) And for 3), this might be the only point I can think that private tests are usefull (I have a private testset for Graph::Easy that I use from time to time, it is not public mainly because it fails/hangs/takes forever and is work-in-progress). However, I have to actually read your article to find out what your proposal solves (compared to me just running thetest once in a while :) Hope that was usefull :) Best wishes, Tels - -- Signed on Tue Nov 15 11:04:21 2005 with key 0x93B84C15. Visit my photo gallery at http://bloodgate.com/photos/ PGP key on http://bloodgate.com/tels.asc or per email. "Now, _you_ behave!" -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.4 (GNU/Linux) iQEVAwUBQ3m0pHcLPEOTuEwVAQF6lQf8CtubDMQjLdCBcEGNczxZj2Y1kTVhOU7Z bvweeJe4RWFfmd2JMw7dwiu3Sjb57hNlVkl4LwN+7vx3tm3DsnhRUoMHvkDtCddC 8bfxpLcOi8WMlySAud+unKnpZVwlwn2rZ/enu2Dd01QKOgOQkBr1HWTJUguPv4No eWT3UiEZhV4hU764gtF7a8raHjbvxpTJcNk22KHnRmyTKX+SugCyI0qkmIQrFntl cQWXyA9GV7V+5bK5/Sp2eWv2MXX3fhNDxtZywkmqun+6/YhPgSDJQp3FcKThZFYy WxPXsrXVIXFJbbtkSs+PGe18VdXqEFCHOI29H1+9gyiC3FW3N6AARw== =GA3y -END PGP SIGNATURE
Re: Private tests
On Nov 15, 2005, at 3:38 PM, chromatic wrote: I posted a small Module::Build subclass that shows one way to do this to Perl Monks: http://perlmonks.org/?node_id=508160 Yeah, I saw that one. Perhaps a better approach is to store these tests in a subdirectory of t/. Beware that M::B has a recursive mode for finding tests. It's set by the author, so you should be safe in this case, but it's a point worth remembering. *light bulb* And in fact, that could be the run-time trigger. Hmmm Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: How to use Devel::Cover?
On Dec 25, 2005, at 3:46 PM, Scott Wang wrote: I am new to use Devel::Cover. We have lots of product Perl modules in our product "lib" folder and we have lots of Perl test scripts to cover those modules, right now, we are trying to get the code coverage metrics for our tests on those modules. I find we might get help by using Devel::Cover, would you please give me some idea on how I should use the Devel::Cover to generate our tests coverage metrics on our Perl modules? Thanks in advance, Scott If you use Module::Build, then simply type "./Build testcover" and then open cover_db/coverage.html in your browser. If you are using a version of M::B older than 0.27_02, you may wish to manually delete the whole cover_db directory between runs. Later M::B versions auto- detect when cover_db files are out of date. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Dependency trees was: CPAN Upload: D/DO/DOMM/Module-CPANTS-Analyse-0.5.tar.gz
On Jan 27, 2006, at 11:23 AM, Tels wrote: Basically something like CPAN, but with much less network traffic and much less hassle for a user. Bonus points if it gives you stuff pre- compiled for windows (all those ppl w/o a compiler). I think you just described ActiveState's Perl Package Manager (PPM). http://aspn.activestate.com/ASPN/Downloads/ActivePerl/PPM/ Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Dependency trees was: CPAN Upload: D/DO/DOMM/Module-CPANTS-Analyse-0.5.tar.gz
On Jan 27, 2006, at 12:01 PM, Tels wrote: On Friday 27 January 2006 18:48, Chris Dolan wrote: On Jan 27, 2006, at 11:23 AM, Tels wrote: Basically something like CPAN, but with much less network traffic and much less hassle for a user. Bonus points if it gives you stuff pre- compiled for windows (all those ppl w/o a compiler). I think you just described ActiveState's Perl Package Manager (PPM). http://aspn.activestate.com/ASPN/Downloads/ActivePerl/PPM/ I lived under the expression that it is: * for windows only * only includes Foo-Bar, but not it's dependecies It will auto-install dependencies just like CPAN, I believe. And, yes, it's currently Windows-only. Didn't you offer bonus points for Windows?? Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Dependency trees was: CPAN Upload: D/DO/DOMM/Module-CPANTS-Analyse-0.5.tar.gz
On Jan 27, 2006, at 12:30 PM, Tyler MacDonald wrote: Chris Dolan <[EMAIL PROTECTED]> wrote: * for windows only * only includes Foo-Bar, but not it's dependecies It will auto-install dependencies just like CPAN, I believe. And, yes, it's currently Windows-only. Didn't you offer bonus points for Windows?? Um, no it isn't! http://ppm.activestate.com/ PPMs are built for a variety of platforms, windows, OSX, and various unixes. You can download ActivePerl for these platforms here: http://www.activestate.com/Products/Download/Download.plex? id=ActivePerl - Tyler Sweet! I didn't know that. Yay ActiveState! Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Dependency trees was: CPAN Upload: D/DO/DOMM/Module-CPANTS-Analyse-0.5.tar.gz
Tels,
Please forgive me for being blunt, but I think it's your fault for
writing fragile META.yml creation code. YAML.pm is not even at 1.00
yet, so an API change is allowed by convention, and lack of backward
compatibility is to be expected. If you were to have wrapped your
whole custom META.yml creation code in a big eval, there would not
have been a problem. Furthermore, it seems to me that you are
reinventing the wheel by adding custom META.yml code to the
Makefile.PL of every package you write instead of, say, using
Module::Build or just omitting the non-essential META.yml customization.
The ultimate solution, of course, is that there will be a standard
way to generate proper META.yml. In the meantime, however, it is my
opinion that we're better off lacking META.yml than having a
proliferation of different solutions to the META.yml issues.
Chris
On Jan 28, 2006, at 4:26 AM, Tels wrote:
Moin,
On Saturday 28 January 2006 11:08, Adam Kennedy wrote:
Graph::Easy installation failing here with YAML 0.50 (newer
versions of
YAML seem to be uninstallable at the moment due to Class::Spiffy +
Spiffy + Test::Base install failures...
Just what I said about dependecies etc. My Makefile.PL uses YAML
for me
(e.g. the author) to create a META.yml file with a license field,
something that MakeMaker (could?) can't do.
I used "require YAML;" to avoid that the user has to has it installed.
Unfortunately, YAML got changed, and now you also need "require
YAML::Node;" for my Makefile.PL working properly. :-(
So you can either:
* patch my Makefile.PL
* patch YAML to work like the previous version, then update it on
CPAN,
then install it
The latter is way more work, and needs time and due to what you wrote
above, might even not work.
I'd say keep the dependencies out of YAML ("KISS"!). Whether YAML
should
also load YAML::Node when you do "require YAML;" is a point for
discussion, but it certainly tripped up a *lot* of existing
Makefiles and
I don't have the time to patch them all because that requires me to
release a dozend or so modules.
Sorry for the problem, but it is only partly my fautl :)
Best wishes,
Tels
--
Signed on Sat Jan 28 11:22:01 2006 with key 0x93B84C15.
Visit my photo gallery at http://bloodgate.com/photos/
PGP key on http://bloodgate.com/tels.asc or per email.
This email violates EU patent EP0394160:
[ ## 66% ]
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Kwalitee in your dependencies (was CPAN Upload: etc etc)
On Jan 30, 2006, at 10:04 AM, David Cantrell wrote: [...] for example, on OS X, HFS+ is case-preserving but case- insensitive. UFS is case-sensitive. And FAT16 smashes case. And to make matters even worse (better?) Apple added a case-sensitive mode to HFS+ in 10.4. It's not widely used. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: YAML and Makefile.PL (was various topics)
On Jan 31, 2006, at 2:30 AM, Adam Kennedy wrote: Solutions that can be implement without either betting incredibly aggressive to users, putting MB or CPANPLUS into the core, or going down the bundled route, would be interesting to hear though. If I've read the perl5-porter summaries correctly, M::B just entered core in 5.9.3. There has long been hope that M::B and CPANPLUS will be in core for 5.10. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: YAML and Makefile.PL (was various topics)
On Jan 31, 2006, at 2:32 AM, Adam Kennedy wrote: But then how are you going to make Bundle::CPAN trigger when there's a critical failure, without requiring the user to be smart? CPAN.pm and CPANPLUS.pm already check their own version against CPAN on every run. All we'd need to do is have Andreas or Jos increment the CPAN.pm/CPANPLUS.pm version and just about everyone would be prompted to upgrade the next time they tried to install something. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: TODO test paradox: better TODO test management?
On Jan 31, 2006, at 10:40 AM, demerphq wrote: While only indirectly related. Lately when ive built blead I see stuff like "10 TODO tests unexpectedly passed!" So far ive not found a way to make harness tell me which they are... So i too would like a report of TODO tests, both those passing and those failing. You could try: make test verbose=1 > out and then search the output for "TODO" Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: [Module::Build] [RFC] author tests
On Feb 1, 2006, at 10:35 PM, Tyler MacDonald wrote: Chris Dolan <[EMAIL PROTECTED]> wrote: There is a class of tests that module authors perform that end users are not expected to run. For example code coverage tests, spelling tests, coding style tests, etc. These tests are either prohibitively expensive or complicated or unpredictable for end users to run. I call these "private tests" or author tests. I really like this idea. But as you pointed out, it's not just authors that need to worry about running these tests, it's packagers (ppm/deb/etc), automated testers (cpants/testers.cpan.org/etc), and hackers. I'd suggest we call these "exhaustive" tests. No, I disagree. I'm specifically talking about author tests, NOT packager tests. Things like Test::Spelling are pointless and difficult for packagers to execute because Test::Spelling relies on an external aspell or ispell program *and* performs differently in the presence of an author's custom dictionary (mine has "Dolan"; does yours?) These specifically are not exhaustive tests but spit-and-polish tests. To make this less abstract, let me list the specific author tests that I employ for most of my CPAN modules, along with an explanation of why it wouldn't work for a packager * copyright.t - Ensures that there is a "Copyright ".([localtime]-> [5]+1900) somewhere in every .pm file. Will break 11 months from now. * distribution.t - Relies on Test::Distribution, which is not in my prereq list * perlcritic.t - Runs Test::Perl::Critic on all .pm files. Will fail without my specific $HOME/.perlcriticrc and will fail with future, more exhaustive versions of P::C * spelling.t - Runs Test::Spelling. Will fail without my custom dictionary * versionsync.t - Checks that the $VERSION is the same in all bin/* and *.pm files. This test is pointless after release, since it's already been tested before release * pod.t - Checks POD validity. This test is pointless after release, since it's already been tested before release * pod-coverage.t - Checks POD completeness. This test is pointless after release, since it's already been tested before release and one I have not yet employed: * coverage.t - Ensures that Devel::Cover totals are higher than some threshold Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: [Module::Build] [RFC] author tests
On Feb 2, 2006, at 9:19 PM, Yitzchak Scott-Thoennes wrote: Chris, how are you currently set up to run these tests only when preparing a release? I make no such distinction. Instead, I see these tests as part of my day-to-day development and run all of them with every "./Build test". Presently, I simply include them all in my MANIFEST.SKIP so they are not included in my uploaded package. That's why I call these "author tests" and not "release tests" or "exhaustive tests". Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Default tests, beta testing, etc.
On Feb 5, 2006, at 2:36 PM, Bryce Harrington wrote: [snip] It would seem to be useful if CPAN were to have a couple really basic tests like this to run for perl modules that haven't created any tests themselves. Obviously, this only benefits immature modules, but this might be a good way to motivate new authors to put more attention into testing. I recommend Test::Distribution for this type of "default" tests. It looks for .pm files in blib and performs a bunch of tests on them -- POD, compile, version, etc. [snip] Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Default tests, beta testing, etc.
On Feb 6, 2006, at 1:37 AM, Adam Kennedy wrote: In fact, what you just asked is already listed in the PITA documentation as within it's scope. For lack of a better name, I've called it Fallout Testing. As opposed to Rot Testing, which is when your module doesn't change, but makes sure it still works as Perl is upgraded and your dependencies evolve. Since I've got a terribly bad habit of inventing things that have existing names, I'm sure someone will correct my on the two above namings. I think those are great names. Very clear. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Surprising use_ok false positive
I encountered a real-world case where Test::More's use_ok() passed
when the specified package had a fatal syntax error. I'm looking for
advice about whether I should file a bug, or fix my code that
triggered the false positive. Read on for details...
Consider two packages Foo.pm and Bar.pm:
--- Foo.pm ---
package Foo;
my $has_Bar = eval {require Bar; 1;};
sub print_msg {
$has_Bar ? Bar->print("Hello") : print "Hello";
}
1;
--- Bar.pm ---
package Bar;
sub print { # egregious syntax error
1;
--- t/use.t ---
use Test::More tests => 2;
use_ok('Foo');
use_ok('Bar');
Then if I run "perl t/use.t" I get the surprising result:
1..2
ok 1 - use Foo;
ok 2 - use Bar;
The problem is that Foo trapped Bar's syntax error in the eval, so %
INC{Bar} is defined and use_ok() sees it as a success.
Advice? While this example is contrived, the "eval { require ... }"
idiom is used often in the wild, so this is not a wholly unrealistic
scenario.
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Surprising use_ok false positive
On Mar 5, 2006, at 3:55 PM, David Wheeler wrote:
On Mar 5, 2006, at 13:52, Chris Dolan wrote:
Advice? While this example is contrived, the "eval
{ require ... }" idiom is used often in the wild, so this is not a
wholly unrealistic scenario.
Of course it should be
eval { require Bar; 1; } or die $@;
But I agree that it seems like if the load failed in your eval, it
should fail for Test::More, too. But maybe even a failed require
sets the record in %INC?
In this case, Bar.pm is intended to represent optional functionality
that Foo.pm conditionally loads. So, adding a die would be
counterproductive. The problem is that I don't know how to
distinguish between a load failure or a compile failure. There must
be a way to do that right?
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Best Practice for testing compilation of scripts
On Mar 15, 2006, at 2:47 PM, Geoffrey Young wrote: chromatic wrote: On Wednesday 15 March 2006 12:25, Jeffrey Thalhammer wrote: I'm sure I could clean this up by opening a pipe instead of using backticks and output redirection. But even that doesn't smell very good. I've looked around on CPAN, but I have not yet found a Test:: module that seems appropriate. I also wondered if fiddling with $^C would do the trick somehow. Any suggestions? Thanks. I've long intended to take t/test.pl from the Perl core distribution and wrap up at least its runperl() in a Test:: module. Perhaps that would work for you? compile_ok() ? --Geoff It is unclear from Geoff's message above whether he is asserting that function exists, or if he is merely proposing it, so I googled. It doesn't appear to exist in any Test:: package (corrections VERY welcome). However, Adam Kennedy wrote one: http://search.cpan.org/src/ADAMK/ThreatNet-Bot-AmmoBot-0.10/t/ 01_compile.t That function should probably be changed to use FindBin instead of updir() calls. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Upgrading core modules on Windows
On Mar 16, 2006, at 8:51 AM, David Golden wrote: ... So why not bundle a snapshot of all the module dependencies for PPM/ cpan.pm into a separate directory and put that at the start of @INC when running PPM/cpan.pm? ... Makes sense to me. PAR would be a very valuable tool to implement this, although that would be a huge addition to the dependency list. With PAR, you could employ a whole separate .exe and probably even do the core updates. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: [PATCH] Forking tests with Test::More
On Mar 28, 2006, at 4:55 AM, Tassilo von Parseval wrote:
Do you think this might work better, or could be implemented as, a
seperate Test::Fork type module?
It certainly could be done. But it would essentially share 90% of its
code with Test::Builder.
Umm, subclassing? :-)
It's simple really: Either my proposed method
is robust in which case it can go into Test-Simple. Or it isn't. Then
there's no need to implement it as a separate module. :-)
I don't think it is that simple. *IF* there are any bugs in this
niche feature, then it could break the most popular test module on
CPAN. Clearly from your proposed patch, any code that needs the fork
feature of Test::More says so explicitly via the $self->{Forked}
property. So specifying a subclass instead, like a hypothetical
Test::More::Forked, should be trivial. I have not yet seen any
justification for adding this feature to Test::More itself.
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Module requirements
On Apr 5, 2006, at 9:06 AM, David Landgren wrote: perl -MModule::CoreList -le 'print Module::CoreList->first_release ($_) for @ARGV' File::Find Memoize 5 5.007003 (um, that can no doubt be golfed, but you get the picture). Yes, it can: % corelist File::Find Memoize File::Find was first released with perl 5 Memoize was first released with perl 5.007003 Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: CPANTS is not a game.
On May 23, 2006, at 8:39 AM, David Golden wrote: How does "is_prereq" improve quality? Or, put differently, how does measuring something that an author can't control create an incentive to improve? is_prereq is usually a proxy metric for software maturity: if someone thinks your module is good enough that he would rather depend on it than reinvent it, then it's probably a better-than-average module on CPAN. is_prereq is usually a vote of confidence, so it is likely a good proxy for quality. In fact I believe that because the author (usually) can't control it directly, is_prereq is one of the best proxies for quality among the current kwalitee metrics. CPANTS by itself does nothing to improve quality. However, by drawing attention to kwalitee metrics, I argue that CPANTS draws attention to quality too. Even if many authors don't understand that, the ones that do makes CPANTS worthwhile. If making it a game helps to further raise awareness, then I think that should be tolerated until CPANTS matures. IMHO, the best way to improve CPANTS and move away from the game mentality is to continually add more tests. Each added test diminishes the weight of previous tests. This will annoy the "gamers" because their modules keep dropping in kwalitee, while those that genuinely care about quality will appreciate the additional measurements. If some gamers get annoyed enough to quit the game, that's not a big deal because they didn't really understand the point of CPANTS anyway. If some keep playing the game by cleaving to the standards the community sets for them, then all the better for the rest of us. As an example, consider pod_coverage. It's a rather annoying metric, most of us agree. Test::Pod::Coverage really only needs to be run on the author's machine, not on every user's machine. However, by adding pod_coverage to kwalitee we got LOTS of authors to improve their POD with the cost of wasting cycles on users' machines. I think that's a price worth paying -- at least until we rewrite the metric to actually test POD coverage (which is a decent proxy for POD quality) instead of just checking for the presence of a t/ pod_coverage.t file (which is a weak proxy for POD quality, but dramatically easier to measure). Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: CPANTS is not a game.
On May 23, 2006, at 10:15 AM, H.Merijn Brand wrote: is_prereq is usually a vote of confidence, I respectfully disagree completely. It's been more than once that I did *not* install a module because it required a module that I did not trust, either because of (the programming style of) the author of that module, or because that module required yet another zillion things I do not want installed (think YAML). I believe we are largely in agreement, but I think your example demonstrates that you missed my point. As you well know, CPANTS is not making recommendations whether a module is a good solution for your problem, or whether you should trust a given module. Instead, it is currently measuring maturity of a module and the author's attention to detail. is_prereq just measures whether *any* other humans trust the module in question. In that way, is_prereq is like a simplistic binary version of Google's page rank. Just because Google rates a page highly doesn't mean it's a good page. Similarly just because CPANTS ranks a module highly doesn't mean it's a good module. However, both is_prereq and page rank are among the current best automatable proxies we have for approximating human judgment of quality. Yes, there are great modules with is_prereq of 0 and there are great web sites with low page ranks. But in both cases they're harder to find than their highly-linked counterparts, except via word of mouth (perlmonks, cpanratings, etc). I keep advocating for is_prereq because currently it's the only non- author-controlled input to CPANTS. That's it's primary value, and it will continue to be important until some better proxy for human confidence comes along, like incorporating cpanratings into CPANTS (I do NOT advocate that!) or getting download stats from CPAN (never gonna happen) or adding voluntary "Someone installed module X" pings from CPAN.pm. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: CPANTS is not a game.
On May 23, 2006, at 10:34 AM, David Golden wrote: Chris Dolan wrote: ... just checking for the presence of a t/pod_coverage.t file (which is a weak proxy for POD quality, but dramatically easier to measure). It doesn't check for the existence of a t/pod_coverage.t file. It checks that a string like "use Test::Pod::Coverage" appears properly formatted. E.g. I believe this is sufficient to get the Kwalitee point: # t/pod_coverage.t __END__ use Test::Pod::Coverage; And, unfortunately, it also misses actual perl that doesn't meet its regex expectations. (E.g. see the bug I recently filed for Module::ExtractUse.) Point taken, apologies for the inaccuracy. However, that supports my argument that pod_coverage is a weak proxy. I say it's much better than nothing, but still weak and the brittleness documented above makes it weaker. Actually, I'd rather see a robust pod_coverage that just checks for the existence of t/.*pod_coverage.t than a slightly brittle that parses that file. That is, I'd rather see false positives than false negatives. To put it another way, I'll tolerate cheaters to avoid annoying the well-intentioned authors. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Continuous testing tools
On Jun 8, 2006, at 10:39 AM, Tels wrote: On my todo (well, wish list) is still a project that works rouhgly like a server/client model. You upload a snapshot to the server, it notifies the clients, they download the package, run the tests and report the result back. Reports are viewed on the server. Is there any interest in such a package? Best wishes, Tels That sounds very similar to Adam Kennedy's PITA project. Yes, there is great interest! Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: TAP Grammar
On Jul 3, 2006, at 6:33 PM, chromatic wrote: On Monday 03 July 2006 09:01, Jonathan T. Rockway wrote: That said, I would be interested. I'm still trying to page all the perl6/parrot grammars (PGE, TGE, etc.) into my brain, so any additional examples would helpful, interesting, and fun. For me, anyway :) Jerry Gay just wrote a PGE TAP grammar: <http://svn.perl.org/parrot/trunk/examples/pge/grammars/TAP.pg> -- c I hacked a rudimentary test case for that grammar, borrowing heavily from Punie: https://svn.clotho.com/public/parrot-experiments/TAP/ To run it, edit the first line of the Makefile to specify the path to PARROT_DIR and run "make". My tests found a few bugs in Jerry's grammar. My fixed version of that grammar is here: https://svn.clotho.com/public/parrot-experiments/TAP/grammars/TAP.pg Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Old and broken versions of Module::Install
On Jul 6, 2006, at 8:10 AM, Steffen Mueller wrote: David Golden schrieb: What about adding NO_BROKEN_INSTALLER as a Kwalitee point for CPANTS? Maybe tongue-in-cheek, but it's actually a good idea. Module::Install pre-0.61 is certainly "broken". What does Module::Build count as? ;) Steffen I agree, this is a really good idea. I would call it "UP_TO_DATE_INSTALLER". For the present, it could simply test for old Module::Install. In the future, it could be expanded to look for compatibility problems in Build.PL or Makefile.PL (e.g. a Makefile.PL using out-of-date Module::Build::Compat boilerplate). If the CPANTS "game" gets any M::I-using authors to upgrade, then it has served its purpose. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Old and broken versions of Module::Install
On Jul 6, 2006, at 9:18 AM, A. Pagaltzis wrote: * Steffen Mueller <[EMAIL PROTECTED]> [2006-07-06 15:20]: What does Module::Build count as? ;) I use M::B with `create_makefile_pl => 'traditional'`. What does that count as? Regards, -- Aristotle Pagaltzis // <http://plasmasturm.org/> IMHO it likely counts as a CPANTS pass for now, but it may be a fail in the future unless you update the Makefile.PL with a newer M::B as bugs are discovered. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Old and broken versions of Module::Install
On Jul 6, 2006, at 10:18 AM, A. Pagaltzis wrote: * Chris Dolan <[EMAIL PROTECTED]> [2006-07-06 16:30]: IMHO it likely counts as a CPANTS pass for now, but it may be a fail in the future unless you update the Makefile.PL with a newer M::B as bugs are discovered. To make sure we’re on the same page: that would be bugs in M::B::Compat, not M::B itself, right? If that isn’t accepted, then I will revert to EU::MM (with apologies to Schwern) until you all figure out the damn installers among yourselves. Yes indeed, that's what I meant. I'm not saying there are M::B::Compat bugs, just *if* any were discovered someday, they could be flagged by CPANTS. Anywhere you have boilerplate code, you have the possibility of bugs that need to be fixed via the good intentions of authors deploying that boilerplate. M::B- and EU::MM-using authors generally create minimal boilerplate and thus have a central point of upgrade. M::B::Compat is the only exception I could think of, so it became my example in my effort *not* to incite an M::I vs. M::B flamefest. Apologies for any confusion. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: [Slightly OT] Understanding Software Licences [was Re: Proposal Suggestion - Test::Run [was Re: [Israel.pm] Fwd: Call for proposals -- Perl Foundation Grants]]
On Jul 7, 2006, at 8:13 AM, Shlomi Fish wrote: This kind of attitude was also said by another responder to this mailing list. It's sort of a "small headed" (see http://www.joelonsoftware.com/items/2004/12/06.html ) "I just want to write code and am not interested in any legal details" attitude. May I suggest a compromise? As the author of the MIT-licensed code in Test::Run, Shlomi has the option of releasing the code under any license he prefers. Shlomi can releases *two* versions of Test::Run with every update -- a mixed license version and an Artistic/GPL version. With that solution, Shlomi himself shoulders the burden of resolving license compatibility and tracking which line of code is under which license. I do believe that the quest for license simplicity in the Perl core is not "small headed" or rooted in ignorance, but is instead inspired. While many developers or TPF itself could easily delve deep enough to decide whether MIT/BSD licensed code in the core is a threat, I think that would be a wasted effort. The increased complexity of licensing (whether real or perceived) could easily turn off third parties with less dedication to Perl, thereby decreasing the attractiveness of the language. After all, software engineering is largely about reducing the exposed complexity of a project. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Time for a Revolution
On Jul 14, 2006, at 8:03 PM, chromatic wrote: On Friday 14 July 2006 17:59, Andrew Savige wrote: I thought Chromatic might be the name of chromatic's father or older brother. No, that's Mixolydian and Ionian, respectively. -- c (Yes, of course my mother is Dorian. What were you thinking?) Whoa, this is becoming an unexpectedly educational thread... Chris -- Chris Dolan, Software Developer, http://www.chrisdolan.net/ Public key: http://www.chrisdolan.net/public.key vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Dependency trees
On Jul 20, 2006, at 10:24 PM, Andy Lester wrote: Is there anything out there that will generate a tree of dependencies, probably based on META.yml? I figure I can pass in Mason, Test::WWW::Mechanize and Catalyst and get back a list of dependencies that those require. It would be the entire tree, so like so: Test::WWW::Mechanize Test::Builder WWW::Mechanize LWP::UserAgent HTTP::Response HTML::Form HTML::Tree Blah::Blah Test::LongString Test::Builder Blah::Blah If it doesn't exist, I'll write it. I just don't want to reinvent the wheel. Others have mentioned good candidates, but I thought I'd mention my own... I've done something not-exactly-like-that in Module::License::Report: % license-report -a Module::License::Report perl CPANPLUS unrestricted File-Slurp perl Module-License-Report perl YAML % license-report -a Test::WWW::Mechanize Can't find a license for Test::LongString perl Carp-Assert perl Carp-Assert-More perl HTML-Parser perl HTML-Tagset perl HTTP-Server-Simple perl Test-Exception perl Test-WWW-Mechanize perl URI perl WWW-Mechanize perl libwww-perl Something like that could be adapted to your purpose, maybe. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Installing Tests
On Sep 11, 2006, at 8:02 AM, Ovid wrote: Last week I was at a testing conference with Acme and he came up with the idea of installing tests. He looked into hacking Module::Build and ExtUtils::MakeMaker. He also considered hacking CPAN.pm and CPANPLUS.pm. While I don't know if he plans to continue working on this idea, he said he didn't mind me posting his idea here for others to consider. Basically, installing tests would be good because then you can run your full test suite against *installed* modules. That would be nice because then you could install a module and rerun your tests for your entire installation and see what broke. I love this idea, but here are some issues that we spotted: 1. How does one install tests for modules already installed? 2. If you install a module with already failing tests, you need to track what the failures are so you can note different failures when you run the test suite in the future. 3. What's the best way to install them? Should a separate tool just for this be built? Anyone want to take a crack at this? I'm also going to post this to Perlmonks. Interesting. A setup like this would have solved a recent bug in Text-PDF-0.27 where installation failed silently due to a bogus pm_to_blib file. However, why install the tests? Why not just keep the latests tarballs for each installed module and periodically do the following for each of the tarballs: tar -xzvf Foo-1.00.tgz cd Foo-1.00 perl Makefile.PL make test cd .. rm -rf Foo-1.00 That seems significantly less fragile than creating a new infrastructure, and still exercises all of the non-Foo dependencies. The most significant drawback of that approach is that it doesn't exercise the installed copy of Foo itself. Perhaps that can be accomplished by simply deleting lib and blib in Foo-1.00 before running tests? Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: TAPx::Parser 0.21
On Sep 11, 2006, at 9:58 AM, Ovid wrote:
- Original Message
From: Michael G Schwern <[EMAIL PROTECTED]>
Ovid wrote:
- Corrected the grammar to allow for a plan of
"1..0" (infinite
stream).
1..0 is currently used as part of the "skip all" syntax.
1..0 # skip Because I said so
Perhaps an infinite stream is just "1.." ?
Ah, I misremembered it.
It would be nice to have the plan indicate that an infinite stream
is forthcoming. That would make it easier to write custom
harnesses for it.
Sorry if I'm jumping into this thread out of context. I hadn't seen
any discussion of infinite streams before now.
How is that infinite stream different from the no_plan case? Is it
truly infinite or just undetermined? That is, are you trying to code
for the following use case?
use Test::More test => 'Inf';
use LWP::Simple qw(get);
while (1) {
ok(get('http://www.example.com/'));
sleep 60;
}
or is it something else entirely?
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: post-YAPC::Europe CPANTS news
complicate analyses and letting authors test those analyses offline exactly as they would be performed online on cpants.perl.org. Thus, I finally get to an action item: CPANTS should encourage authors to run Module::CPANTS::Analyse offline before uploading to CPAN. I assert that if we can convince authors to perform more thorough tests of their packages at author-time, then the quality of CPAN will improve. And the more closely the metrics match our real quality goals, the bigger the quality delta we will achieve. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: post-YAPC::Europe CPANTS news
On Sep 12, 2006, at 6:54 PM, Adam Kennedy wrote: [...] That said, I've have come around a little on the subject of author tests. The one advantage they do have, is that their inclusion means that while I may check POD structure, a pod.t ensures that in the event of someone else taking over the module, these things will continue to be checked. But with that in mind, I still don't see much point in running them at install-time, so lately I've modified my pod.t test so that it's skip message is now "skipped: Author tests not required for installation" or the like, and the tests now only run when AUTOMATED_TESTING is on. So now the tests will still get run during CPAN Testers and the like, but regular installation will not be impacted. That's fine, but in my opinion, a slightly better solution for author tests is to include them in revision control (i.e. SVN) but to exclude them from the public distro via MANIFEST.SKIP. That's what we do for the Perl::Critic self-compliance tests, for example, which are time-consuming and not relevant to the functionality of Perl::Critic. However, your approach to author-test skipping works well, and is more appropriate for modules which may not have a publicly-exposed revision control system. *shrug* I could probably be convinced that your approach is better. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Installing Tests
On Sep 12, 2006, at 8:20 PM, Adam Kennedy wrote: Frankly, I think anything you might write to try and pick and choose what to install is going to have problems. Since we're wasting... ermm... using all this extra disk space for tests anyway, why not just do the lot. Take the entire distribution post-make and just tarball that up and save it somewhere. Then if needed, you can unroll it later, rerun the tests with including blib, and there you go. It's a little on the brute force side, but it should work. Adam K I agree with all this except the post-make idea. The "perl Makefile.PL/Build.PL" step often does runtime things, like check the version of Module::Build or insert boilerplate into Makefile. If your install tools are updated between initial install and later re- test, this could go haywire. Except for processor time, I assert that you're better off starting with a clean tarball as if it's a fresh install. The only reason I can think of for the post-make argument is to preserve choices made during Makefile.PL, like Module::Install optional prereqs. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
CPANTS quality brainstorming
The recent discussion(s) about quality goals and metrics got me to thinking about what CPANTS is trying to do vs. what it's actually doing. As a brainstorming exercise, I wrote down all of the software quality goals I could think of quickly. For each existing CPANTS metric, I decided which of those goals the test was trying to solve. Then I tossed in some other possible (or impossible?) metrics that have been mentioned on this list or which occurred to me as I was brainstorming. I posted all of my thoughts on the Perl-QA wiki here: http://perl-qa.yi.org/index.php/CPANTS_Quality_Goals I'd be thrilled if others would look at what I wrote and improve it or tear it apart. :-) I hope this isn't too much wheel re-invention... Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: post-YAPC::Europe CPANTS news
On Sep 12, 2006, at 9:44 PM, Adam Kennedy wrote: That's fine, but in my opinion, a slightly better solution for author tests is to include them in revision control (i.e. SVN) but to exclude them from the public distro via MANIFEST.SKIP. That's what we do for the Perl::Critic self-compliance tests, for example, which are time-consuming and not relevant to the functionality of Perl::Critic. However, your approach to author- test skipping works well, and is more appropriate for modules which may not have a publicly-exposed revision control system. *shrug* I could probably be convinced that your approach is better. For something highly visible like Perl::Critic that's fine, but what tipped it over the line for me was that for ordinary modules repositories are transient, and can't always be relied on. The only canonical and reliable source we have is what gets uploaded. [...] That's a very good point. I'm convinced. To restate what you've said, CPAN is a better long-term software repository than any other system for open source Perl programmers. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Module license determination (was Re: CPANTS quality brainstorming)
On Sep 13, 2006, at 8:06 AM, Gabor Szabo wrote: On 9/13/06, Thomas Klausner <[EMAIL PROTECTED]> wrote: Maybe it would be reasonable to also check for a POD-Heading named LICENSE, but that's definitly more error-prone. Maybe this metric should even check if there's a license in META.yml AND a pod heading namen LICENSE, so that humans and tools can figure the license out... It should also check if the license field in META.yml is one of the 'approved' licenses, I guess this would be the list of words that can be used in Module::Build. It should also check if the license in META.yml and in the docs match. the problem of course is that this is very difficult to do correctly. If you want a more general approach to extracting the license information, consider my Module::License::Report package. It uses a bunch of heuristics to search README, meta.yml, Build.PL, Makefile.PL, LICENSE/COPYING/COPYRIGHT, dslip, etc. It requires CPANPLUS, so it might not be directly suitable for CPANTS, but there is some stuff in there that could help. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: CPANTS quality brainstorming
On Sep 14, 2006, at 4:36 AM, David Cantrell wrote: Adam Kennedy wrote: Agreed, while the META.yml key is the simplest way, it should be possible to locate the most common patterns for licensing, such at the typical COPYRIGHT POD block that often contains "... is licensed ..." etc. Or licenced. Please remember to take account of such a common mis- spelling. Again, I plug: My Module::License::Report package looks for spelling errors like that. It does a weighted estimation of the license: meta.yml counts most, then DSLIP, then .pm POD, then .pod POD, and finally miscellaneous files like README, Build.PL, Makefile.PL. Here's an example run: % bin/license-report -a DateTime Can't find a license for Class::Singleton perl DateTime perl DateTime-Locale perl DateTime-TimeZone perl Params-Validate Of those five, DateTime-Locale and DateTime-TimeZone have META.yml license info and the others do not. Class::Singleton confuses my module by having an unusually stated license for Artistic only: COPYRIGHT Copyright (C) 1998 Canon Research Centre Europe Ltd. All Rights Reserved. This module is free software; you can redistribute it and/or modify it under the term of the Perl Artistic License. === I'm plugging module this because I think it's an important tool that needs some improvement. Before this post did any of you know that the popular DateTime module had a non-Perl-licensed dependency? I didn't. https://svn.clotho.com/clotho/Module-License-Report/ Improvements needed: * Check if it works with a local CPAN mirrror * Reduce reliance on CPANPLUS? * Better license heuristics (like detecting Artistic in the above example) * Test against more of CPAN * Better Build.PL/Makefile.PL parsing - PPI - Module::MakefilePL::Parse (but that uses eval) - fancier regexps - http://www.nntp.perl.org/group/perl.makemaker/ 2523 * Better user control over weighting (like, a --yml flag to only look at META.yml) * Think harder about weighting: - Certainty of heuristic should matter (META.yml is certain, regexps on POD prose is uncertain) - Partial success should matter (found a POD "COPYRIGHT" section, but couldn't parse it) - Locality to code should matter (POD more significant than DSLIP) and finally: * Advocacy to get popular modules to state their licenses more prominently * Advocacy to get modules with inconsistently stated licenses (e.g. DSLIP vs. POD) corrected Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: New TAP Grammar
A few minor corrections:
On Sep 14, 2006, at 3:47 PM, jerry gay wrote:
grammar TAP;
## Ovid's TAP grammar, translated, corrected, and rendered idiomatic
## NOTE: not yet extended to deal with 'Bail out!' etc.
token tap { | * }
token plan{ <'1..'> \d+ \n }
That \d+ is bad because it allows '000'. Instead:
token plan{ <'1..'> [ <'0'> | <[1..9]> \d* ] \n }
token lines { + }
token line{ | }
token test{
[ <' '> (<[1..9]> \d*) ]? ## assumes a single
space, not \h+
\h* ? \h* ? \n
}
token status { <'not '>? <'ok'> }
token description { <-[#\n]>+ }
The description can't start with a number if there is a test number:
token description { <-[#\n\d]> <-[#\n]>* }
token directive { <'# '> [:i todo | skip ] \N* }
token comment { <'#'> \N* \n }
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Terrible diagnostic failure
On Sep 18, 2006, at 11:55 AM, Ovid wrote:
I have a bit of a problem, I think. It could simply be a matter of
misunderstanding how things work, but I have the following bit of
code in TAPx::Parser::Source::Perl:
my $sym = gensym;
if ( open $sym, "$command 2>&1 |" ) {
return TAPx::Parser::Iterator->new($sym);
}
else {
$self->exit($? >> 8);
$self->error("Could not execute ($command): $!");
warn $self->error;
return;
}
I've gotten a report that the open command fails on Windows. Not a
surprise, now that I think about it. However, I don't know of any
portable way of forcing STDERR to STDOUT (and I don't have a
Windows box handy). This means that my 2000+ TAPx::Parser tests
are in trouble. If Test::Builder accepted an environment variable
which allowed me to override this, I might have a way out. So far
removing the 2>&1 seems to make my tests pass on a Linux box, but
that strikes me as bizarre as I thought STDERR wouldn't get read
that way. What the heck am I misunderstanding?
Cheers,
Ovid
Try IPC::Open3, it's in the Perl core.
http://search.cpan.org/perldoc?IPC::Open3
IPC::Run3 is supposed to be good on Windows, but I haven't tried it
enough.
http://search.cpan.org/perldoc?IPC::Run3
Finally, you can try duplicating the STDOUT filehandle and saving it
in STDERR. Does this work as expected in Windows?
open STDERR, '&STDOUT';
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: A short rant on the purpose of the CPAN install chain.
On Sep 19, 2006, at 7:00 AM, David Golden wrote: David Cantrell wrote: clearly. So my attempt to make my tests good will mostly consist of applying the same coding standards to the test suites as I do to the rest of the code. OK, Lazy Web, who's going to write Test::Test::Perl::Critic? :-) Yeesh, Perl::Critic tries to be self-compliant, but even we don't run Perl::Critic on it's own test suite, let alone on Test::Perl::Critic. In seriousness, though, that is a pretty good idea. There definitely are different standards for writing test code than writing live code. A set of policies optimized for judging style of test code would be very valuable. Like, for example, policies that enforce labeling tests if Test::More is in use. Any tips on what - other than comprehensiveness, clarity and maintainability - I should aim for specifically in test suites would be greatly appreciated. No claim to best practice here, but this some of what I aim for: I try to reduce the amount of copy/paste test code I use. Once is OK, but twice makes me start thinking about putting it into a module that I pull in with "use t::Common" or something. I try to separate my test data from my test code. Many of my test files are data structures of test inputs, outputs, and labels that get processed in a small loop. I try to have the test labels be self documenting -- I'll often throw in a pass() to help me see the flow of execution when run verbosely I try to have each test file test a single feature -- with minimal dependencies on other tests being successful (not always possible). I try to skip subsequent tests that depend on something being successful so the real point of failure is apparent. Those are very good points. A few more: * Keep the .t files on topic. If you are testing independent features, consider using a separate .t file * Name your .t files in a lexicographic order that tests the simple stuff (like syntax errors) first and the less important stuff (like Perl::Critic compliance) last, with the meat in between * Write readable test code, and comment the intentions of every .t file Consider that your tests are sometimes the best working example of usage of your code. * Use Devel::Cover, but don't assume that high coverage always means thorough testing * Every time you get a valid bug report, write a test that reproduces the bug. and of course * Read "Perl Testing: A Developer's Notebook" for more ideas http://www.oreilly.com/catalog/perltestingadn/ Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Testing for test labels
On Sep 24, 2006, at 12:41 PM, Christopher H. Laco wrote:
I've got a crap loads of tests in Handel at the moment; 1+
checkpoints, and enough files that I hit the dreaded "command line too
long" under win32 [now fixed]. That's not saying all the tests are
glorious and not repetitive. :-)
I've still got a ways to go before I'm happy without the dist
enough to
declare the 1.0 version done. Among things on my list are cleaning up
the current test code. A big offender is that not all of my tests have
labels.
I could've swore that sometime in the last month I saw some chatter
about a Test::TestDescriptions module, or a ::TestsHaveDescriptions
type
thing for Perl::Critic. Assuming I'm not just making that up, has
anyone
seen mention of such a thing?
If no such a beast exists thus far, what would be the easiest way
to get
such functionality? I was thinking that I could just run a subclass of
Test::More in my tests that dies whenever no label argument is
supplied.
That's not a terrible thing since I already have a Handel::Test module
that deals with SQLite setup and teardown in most tests.
Done. I created Perl::Critic::Policy::Testing::RequireTestLabels and
added it to the Perl::Critic SVN at
http://perlcritic.tigris.org/svn/perlcritic/trunk/Perl-Critic
(username: "guest", password: "")
It was pretty simple to write. Below are the important bits of the
code. I encourage others to write Test::More-specific policies for
Perl::Critic!
Chris
==
my %label_arg_pos = (
ok => 1,
is => 2,
isnt => 2,
like => 2,
unlike => 2,
cmp_ok => 3,
is_deeply => 2,
pass => 0,
fail => 0,
);
sub default_severity { return $SEVERITY_MEDIUM }
sub applies_to { return 'PPI::Token::Word' }
sub violates {
my ($self, $elem, $doc) = @_;
my $arg_index = $label_arg_pos{$elem};
return if !defined $arg_index; # this is the fastest
conditional, so do it first
return if !is_function_call($elem);
return if !_has_test_more($doc);
# Does the function call have enough arguments?
my @args = parse_arg_list($elem);
return if ( @args > $arg_index );
return $self->violation( $desc, $expl, $elem );
}
sub _has_test_more {
my ( $doc ) = @_;
my $includes = $doc->find('PPI::Statement::Include');
return if !$includes;
return any { $_->module() eq 'Test::More' } @{ $includes };
}
==
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Testing for test labels
On Sep 24, 2006, at 5:42 PM, Christopher H. Laco wrote: Ok, I'll play your game. :-) http://perlcritic.tigris.org/svn/perlcritic/trunk/Perl-Critic/lib/ Perl/Critic/Policy/Testing/ [Assuming I'm not silly] Empty! Rev. 667 -=Chris D'oh! SVN commits work better if you "svn add" first... Fixed, should work now. :-) Chris -- Chris Dolan, Software Developer, http://www.chrisdolan.net/ Public key: http://www.chrisdolan.net/public.key vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Testing for test labels
On Sep 25, 2006, at 8:16 PM, Christopher H. Laco wrote:
Now, for part 2. :-)
What's the sanest way to test t/*.t files without also testing
everything in t/lib ?
If I'm already using all_critic_ok(), I can't just call critic_ok() on
all the t/*.t files because a plan has already been set.
Would it be possible to have all_critic_ok also take globs:
all_critic_ok('blib', 't/*.t')
-=Chris
Here's what I would do:
use Test::More;
use Test::Perl::Critic qw(critic_ok);
use File::Slurp qw(read_dir);
use Perl::Critic::Utils qw(all_perl_files);
my @files = all_perl_files('blib'), grep {m/\.t\z/} read_dir('t');
plan tests => scalar @files;
for my $file (@files) {
critic_ok( $file, $file );
}
Of course, you could easily replace that use of File::Slurp with
opendir,readdir,closedir.
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Suggestions for cpantesters
On Sep 29, 2006, at 5:16 PM, Alexandr Ciornii wrote: 6. Add posibility to module developers (or anybody) to subscribe to FAIL reports. Alex, This ones already possible in a limited sense. I subscribe to http://testers.cpan.org/author/CLOTHO.rss to see all pass/fail announcements for my modules. Via that feed, I noticed this morning that your smoke machine has a problem with my FLV::Info module (now fixed, thanks for the report!) Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Testing for test labels
On Sep 27, 2006, at 5:53 PM, Christopher H. Laco wrote: Ha ha. I just realized that I totally screwed myself out of using this policy. I'm using a subclass of Test::More, not Test:More itself. See "Don't 'rm -fr /' when testing" for the topic of that. Would it be possible, or even desirable to flip some sort of config to make this test all t files, or tell this policy that my test class eq 'Test::More' in this instance? Good point. I just added this feature to Perl::Critic svn. Do like this in your perlcriticrc file: [TestingAndDebugging::RequireTestLabels] modules = My::Test::SubClass Some::Other::Module Test::More is always implicitly included. Are there any common Test::More subclasses that I should add to the default list? Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Proposal for author test envvar standard (was Re: Suggestions for cpantesters)
On Oct 3, 2006, at 11:13 AM, David Golden wrote:
Given what you use, perhaps qr/AUTHOR_TEST/ is a good idea.
That's cool. Then I could do C
in my .t files and just set that to 1 in my .cshrc for all time.
I've seen discussion of the right env var name has been batted around
here and on the Module::Build list a couple of times without
resolution or blessing. So, I hereby propose new .t boilerplate:
use Test::More;
if (!$ENV{AUTHOR_TEST} && !$ENV{AUTHOR_TEST_}) {
plan skip_all => 'Author test';
}
plan ...;
where is either the maintainer's PAUSE username or, for
large projects, some unique package/family identifier. For example,
my PAUSE ID of "CDOLAN" or a project ID like "PLAGGER" or
"PERL_CRITIC" or "DATETIME".
People who want to be really thorough can set the AUTHOR_TEST=1 and
run through all of the optional tests on all installed CPAN modules.
This should not be set on automated/smoke systems because an author
test is one which is only needs to succeed (or perhaps only *can*
succeed) on a developer's machine.
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Proposal for author test envvar standard (was Re: Suggestionsfor cpantesters)
On Oct 3, 2006, at 10:04 PM, Adam Kennedy wrote:
Christopher H. Laco wrote:
Chris Dolan wrote:
I've seen discussion of the right env var name has been batted
around
here and on the Module::Build list a couple of times without
resolution
or blessing. So, I hereby propose new .t boilerplate:
use Test::More;
if (!$ENV{AUTHOR_TEST} && !$ENV{AUTHOR_TEST_}) {
plan skip_all => 'Author test';
}
plan ...;
Just for the sake of mentioning other things...in terms of this
topic,
what is an "author" in AUTHOR_TEST_ ?
Is that the original author? The last person who uploaded the dist? A
random author from a group of "core developers"? And how woulod I
know
which ones to choose as the person using AUTHOR_TEST_?
Sometimes the author isn't the author, and authors change.
Sometimes, a
module's "author" varies in the eyes of the public depending on
who last
uploaded the dist.
Personally, I use TEST_PRIVATE instead to avoid any implied ties
like that.
But I'm silly like that. :-)
Perhaps, PERL_RELEASE_TESTING or something then...
We've been discussing the name of this envvar for almost a year now,
as best I can count. I'm really hoping we can settle on a name very
soon. Here are the points I think are important, gleaned from
people's recent comments:
* Should have two forms: a general one and a specific one (like
AUTHOR_TEST and AUTHOR_TEST_CDOLAN)
* Should start with PERL_
* Should contain TEST or TESTING
* Should not be too long, so it doesn't need to wrap in code
* Should clearly indicate that the test is not for end users
* Should indicate that it is valuable for any contributing developers
(i.e. I don't like PRIVATE -- too forbidding)
So, I revise my proposal to:
PERL_DEV_TEST
and
PERL_DEV_TEST_
where is any string likely not to collide with other developers
or projects. It would be used like this in a .t file:
use strict;
use Test::More;
if (!$ENV{PERL_DEV_TEST} && !$ENV{PERL_DEV_TEST_PERLCRITIC}) {
plan skip_all => 'Developer test';
}
plan ...
Is that good enough?
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Proposal for author test envvar standard (was Re: Suggestions for cpantesters)
On Oct 4, 2006, at 12:14 AM, A. Pagaltzis wrote: If it’s worth boilerplating, it’s worth abstracting. How about use Test::More::ForAuthor [ 'ARISTOTLE' ], plan => 42; which loads and imports Test::More, then does the exact equivalent of the boilerplate you showed? I disagree. It should be easy for a 3rd-party developer to glance at the .t file and figure out which envvar to use to turn on the author tests. I think your abstraction is too obscure. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Proposal for author test envvar standard (was Re: Suggestionsfor cpantesters)
On Oct 3, 2006, at 11:04 PM, Adam Kennedy wrote: I'd ideally like to keep this constrained to a single environment variable. These tests are all the same sort of thing, tests that you run before you release, but not otherwise. If so, there's the existing AUTOMATED_TESTING flag. So that would give us two options, one for general bulk testing, another for when releasing modules. I like the simplicity. OK, I can accept that. It sacrifices a small convenience for elegance. Then my one remaining itch is your use of the term "release". These tests are run frequently during development, not just at release time. That's why I've been calling them author tests or developer tests. Yes, they are often prereqs for release, but do we want to tie them that intimately to release? After all, some modules never get released... I'd be much happier with a term that points at the development or authoring process. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: AnnoCPAN Doc Patch Maker
On Oct 3, 2006, at 3:58 PM, brian d foy wrote: Even better would be something like Pod::Spellchecker. I've started the project several times but never had that much motivation to finish it. Things would be simple if it could just spell-check everything, but I want it to be able to skip verbatim blocks, things in C<>, and so on. It's not an especially tough problem, just not as fun for me as other not especially tough problems. Happily, this problem has already been solved! See Test::Spelling. Note that it requires an external checker like aspell. Here's what we use in Perl::Critic: http://perlcritic.tigris.org/source/browse/*checkout*/perlcritic/ trunk/Perl-Critic/t/97_spelling.t?content-type=text/plain Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Pod Spelling
On Oct 4, 2006, at 3:29 PM, Ovid wrote: --- Ivan Tubert-Brohman <[EMAIL PROTECTED]> wrote: Have you tried Pod::Spell and Test::Spelling? I think they already do that. I've tried Test::Spelling only to discover that there are so many exceptions which it doesn't recognize that it was very painful to try and include in a test suite. The list of stopwords for large systems just became very unwieldy. The idea behind the module is great, though. Cheers, Ovid That's an aspell/ispell problem though, and not Test::Spelling's fault, right? You would have that same problem with any spell- checking solution. In addition to my own modules, I've used it on Perl::Critic and PPI and (aside from Adam Kennedy's weird Aussie spelling!) and it worked great. That's about 29,000 lines of well-documented code -- I'm not sure if you consider that large or not. Perl::Critic currently has 29 stopwords not included in aspell-en. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: AnnoCPAN Doc Patch Maker
On Oct 4, 2006, at 3:16 PM, brian d foy wrote: Not really. :) The spell checking is done by external programs, and the Pod::Spell module outputs something that doesn't have all of the original docs in it. You can find out if there are spelling errors, but you don't get back something that fixes it, and it's not interactive. Gotcha. You want something like the interactive mode in ispell/ aspell/Emacs, but with smarts about what's prose and what's code. I cooked up a prototype POD filter for ispell. ispell isn't as cool as aspell, but it supports external filters. https://svn.clotho.com/public/ispell-perl/ Hopefully I haven't reinvented any wheels... Use it like the following cmdline. It prompts you for every misspelled word and writes the file back out with changes. ispell -F ispell-perl-filter program.pl Implementation details: ispell filters take in code and output the *exact same number of bytes* with text you don't want checked turned into whitespace. I used PPI to find the POD in a roundtrip-safe way. I had to hand-roll a POD parser, because Pod::Parser is lossy (at minimum, it drops "=cut"). A future version of this should try to use Pod::Parser like Pod::Spell does. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: New Kwalitee Metric?
On Oct 24, 2006, at 9:32 AM, David Golden wrote:
I think characterizing "the basics" as being based on Module::Starter
is a little too module-specific for starters. Do you want to also
make sure that there are files other than the boilerplate created by
all the other module skeleton modules? What if someone puts real
tests into a boilerplate file and doesn't rename it. And how do you
plan to address test.pl?
Moreover, I'm loath to encourage too many (more) of the "file
exists" tests.
$ touch t/foo.t
"Wow, my distribution has more Kwalitee, now".
The examples metric is the same:
$ touch examples/not_really_an_example
That scores the extra point. And so what?
I'd be much more interested in seeing effort put into things like
checking if detectable prerequisites are listed in the META.yml.
Regards,
David Golden
As I've said before, Kwalitee should focus on rewarding good authors
and catching honest mistakes, while ignoring cheaters. If someone
adds important tests to boilerplate.t, that's a bad practice worth
noting. If someone games the system and creates an empty foo.t, I'm
content to give them a Kwalitee point they don't deserve. Maybe
we'll just create a Kwalitee test that detects any zero-length files
and penalize that cheater later.
It seems easy and fairly harmless to do this:
sub has_substantive_tests {
return 1 if (-e 'test.pl');
return 0 if (! -d 't');
for my $file (File::Slurp::read_dir('t')) {
next if ($file !~ /\.t$/);
next if ($file =~ /^(?:\d+[_-]?)?load.t$/);
next if ($file =~ /^(?:\d+[_-]?)?boilerplate\.t$/);
next if ($file =~ /^(?:\d+[_-]?)?pod\.t$/);
next if ($file =~ /^(?:\d+[_-]?)?pod[_-]?coverage\.t$/);
# more boilerplate to be added later...
return 1;
}
return 0;
}
That said, I'm not a Module::CPANTS developer, so I'm not going to
dictate priorities. The META.yml prereqs check sounds great.
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: New Kwalitee Metric?
On Oct 24, 2006, at 9:32 AM, David Golden wrote:
I think characterizing "the basics" as being based on Module::Starter
is a little too module-specific for starters. Do you want to also
make sure that there are files other than the boilerplate created by
all the other module skeleton modules? What if someone puts real
tests into a boilerplate file and doesn't rename it. And how do you
plan to address test.pl?
Moreover, I'm loath to encourage too many (more) of the "file
exists" tests.
$ touch t/foo.t
"Wow, my distribution has more Kwalitee, now".
The examples metric is the same:
$ touch examples/not_really_an_example
That scores the extra point. And so what?
I'd be much more interested in seeing effort put into things like
checking if detectable prerequisites are listed in the META.yml.
Regards,
David Golden
As I've said before, Kwalitee should focus on rewarding good authors
and catching honest mistakes, while ignoring cheaters. If someone
adds important tests to boilerplate.t, that's a bad practice worth
noting. If someone games the system and creates an empty foo.t, I'm
content to give them a Kwalitee point they don't deserve. Maybe
we'll just create a Kwalitee test that detects any zero-length files
and penalize that cheater later.
It seems easy and fairly harmless to do this:
sub has_substantive_tests {
return 1 if (-e 'test.pl');
return 0 if (! -d 't');
for my $file (File::Slurp::read_dir('t')) {
next if ($file !~ /\.t$/);
next if ($file =~ /^(?:\d+[_-]?)?load.t$/);
next if ($file =~ /^(?:\d+[_-]?)?boilerplate\.t$/);
next if ($file =~ /^(?:\d+[_-]?)?pod\.t$/);
next if ($file =~ /^(?:\d+[_-]?)?pod[_-]?coverage\.t$/);
# more boilerplate to be added later...
return 1;
}
return 0;
}
That said, I'm not a Module::CPANTS developer, so I'm not going to
dictate priorities. The META.yml prereqs check sounds great.
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Sparse Test Output
On Oct 26, 2006, at 9:37 AM, Paul Beckingham wrote:
> and waiting for the test output/TAP to be parsed takes quite a
while.
> Just my $0.02 worth.
> -=Chris
To each his own, but my thoughts were not that it takes time to
parse, or that there is any unreasonable performance issue here -
just that it is so completely *unnecessary* to say "ok" lots of times.
It's only unnecessary if the test program doesn't crash! Compare
ease of debugging for these two cases:
1..10_000_000 sparse
*segfault*
vs.
1..10_000_000
ok 1 - use_ok Foo::Bar
ok 2 - input = 0
ok 3 - input = 1
[... snip ...]
ok 99 - input = 97
ok 100 - input = 98
ok 101 - input = 99
*segfault*
In the former case, perhaps all 10_000_000 tests passed and the test
program failed in END{}, or perhaps it died on the first test. You
just don't know without re-running the tests in a debugger, or
turning off sparse.
In the latter case, the harness can tell you how far along you were
before the tests failed. If the harness is smart enough to report
the last test result before the crash, you can even guess that "input
= 100" caused the crash.
In even scarier terms, would you prefer to see an RT or smoke report
about your module with the former or the latter detail?
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Test.pm does not return false on failed tests.
On Oct 30, 2006, at 11:34 AM, Michael G Schwern wrote: The solution is to get authors to stop using test.pl. That sounds like an important Kwalitee test to me! Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Test.pm does not return false on failed tests.
On Oct 30, 2006, at 11:58 AM, Jos Boumans wrote: You're right, Test.pm does not, never has and does not have to. Exiting non-zero on failure is a little "feature" I put in to make using Test::More without Test::Harness a little easier but I plan on removing it. Hmm, I hope this doesn't mean you plan on removing the 'non-zero exit status on failed make test'.. that's be a real PITA for automated installers... Jos, I believe Michael was saying that he wants to remove the exit code from Test::More, not from Test::Harness. If I understand correctly that change should not affect CPAN/CPANPLUS except in the case of solitary test.pl, which is broken anyway. Given today's enlightening discussion, I do believe the right course of action is to do nothing except to encourage authors to use t/*.t and not test.pl. Any other solution simply invites unwelcome breakage. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: recursive_test_files in Module::Build and in ExtUtils::MakeMaker
On Nov 2, 2006, at 11:01 AM, Gabor Szabo wrote: In Module::Build I can set recursive_test_files => 1, so I can organize my test files in subdirectories of t/ I am using create_makefile_pl=> 'traditional', to create a readme but the Makefile.PL does not have any mention to this recoursiveness. Is this a bug in the makefile creation or is this feature not (yet) supported by ExtUtils::MakeMaker. In either case what should I do so people using Makefile.PL on this module will also execute all the tests? It's not an EU::MM bug -- it's a new M::B feature. What should you do? You're not going to like this answer: Don't use recursive test directories. :-) Chris -- Chris Dolan, Software Developer, http://www.chrisdolan.net/ Public key: http://www.chrisdolan.net/public.key vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: CPANTS and META.yml
On Nov 2, 2006, at 11:01 PM, Thomas Klausner wrote: metayml_conforms_spec currently very much busts the CPANTS game. I'm checking if the files comply to META.yml spec 1.2. Most don't, because they seem to use 1.0 Should I switch to 1.0-checking? No. The CPANTS game is a tool for change. We might as well encourage people to change to the current spec. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: P::C or PPI bug?
On Nov 7, 2006, at 6:15 PM, Christopher H. Laco wrote: I just wanted to get some thoughts on this before I filed a bug report with either PPI or Perl::Critic: I'm going through and testing all of my *.t files for RequireTestLabels. I was humming right along when I ran into an oddity. The newest Perl::Critic 0.21 + PPI 1.118 complains that the tests don't have labels, yet they clearly do. After some tinkering, I stumbled across the culprits: This causes a RequireTestLabel failure: is_deeply([$storage->primary_columns], [qw/id otherpk/], 'return DBIC primary keys from schema'); other is_deeply tests pass, like the one before it: is($storage->_primary_columns, undef, 'no primary columns defined'); I looked through RT, and I don't see anything that really points to anything on this one in either dist. Ideas? This is fixed in the latest PPI developer release v1.199_xx. The array constructor stuff should be perfect. The hash constructor parsing still needs some work to further disambiguate it from anonymous blocks. (like map, grep, eval, sort, etc) It never hit RT because I just fixed it myself instead of reporting. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: P::C or PPI bug?
On Nov 8, 2006, at 7:35 AM, Christopher H. Laco wrote: I gave the latest PPI + Perl::Critic SVN a shot late last night. That problem is indeed fixed for me, but another on cropped up with UseStrict, even through I am, and it only happens in 2 of 30 files. As soon as I get ouf of my meetings this morning, I'll update on the local box and have another go and post the actual error. I recommend moving this discussion to either the Perl-Critic or PPI mail list (probably the former). http://perlcritic.tigris.org/servlets/ProjectMailingListList https://lists.sourceforge.net/lists/listinfo/parseperl-discuss Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: P::C or PPI bug?
On Nov 7, 2006, at 11:37 PM, Adam Kennedy wrote: I'm not sure you can reliably test that though, not to 100% anyway, given the problem of implicit params. is( foo $bar, $baz, $expected ); which means is( foo($bar, $baz), $expected ); Yes, his specific case is ok, but I think you need to be a bit cunning about how you check... Well, the former is just bad coding style. So if Perl::Critic or PPI misinterprets it that's the coder's fault, not the tool's fault! :-) One of the best lessons in Perl Best Practices: if your code is ambiguous without looking at the core Perl docs (or source code!), then it should be rewritten to be less ambiguous. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: Perl::Metrics::Simple 0.30
On Dec 14, 2006, at 11:20 PM, Matisse Enzer wrote:
On Dec 14, 2006, at 3:05 PM, Michael G Schwern wrote:
Matisse Enzer wrote:
sub complexity_of_six {
my $bar = shift;
my $total = 0;
my $type = ref $bar;
if ( ! $type ) {
$total = $bar;
}
elsif ( $type eq 'ARRAY' ) {
foreach my $baz ( @{$bar} ) {
$total += $baz;
}
}
else {
confess("Don't know how to handle refs to $type");
}
return $total;
}
I'm missing something, I only count 3 paths. if, elsif and else.
The confess() might count (die/throw exception) but its the only
path through the else. Same with the loop, its the only path
through the elsif.
I over-simplified my explanation. It's not exactly paths-through-
the-code.
The complexity "points" in that example are:
1 for the subroutine itself.
1 for the "if"
1 for the ! logic operator in the if condition
1 for the elsif
1 for the "ne" logic operator in the elsif condition
1 for the else
6 total
That can't be right. Negation does not contribute to complexity.
Instead, I believe it is the for loop and the exit points that are
increasing your count. Consider rewriting the for as ifs and gotos:
sub complexity_of_six {
my $bar = shift;
my $total = 0;
my $type = ref $bar;
if ( ! $type ) {
$total = $bar;
}
elsif ( $type eq 'ARRAY' ) {
my $_i = 0;
LOOP:
goto DONE if ($_i >= @{$bar});
my $baz = $bar->[$_i];
$total += $baz;
$_i++;
goto LOOP;
DONE:
}
else {
confess("Don't know how to handle refs to $type");
}
return $total;
}
Then the decision points are:
1) if ( ! $type )
2) if ( $type eq 'ARRAY')
3) if ($_i >= @{$bar})
4) else
and the end points are
1) confess("Don't know how to handle refs to $type");
2) return $total
So I actually count a complexity of 7 (num decision points + num
endpoints + 1) if I've understood the wiki definition correctly.
http://en.wikipedia.org/wiki/Cyclomatic_complexity
Chris
--
Chris Dolan, Software Developer, Clotho Advanced Media Inc.
608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Clotho Advanced Media, Inc. - Creators of MediaLandscape Software
(http://www.media-landscape.com/) and partners in the revolutionary
Croquet project (http://www.opencroquet.org/)
Re: Perl::Metrics::Simple 0.30
On Dec 15, 2006, at 10:22 PM, Matisse Enzer wrote:
On Dec 15, 2006, at 7:52 AM, Chris Dolan wrote:
That can't be right. Negation does not contribute to complexity.
I think it is fair to say, that to a human, negation *can* increase
complexity:
if ( $foo ) {
# do something
}
is a little bit easier to understand than:
if ( ! $foo ) {
# do something
}
Instead, I believe it is the for loop and the exit points that
are increasing your count.
It certainly is possible to create a better, more sophisticated
measure of cyclomatic complexity - and I think you are probably
right, or at least "more right" in the way you are suggesting
complexity be counted. Perl::Metrics::Simple simply gives one
"point" for each of the following:
qw( ! && || ||= &&= or and xor not ? <<= >>= );
qw( for foreach goto if else elsif last next unless until while );
See themeasure_complexity() method in
Perl::Metrics::Simple::Analysis::File
OK, I see. Perhaps I was distracted from your main point by mention
of cyclomatic complexity, which has a rather specific definition.
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Measuring Complexity (was Re: Perl::Metrics::Simple 0.30)
On Dec 16, 2006, at 12:07 PM, Matisse Enzer wrote: On Dec 15, 2006, at 10:13 PM, Chris Dolan wrote: OK, I see. Perhaps I was distracted from your main point by mention of cyclomatic complexity, which has a rather specific definition. Mea culpa. In the next release I will change the documentation for Perl::Metrics::Simple to make clear how exactly the complexity score is calculated, and that it is only an approximation of true cyclomatic complexity. Meanwhile, I am open to making the measurement more useful. For example, should if ($a eq $b) or if ( $a == $b ) be counted as more complex than: if ( $a ) *shrug* It's really up to you and your user base, as long as you are consistent. Certainly the former examples require more tests to get full coverage. Personally, I use Devel::Cover's concept of complexity implicitly. Chris -- Chris Dolan, Software Developer, http://www.chrisdolan.net/ Public key: http://www.chrisdolan.net/public.key vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Measuring Complexity (was Re: Perl::Metrics::Simple 0.30)
On Dec 19, 2006, at 2:43 PM, Matisse Enzer wrote: I agree that ( $a == $b ) is not an extra branch, but, it is harder for a human to understand than ( $a ) When did this thread change from code complexity to human understandability? Those are rather different topics. ( $a ) of course translates to ( defined $a && $a ne '' && $a ne '0' ) which is likely harder to understand than ( $a == $b ). Undoubtably, the former leads to more subtle bugs than the latter. I think that for true Cyclomatic Complexity ($a==$b) is the same as ($a) but there is some other thing going on with ($a == $b) that perhaps, should be considered some other kind of complexity. You're likely biting off more than you can chew. I recommend that you take a lesson from Kwalitee and focus on implementing easily- defined metrics first. Chris -- Chris Dolan, Software Developer, http://www.chrisdolan.net/ Public key: http://www.chrisdolan.net/public.key vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Uncoverable syntax (was Re: testing module loading output and testing under the debugger)
On Dec 20, 2006, at 11:06 AM, Michael G Schwern wrote:
Well, maybe not exactly - I can probably make good arguments for
inline
annotation.
Allow me. :)
The current scheme won't track changes to the source file well.
Using file + md5 rather than file + line helps some, but problems
remain. Not dealing well with code change is a much worse problem
than having to markup your source. You markup once. You change
code a lot. O(1) vs O(n).
* What happens when I trivially alter an uncoverable line? Add a
comment, change the spacing,
move it to a different indentation level?
* What happens when I have two duplicate lines in the same file,
one I want to cover one I don't?
* What happens when I rename a file with uncoverable elements?
* What happens when I move code from one file to another?
And the usual drawbacks of not being inline.
* Having to drag around a separate file just for coverage.
* Having to shell out to a separate utility and describe to it
which bit not to cover.
Additionally, the use of an md5 sum to identify the line means I
cannot easily know which lines are marked as uncoverable even with
the uncover file open in front of me.
I predict a lot of churn of the form:
1) Edit the code
2) Run the coverage
3) Read warnings about uncover lines no longer existing in the
source file
(maybe, they've just been altered, maybe they've been deleted.
Who knows?)
4) Carefully scanning my coverage results to see what was marked as
uncovered has lost
that marker because of a trivial change to the line.
5) Updating the uncover file.
This is no fun.
If I might make a suggestion for an inline syntax. This is
statement, subroutine, branch and condition respectively.
statement; # uncoverable "Optional Reason"
sub foo { # uncoverable "Optional Reason"
...
}
if( condition ) { # uncoverable "Optional Reason"
if( this || that ||
unreachable# uncoverable "Optional Reason"
)
{
...
}
The condition coverage marker does require some reformatting of the
code. That probably won't be a big deal given how infrequently it
should be used. However, maybe something a bit more explicit could
be used.
if( this || that || unreachable ) { # uncoverable condition/3
"Optional Reason"
...
}
That states that the 3rd part of the condition on that line is
uncoverable.
IF you want a powerful, inline markup syntax for Devel::Cover (and
that's a BIG if), you might want to look at what we did for
Perl::Critic. We have scoped, blocked, and line-oriented pragma-like
comments that selectively disable Perl::Critic. An optional
parenthesized substring turns off just matching policies.
Single line style:
eval $str; ## no critic(StringyEval)
On/off-style:
## no critic(StringyEval)
eval $str;
## use critic
Scoped style:
sub {
## no critic(StringyEval)
eval $str;
}
eval $str; # triggers a violation...
The scoped style is possible because we are using PPI to parse the
code into a DOM.
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Test::More and Fatal
On Jan 24, 2007, at 9:42 AM, Nik Clayton wrote:
Has anyone ever used Test::More and Fatal together?
I have a test script, where each test builds upon the work of the
previous step (it's part of the Subversion Perl bindings test
suite, and it checks out files, makes changes to them, commits
them, and so on).
If any of these tests fails I need to abort the whole .t file (but
not necessarily the entire test run, so BAIL_OUT isn't appropriate).
I thought something like:
use Test::More;
use Fatal qw(ok is isa_ok open print close ...);
should do the trick. Anyone used this approach before, or done
anything similar?
N
I don't like that very much. The implicit die() on test functions
will probably confuse subsequent readers of the code. Furthermore, I
cannot believe that ALL of your tests are critical. It's better,
IMHO, to explicitly mention the critical tests and abort explicitly
on failure. For example, I do the following (in your case, change
BAIL_OUT to die):
use Test::More tests => 348;
sub bail { BAIL_OUT('Failed a critical test'); }
my $mech = Test::WWW::Mechanize->new();
$mech->get_ok('http://localhost/', 'get home page') || bail;
$mech->content_contains('Home');
...
In this example, failing to fetch HTML is clearly bad news for all
subsequent tests and the C< || bail> is good self-documentation.
Other tests are less critical, so should not fail.
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Test::More and Fatal
On Jan 24, 2007, at 1:25 PM, Nik Clayton wrote:
Chris Dolan wrote:
I don't like that very much. The implicit die() on test functions
will probably confuse subsequent readers of the code.
# Cause the listed functions to die on error
use Fatal qw(...);
Furthermore, I cannot believe that ALL of your tests are critical.
150 or so out of 158. So the
... or die "redundant error message: $!";
gets quite old quite fast.
N
Wow, 150/158? That's a high fraction. I still think the following
is somewhat more readable, but you are clearly in a different space
from the testing I usually do, so perhaps MHO doesn't apply.
sub bail { die "redundant error message: $!"; }
then 150 instances of "... or bail;" One advantage of that approach
is that you can add cleanup code to bail() if needed, although you
could probably do that in an END{} block instead.
Chris
--
Chris Dolan, Software Developer, http://www.chrisdolan.net/
Public key: http://www.chrisdolan.net/public.key
vCard: http://www.chrisdolan.net/ChrisDolan.vcf
Re: Parallelizing TAPx::Harness?
On Feb 5, 2007, at 5:41 AM, Adam Kennedy wrote: One thing that strikes me as a problem is transitory resources used by tests. I know I have a number of modules that will create (deleting if they already exist) various files during their run (cleaning up at the end). Each test is still independent (they randomize just fine) but if you ran two at the same time they could wallop each other's data. Short of running each test in some sort of transactional file system (do such things exist? Are they common?) or cloning the build directory before each test, the only solution I can imagine is some sort of thing in the (Makefile|Build).PL that explicitly states that the tests are allowed to run in parallel, which can then be passed on to the testing process... A solution might be to use File::Temp::tempdir(CLEANUP => 1) in that case. You can build the whole tree of files per-test and wipe clean upon exit. It seems to me that writing temp files to the build dir should be discouraged. Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Parallelizing TAPx::Harness?
On Feb 5, 2007, at 5:41 AM, Adam Kennedy wrote: Short of running each test in some sort of transactional file system (do such things exist? Are they common?) or cloning the build directory before each test... Here's a terrible, non-portable idea: * create a new, empty temp directory * use UnionFS[1] or Aufs[2] to mount that temp dir read-write overlaid on your build folder as read-only. * chdir to that new filesystem (or chroot?) * run your tests (which copy-on-write through the unioned FS to the temp dir) * chdir back * unmount the unioned FS * delete the temp dir Rather than being transactional, it's instead a throw-away branch of your filesystem. Like a BEGIN where you never plan to COMMIT. Chris [1] http://en.wikipedia.org/wiki/UnionFS http://lwn.net/Articles/219827/ (subscribers only until this Thursday) [2] http://aufs.sourceforge.net/ -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Generating test data and testing it
On Feb 20, 2007, at 9:12 AM, Ovid wrote:
Here's a common enough problem that I assume someone else has solved
it.
[snip]
The idea is, in this scope, we'd convert the 'get_ok' and
'content_like' methods to *not* be tests for the current scope. This
allows us to use our tests as test fixtures rather than duplicate this
logic.
I think this is a horribly clumsy idea, but the above is a simplified
example and some test data could get hairy to setup and if we change
our Web pages, I'd hate to find all of the duplicated logic for
setting
up this test data and testing that it actually works (note that we
could be testing a remote server and not always be in a position to
access the database directly).
Thoughts?
Cheers,
Ovid
Clever and scary. The following is a much more straightforward
alternative, but heavier weight:
sub add_user : Tests(5) {
my $test = shift;
my $mech = $test->mech;
$mech->get( $add_user_page );
ok($mech->success, 'get the "Add User" page') if $testing;
...
}
That is, unwrap the Test::WWW::Mech stuff for this case. That's
duplicating some T::W::M code, but it won't make anybody's head
explode trying to read the test code a couple years down the road.
Chris
--
Chris Dolan, Equilibrious LLC, http://equilibrious.net/
Public key: http://chrisdolan.net/public.key
vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: a safer way to use no_plan?
On Mar 4, 2007, at 6:33 AM, Dominique Quatravaux wrote:
And what if you are running a variable number of tests depending on
stuff such as compile-time options, maintainer mode enabled or not, or
whatever?
It's really not that hard. It's OK to declare a plan at runtime as
long as you declare your plan before you run your first test. If
your test program dies before that plan is emitted, the harness will
notice that zero tests ran and will complain accordingly.
For example, in CAM::PDF I have a number of test PDF documents that I
use to exercise the code, but I only ship a subset of them to CPAN
(the rest are too big or are license-incompatible). So, I do roughly
like so:
use Test::More; # plan declared at runtime below
my @test_docs = grep { -f $_ } 't/test1.pdf', 't/test2.pdf', ...;
plan tests => 12 + @test_docs * 20;
... run basic tests ...
for my $pdf_file (@test_docs) {
... run per-doc tests ...
}
Chris
--
Chris Dolan, Equilibrious LLC, http://equilibrious.net/
Public key: http://chrisdolan.net/public.key
vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: What was I thinking?
On Mar 28, 2007, at 9:17 PM, Kirrily Robert wrote: In September 2001, apparently all aglow in the excitement of all the talk that was going on about testing and whatnot, I wrote a module called CPAN::Test::Reporter. (http://search.cpan.org/~skud/CPAN-Test-Reporter-0.02/) Looking at it, I have no idea what I was thinking or why I thought it was necessary at the time. The only hint is in the credits where I say that it's based on the cpantest script by Kurt Starsinic. Presumably I thought it would be useful to have a modular version of it. I'm pretty sure nobody uses it (CPANTS reports that it's not a dependency for any other module). It looks like Test::Reporter is a newer module that does what I was trying to do, and is more up to date. What I want to know is, should I just bin it? Or might it, in fact, be useful to someone somehow? Or is it just bad form to completely delete CPAN modules? I guess I could update it with a new version that says "Don't use me." K. Were I you, I'd probably just post an annocpan and/or cpanratings comment recommending an alternative. Uploading a new version would probably be OK. However, that would alter the last upload date to 2007. I often use the date of last upload as a clue when searching for CPAN modules. *Usually* the stuff updated in 2006/2007 is better than the stuff from 2001. Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Custom Test::More problem
On Mar 28, 2007, at 8:18 AM, Ovid wrote:
The second test dies but this is ignored. Change My::Test::More to
Test::More and the failure is correctly reported. What did I miss in
My::Test::More?
If you add a "print $SIG{__DIE__}" before the first "ok", you'll see
that with Test::More it's a coderef, but not with My::Test::More. My
wild guess is that something in Exporter::Heavy up in
My::Test::More's ISA chain is blowing away the __DIE__ handler after
it gets set in Test::Builder.
Chris
--
Chris Dolan, Equilibrious LLC, http://equilibrious.net/
Public key: http://chrisdolan.net/public.key
vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Passing parameters to test scripts
On Apr 4, 2007, at 1:41 PM, Eric Wilhelm wrote: What I was thinking of here was more along the lines of how to get Test::Class to play that game. It already does. I routinely do stuff like this with Test::Class: env TEST_METHOD='student_.*' ./Build testcover to test, say, the student portion of the student/instructor Catalyst web site I'm building. Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Test::Kwalitee
On May 8, 2007, at 8:50 AM, Nathan Haigh wrote: I hope this is the correct place for this question! I'm trying to use Test::Kwalitee with the following test script: -- test script start -- [...snip...] -- test script end -- However, it seems to die unexpectedly with the following output: -- output start - [...snip...] -- output end -- Any idea what this might be about? Cheers Nathan Take a look at this report: http://rt.cpan.org/Public/Bug/Display.html?id=24832 It seems that Test::Kwalitee hard-codes the names of the Kwalitee tests, but Module::CPANTS::Kwalitee has changed since then. So, Test::Kwalitee either needs a quick patch to catch up to now or an overhaul to auto-detect the kwalitee tests from M::C::K. Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: CPAN testers generates Makefile.PL without prerequisites
On May 20, 2007, at 10:55 AM, Matisse Enzer wrote: I uploaded a new version of Perl::Metrics::Simple (0.032) and noticed that 3 failing reports from CPAN testers, all because I didn't include a Makefile.PL and the generated one (from my Build.PL) didn't include the prerequisites. Here is one of the failure reports: http://www.nntp.perl.org/group/perl.cpan.testers/2007/05/ msg482176.html So - what's the "right" way to remedy this? - The author (that's me!) provides a Makefile.PL as well as Build.PL? - Fix something in the CPAN tester system? - Other? - Some combination of the above ? :-) -Matisse Personally, I've found that adding create_makefile_pl => 'traditional' to my Build.PL file is the best solution for pure-perl packages. The Makefile.PL that is generated has all of the Build.PL optional prereqs turned into mandatory ones, and any dynamic config you do in Build.PL becomes static. However, if you make good default choices in Build.PL that may just mean that the Makefile.PL-using end user installs more of CPAN than they really need, which is definitely a lesser evil than a broken build. Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: CPANTS reports in e-mail
On May 31, 2007, at 2:28 AM, Eric Wilhelm wrote:
And for the rest of us to stay current, maybe a low-volume mailing
list
solely for notifications of various new/changed perl/cpan services?
Posting to module-authors should be fine if it's low volume. No need
for yet another mailing list ("yaml" is already taken)
Chris
--
Chris Dolan, Equilibrious LLC, http://equilibrious.net/
Public key: http://chrisdolan.net/public.key
vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Pod::Critic?
On Jun 6, 2007, at 10:08 AM, Ian Malpass wrote: I've been documenting lately, so I'm a bit POD-focussed at the moment :) I was pondering creating Pod::Critic, as a documentation analogue of Perl::Critic. Clearly it's not so easy to give hard-and-fast rules about documentation, but I thought it might be useful as a framework for enforcing internal documentation standards and "house styles", even if it didn't ship with many policies itself. Policies I've thought of: * Has NAME * Has SYNOPSIS * Has copyright details * Has license details * Method docs have examples * No spelling errors (borrowing Pod::Spell) Other more vague/less useful ones, perhaps: * Module names are links * Method names are in C<> sequences Some or all of these may not be relevant for particular instances, and I suspect it'll be less common to use an out-of-the-box set of policies. Anyone got any thoughts? Ian As Nadim said, several of these are already implemented in Perl::Critic. It's fairly straightforward to write new policies for Perl::Critic, so I recommend you give that a try. The module name and method name ones would be particularly welcome. I'd be happy to review any policies you write. The Perl::Critic mailing lists: http://perlcritic.tigris.org/servlets/ProjectMailingListList (the dev one has the most traffic) Perl::Critic developer tutorial: http://search.cpan.org/dist/Perl-Critic/lib/Perl/Critic/DEVELOPER.pod Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: Code coverage awesomeness
Josh,
I'm not sure that I understand this message (probably because I'm not
a perl guts hacker), but I am intrigued by your enthusiasm. I
downloaded and read the Runops::Trace code -- it's always a little
surprising and interesting to me to see which parts of the Perl
internals are malleable and which are not.
Josh, can you explain to us in a little more depth what this means?
Are you showing that certain input values follow the same path
through the code? It looks like the full path through the code is
the key to your hash of runs. If I've understood that part
correctly, then I'm still having trouble understanding where you go
from there. How is that a measure of code coverage? Are you
planning to then compare those paths to the full tree of opcodes?
Chris
On Jun 16, 2007, at 11:42 PM, Joshua ben Jore wrote:
I took chromatic's Runops::Trace and hacked it up so I could do code
coverage. I can now tell which code path is triggered by which inputs.
Check this badness out. The punchline is the lists of equivalent
inputs. Roughly. Ops can return different things and I'm not paying
attention to the inputs - it's only the path traversed.
---
nextstate=8207260 pushmark=8205ea0 padsv=8205e60 entersub=8207240
nextstate=81326d0 aelemfast=8132600 const=81325d0 modulo=8143d28
const=8143d50 eq=8143a80 cond_expr=81b6d18 const=8174510
leavesub=81749b0 nextstate=8203f08 pushmark=8203ea0 gv=8201128
rv2cv=8201078 refgen=8203ec0 gv=82010e8 rv2gv=8201108 sassign=8203ee0:
- 1
- 3
- 5
- 7
- 9
nextstate=8207260 pushmark=8205ea0 padsv=8205e60 entersub=8207240
nextstate=81326d0 aelemfast=8132600 const=81325d0 modulo=8143d28
const=8143d50 eq=8143a80 cond_expr=81b6d18 const=8174558
leavesub=81749b0 nextstate=8203f08 pushmark=8203ea0 gv=8201128
rv2cv=8201078 refgen=8203ec0 gv=82010e8 rv2gv=8201108 sassign=8203ee0:
- 0
- 2
- 4
- 6
- 8
- 10
use YAML 'Dump';
use Runops::Trace 0.02;
my %INPUTS;
sub is_even { $_[0] % 2 == 0 ? 'a' : 'b' }
for ( 0 .. 10 ) {
push @{$INPUTS{join ' ', trace( \&is_even, $_ )}}, $_;
}
print Dump( \ %INPUTS );
sub trace {
my $f = shift @_;
local @OPS = ();
*Op::display = \ &do_trace;
&$f;
*Op::display = \ &no_trace;
return @OPS;
}
sub do_trace { push @OPS, sprintf '%s=%0x', $_[1], $_[2] }
sub no_trace { }
I needed a new version of Runops-Trace. Get the original at
http://www.advogato.org/person/chromatic/diary.html?start=234.
diff -ru Runops-Trace-0.01/lib/Runops/Trace.pm
Runops-Trace-0.02/lib/Runops/Trace.pm
--- Runops-Trace-0.01/lib/Runops/Trace.pm 2005-10-05
19:01:31.0 -0700
+++ Runops-Trace-0.02/lib/Runops/Trace.pm 2007-06-16
21:09:18.0 -0700
@@ -7,14 +7,11 @@
use base 'DynaLoader';
-Runops::Trace->bootstrap( $VERSION );
+Runops::Trace->bootstrap($VERSION);
package Op;
-sub display
-{
- my ($class, $name) = @_;
- print "# Found op named $name\n";
+sub display {
}
1;
diff -ru Runops-Trace-0.01/MANIFEST Runops-Trace-0.02/MANIFEST
--- Runops-Trace-0.01/MANIFEST 2005-10-05 19:01:31.0 -0700
+++ Runops-Trace-0.02/MANIFEST 2007-06-16 20:46:07.0 -0700
@@ -1,6 +1,6 @@
Changes
lib/Runops/Trace.pm
-lib/Runops/Trace.xs
+Trace.xs
Makefile.PL
MANIFEST
t/01compile.t
diff -ru Runops-Trace-0.01/Trace.xs Runops-Trace-0.02/Trace.xs
--- Runops-Trace-0.01/Trace.xs 2005-10-05 19:01:31.0 -0700
+++ Runops-Trace-0.02/Trace.xs 2007-06-16 21:22:57.0 -0700
@@ -19,6 +19,7 @@
PUSHMARK(sp);
XPUSHs( sv_2mortal( newSVpv( "Op", 2 ) ) );
XPUSHs( sv_2mortal( newSVpv( PL_op_name[
old_op->op_type ], 0 ) ) );
+ XPUSHs( sv_2mortal( newSVuv( PTR2UV(old_op) ) ) );
PUTBACK;
call_method( "display", G_VOID | G_DISCARD );
Josh
Re: Code coverage awesomeness
On Jun 17, 2007, at 12:56 AM, Joshua ben Jore wrote: On 6/16/07, Chris Dolan <[EMAIL PROTECTED]> wrote: Josh, Josh, can you explain to us in a little more depth what this means? Are you showing that certain input values follow the same path through the code? Yes. It looks like the full path through the code is the key to your hash of runs. If I've understood that part correctly, then I'm still having trouble understanding where you go from there. How is that a measure of code coverage? Are you planning to then compare those paths to the full tree of opcodes? Ah, sorry. I have a problem at work where there's a pile of nasty code which deals in really gigantic piles of input per day. Much of the data is similar and I assume triggers identical paths through the code. If I wanted to get "full" coverage I could probably use a suitably gigantic pile of input because that would probably be sufficient to cover all or many of the potential code paths. It is impractical to write tests using those monstrously large piles of input. I want to filter out input which triggers redundant code paths and retain only those data that cause a unique behaviour. My example was an evenness detector. In one possible scenario I figure "several thousand" numbers are sufficient to cover all the possible code paths because "I don't understand this function." Maybe I really have hundreds of millions of numbers being tested and I really want to know if I get any additional behaviors by trying 4 if I've already tried 2. What I learn from using this is that I can get equivalent coverage from both inputs 2 and 4. I can now opt to write my tests using just the number 2 because I don't learn anything new by using any additional numbers like 4, 6, or 8. So I can test for which inputs will add to my code coverage and which will not. Josh Got it. Thanks very much for the explanation. Does anyone know if there's been research in a topic like this? That is, what size/ complexity can your code grow to be before small changes in inputs create wildly divergent code paths? It sounds (distantly) related to the code tracing that software like Coverity is doing, but on a macro scale. I wonder if path-finding algorithms (computer games, highway route finders, etc) have any relevance in this space? Hmm, on further thought that's probably wrong since (to put it mildly) code has much stronger opinions about which path to take than people or bots. Chris
Re: faking time() across processes for testing
On Jun 28, 2007, at 10:30 PM, Joshua ben Jore wrote: I had a terrible and powerful idea just now. Use Runops::Trace to provide op-level synchronization points. You could have N threads or processes all doing their individual ops (or statements if you prefer that granularity) in an order that you prefer. You'd have each process or thread do blocking reads off some socket or whatever and your central program with the master script would tell each one when it's ok to run an op. Heck... you could make it bi-directional and have the children send their current op up to the master script so it could even make decisions about it. This is such a sweet idea, I might actually want to try threads just to play with it. The module isn't threadsafe but that's only because I didn't try to make it so. Perhaps it isn't too difficult to do that for it. I think it would be better to implement this with ordinary thread locking instead of a core hack. Something like this would do the trick: http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ CyclicBarrier.html Perl thread locking: http://search.cpan.org/~nwclark/perl-5.8.8/lib/Thread/Semaphore.pm http://search.cpan.org/~nwclark/perl-5.8.8/pod/ perlthrtut.pod#Synchronization_and_control And just to tease Josh a little: http://www.everything2.com/index.pl?node_id=987739 Chris -- Chris Dolan, Equilibrious LLC, http://equilibrious.net/ Public key: http://chrisdolan.net/public.key vCard: http://chrisdolan.net/ChrisDolan.vcf
Re: [ANNOUNCE] Test::Builder/More/Simple 0.72
On Sep 22, 2007, at 9:06 AM, Salve J Nilsen wrote: - Set up some kind of syndication feed [RSS, Atom] [somewhere sensible] where one can read which distributions currently have a "pre-release" status. The feed might be called "NEED TESTING" or something like that. Make this feed visible on central Perl-related websites (Perlmonks, use.perl.org, etc.) and make it easy for Perl Monger groups to include this info on their websites. - Add some way for authors to state what the purpose of a module release is (e.g. "stable", "security update", "pre-release", "dev- release") and make this available in the distribution META.yml file. Well, one option might be something like: http://www.cpanforum.com/tags/name/helpwanted Gabor, would it be easy to add an Atom/RSS feed for a particular tag? Chris
Re: Perl 5 Wiki Improvement Drive: Recommended Modules For Testing
On Sep 24, 2007, at 4:15 AM, David Cantrell wrote: Andy Lester wrote: On Fri, Sep 21, 2007 at 11:23:21AM +0100, David Cantrell ([EMAIL PROTECTED]) wrote: It appears that I can't make any changes without registering for yet another bloody account. I already have too many. You have a limit? I have more than I can remember. Does that answer your question? I have hundreds more than I can remember, and most of them are random characters so they're nearly impossible to remember anyway. That's what encrypted keychains and/or encrypted disk images are for. Is this seriously your reason for not contributing to the wiki? My own reason is that I'm too busy with Perl::Critic and children that despise sleeping, not some biological shortcoming with a trivial technological workaround. Oh, wait, that gives me an idea... :-) Chris
Re: Overriding CORE::GLOBAL::print
On Oct 28, 2007, at 9:31 AM, A. Pagaltzis wrote: * Matisse Enzer <[EMAIL PROTECTED]> [2007-10-27 21:20]: I suppose we need to bite the bullet and just find and replace all the calls to 'print' with something else. Select a tied filehandle? Not efficient… but quick to add. That's a popular trick for CGI-like programs -- I used it in CGI::Compress::Gzip. Chris
Re: Overriding CORE::GLOBAL::print
On Oct 28, 2007, at 6:12 PM, Scott McWhirter wrote: You could use an IO::Handle based object wrapped around STDOUT/STDERR. It doesn't really matter whether it's a glob filehandle or an IO::Handle or a tied handle. The important part is the select() which lets you avoid changing each and every print statement. [Of course, you get nailed when the code explicitly says "print STDOUT ..."] Chris
