Re: version.pm and developer version numbers
It's crazy. But it's been that way since Perl 5.10.0 in 2007. So now we're stuck with it. The C$VERSION = eval $VERSION idiom avoids some issues with it by losing the underscore at runtime, but that might not be enough for things that parse your distribution tarball version number and try to do comparisons. (see http://www.dagolden.com/index.php/369/version-numbers-should-be-boring/) -- David On Fri, Jul 27, 2012 at 3:48 PM, Jeffrey Thalhammer j...@imaginative-software.com wrote: I just discovered that version.pm always treats version numbers with an underscore as less than the equivalent version number without the underscore. So 6.63_02 is less than 6.6302. Is it it just me, or does that seem crazy? Dealing with $VERSION numbers in Perl is hard enough as it is. This doesn't help. -Jeff
Re: Testing Output and Functionality
On Tue, Apr 24, 2012 at 12:35 AM, Jed Lund jandrewl...@gmail.com wrote:
I understand that this is a fairly basic question but I am stuck trying to
find the right path forward.
It's not entirely basic, because you're testing an array of three
outputs for each functionality test instead of just one.
I'd probably use Capture::Tiny:
use Capture::Tiny qw/capture/;
my ($out, $err, $result) = capture { your_code_here };
# then test $out, $err, $result however you want
If that's repetitive, stick it in a loop or write a function to
abstract testing the three results against your expectations.
-- David
Re: Perl QA Hackathon: What was accomplished?
On Fri, Apr 6, 2012 at 8:46 PM, James E Keenan jk...@verizon.net wrote: Could someone post a summary of what was done/not done at this year's QA hackathon in Paris (http://2012.qa-hackathon.org/qa2012)? I don't know if this page is on the main page, yet, but this is the running summary page: http://2012.qa-hackathon.org/qa2012/wiki?node=Results -- David
Re: What is the best code on the CPAN?
On Tue, Feb 7, 2012 at 9:29 PM, Jeffrey Thalhammer j...@imaginative-software.com wrote: So in your opinion, which distribution on the CPAN best demonstrates these qualities? Dist::Zilla Leaving aside the fact that I don't think your full wishlist actually exists in any single thing on CPAN, I would nevertheless point you to that as a good example of almost all the things on your list. -- David
Re: Fatal wide character warnings in tests
Hook %SIG? David On Jan 29, 2012 4:56 PM, Ovid publiustemp-perl...@yahoo.com wrote: How do I make Wide character in print warnings fatal in tests? This test passes; use Test::More; use strict; use warnings; use warnings FATAL = 'utf8'; use utf8::all; my $string = '日本国'; my $length = length($string); is $length, 3, $string should have $length characters; diag $string; done_testing; That's passing because the warnings pragma is lexically scoped and the actual warnings are emitted in Test::Builder guts (utf8::all will let the test pass because that package's code is now marked as utf8, but it doesn't fix Test::Builder's filehandles). I can make the warnings go away with this: my $output = Test::Builder-new-todo_output; binmode $output, ':encoding(UTF-8)'; $output = Test::Builder-new-failure_output; binmode $output, ':encoding(UTF-8)'; But I'd really like a clean way of just saying kill my code if I ever see 'Wide character in print' regardless of which package the error is emitted from. Cheers, Ovid -- Live and work overseas - http://overseas-exile.blogspot.com/ Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog - http://blogs.perl.org/users/ovid/ Twitter- http://twitter.com/OvidPerl/
Re: [QA2012] IRC Meeting 2011-01-05
On Thu, Jan 5, 2012 at 10:33 AM, Philippe Bruhat (BooK) philippe.bru...@free.fr wrote: - elbeho asked for advice regarding lunches (a break to eat outside vs. snacks in the dev room) - elbeho planned to book restaurants for the evenings - daxim's advice: avoid to leave the building even if it's not a decent lunch meal. proper dinner compensates I strongly concur. Lunch onsite is a nice for a short break that doesn't cut into hacking time too much. What I liked in Birmingham was that lunch was in the building, but in a separate room and for an hour, we all got to leave, have our stuff locked safely away and eat/chat with our fellows, but it was *only* an hour break. Lunch in the room is OK, too, but probably means less social interaction during lunch. That's still better than a long interruption for lunch. -- David
Re: Need suggestions for terminology
On Tue, Dec 6, 2011 at 7:32 AM, David Cantrell da...@cantrell.org.uk wrote: On Mon, Dec 05, 2011 at 06:21:13PM -0600, brian d foy wrote: In article 20111205154758.gh17...@bytemark.barnyard.co.uk, David Cantrell da...@cantrell.org.uk wrote: There's at least one other significant difference: CPAN has an up-to-date index. BackPAN doesn't. Well, CPAN's index is also BackPAN's index. That is, neither of them include all of the files in the repository. Various backpan mirrors seem to have different 02packages files. In general you can't rely on them to be the same as CPAN's. More broadly, different mirrors (BackPAN and otherwise) can have different 02packages files and you can't rely on them to be the same as PAUSE's, which is the one true index, of which all others are but shadows. (hat tip, Roger Zelazny, RIP) -- David
Re: Need suggestions for terminology
On Mon, Dec 5, 2011 at 10:47 AM, David Cantrell da...@cantrell.org.uk wrote: There's at least one other significant difference: CPAN has an up-to-date index. BackPAN doesn't. Further to that point, I think the idea of the index should be separated from the idea of the repository. The repository may contain an index, it may or may not be up to date -- which itself isn't a well defined concept given the lag between submission to PAUSE (and fast mirroring) and inclusion in the 02packages file. I plan to expand further on this idea of separation at the QA Hackathon in the spring (if I don't start working on it sooner). I'd like to get all CPAN clients able to use an index completely separate from a given repository, where the index could, for example, provide URLs to different repositories, such as internal or external, or CPAN and BackPAN, etc. Among other things, this would allow a project to freeze (and version control) a CPAN index and use it for repeatable deployment of a specific dependency chain. -- David
Re: Need suggestions for terminology
On Mon, Dec 5, 2011 at 6:37 PM, Michael Peters mpet...@plusthree.com wrote: Among other things, this would allow a project to freeze (and version control) a CPAN index and use it for repeatable deployment of a specific dependency chain. Isn't this what carton is trying to do? https://github.com/miyagawa/carton Yes, but carton (as I understand it) takes a current CPAN index and freezes the dependencies as tarballs into the project directory. (And I think does a local library install, too.) That works, but it's clunky. (Sorry, Miyagawa -- it's freaking brilliant, but still clunky). So every project winds up with copies of tarballs, libraries, etc. That seems to me to be hard to manage. I want to work it from the other end where you (or your company or team or whoever) manage the *index* and the net effect is the right set of tarballs get pulled in when needed. The two could actually work quite well together, since a better index makes carton better at managing DarkPAN resources or a locally modified index (no I want the *dev* version of Foo, because the current Foo breaks stuff and the old version doesn't have feature X we need) while still letting carton freeze specific tarballs down as needed. -- David
Re: Need suggestions for terminology
On Mon, Dec 5, 2011 at 6:27 PM, Ricardo Signes perl...@rjbs.manxome.org wrote: We have a tool to do this, at work. It isn't really in a state that I'd suggest others use it, but I would love to see something better than it become popular, as it has been *very* useful. So, I will be happy to help, at least by telling you what we learned. https://github.com/pobox/xpan I remember you telling me a bit about it and I'll check it out again. Frankly, I'm less concerned about constructing indices as that's a relatively easily iterated problem. The big hurdle I want to bridge is to get regular old cpan(plus|m) using a new form of index that maps packages to URLs (possibly URI::cpan::distfile) allowing any of these as targets for fulfilling a request for a package: * cpan:///distfile/AUTHOR/Foo-Bar-1.23.tar.gz (e.g. a default CPAN repo) * cpan://backpan.perl.org/AUTHOR/Foo-Bar-1.23.tar.gz (e.g. alternate repo) * http://dev.example.com/Foo-Bar-1.23.tar.gz (e.g. internal tarball repo) The point is that all we really need is package - tarball location map and if we teach CPAN clients to work from that, then we just need to play around with index CRUD tools for PAUSE and private use. -- David
Re: [QA2012] IRC Meeting 2011-12-01
On Fri, Dec 2, 2011 at 6:26 AM, Philippe Bruhat (BooK) philippe.bru...@free.fr wrote: - Laurent started to contact sponsors of past FPW, and also obtained more contact information for a potential sponsor Do you have a pitch for sponsors? E.g. what's in it for them, a summary of results of previous hackathons, what they get for various donation levels? If you do, I could approach a company in NYC that has indicated they're looking for ways to support the community. I think they're mostly looking for publicity to help with hiring, so a French conference doesn't help directly, but the hackathon gets blogged about a lot and if we ask people blogging to credit/thank the sponsors, maybe we can make the publicity bump more interesting to them. -- David
Re: Need suggestions for terminology
On Fri, Dec 2, 2011 at 8:21 PM, Jeffrey Thalhammer j...@imaginative-software.com wrote: Hi everyone- I need some suggestions for terminology to use in my code and documentation. I'm picky about names, so this is important to me (perhaps more than it should be). The context is Pinto, which is yet-another suite of libraries and tools for building a private CPAN-like repository. Here's what I have so far... I would encourage you to use existing names/conventions whenever possible. Some references to consider if you haven't. http://www.dagolden.com/index.php/308/packages-modules-and-distributions/ https://metacpan.org/release/URI-cpan https://metacpan.org/module/CPAN::DistnameInfo (which you've seen) Distribution: A Distribution is an abstract concept that defines relationships between packages. The minimal concrete implementation of a Distribution would be just a META.json (or equivalent) file. Distributions also have names and versions like Foo-Bar-1.2 No. At best a distribution is a collection of zero or more modules. (It could be all scripts and no modules.) META is not required. Distribution Archive: A Distribution Archive is the physical manifestation of a Distribution, and corresponds to an actual file on the local disk. For example, /home/jeff/Foo-Bar-1.2.tar.gz or C:\MyDocuments\Foo-Bar-1.2.tar.gz I don't think you gain much by distinguishing this from distribution. If you need to, I would consider abstract distribution for the concept of a collection of modules and distribution for the archive file as that is how it's commonly referred to elsewhere. Distribution Path: The Distribution Path is how an Archive is identified in a CPAN index. It is basically a URL fragment that looks like A/AU/AUTHOR/Foo-Bar-1.2.tar.gz. This is the term I'm having the most trouble with. CPAN::DistnameInfo calls this the prefix but I don't really like that either. A distribution file can only be uniquely identified on CPAN by AUTHOR/Foo-Bar-1.23.tar.gz. This is why I think that separating the concept of distribution from path is problematic. If you define distribution to be the tarball, the the name of the distribution is AUTHOR/Foo-Bar... etc. (The A/AU is unnecessary as it can be computed) It's what rjbs and I called a cpan::distfile URI. If you need, you might discuss distribution name as the bundle that may span multiple releasing authors, but with the huge caveat that two distribution files with the same name (sans author) have anything to do with each other. DAGOLDEN/Foo-1.23.tar.gz THALJEF/Foo-1.23.tar.gz These might be the same inside or they might be different. My Foo distribution could contain module ACME::Foo and yours might contain Acme::FOO. Those would be two totally legal, indexable distributions as far as PAUSE is concerned. Likewise distvname from DistnameInfo is a useful concept, but not sufficient to identify uniqueness. My general conclusion is what I said in my blog post. A distribution is an archive file 'AUTHOR/Tarball-version.suffix' that contains 0 or more modules. That is the only definition that doesn't get you into trouble with edge cases. Package: A package is just a package, in the usual way. That is, something declared with the package keyword. Module: I actually avoid using the term Module because I think it is often misused. Agreed. See my blog post about it. A Module is something that you can give to use or require and get a thing loaded. It may contain zero or more Packages (i.e. namespaces) Repository: A repository is a general term for any CPAN-like pile of files. This includes CPAN mirrors, as well as any DarkPAN or mini-cpans. A repository has a URL that identifies the entry point. For example: http://cpan.perl.org Fine, though you imply but don't state that a repository contains distributions in a particular directory layout below the base URL. -- David
Re: TB2::Mouse will be internal use only... with one hitch.
On Wed, Nov 30, 2011 at 11:07 PM, David E. Wheeler da...@justatheory.com wrote: Yes, but I would view this as a symptom of a failure of core Perl more than anything else. We really need a MOP and the attendant infrastructure and syntax for classes, methods, and roles in the language. The alternative is having to deal with issues like this. +1 Amen! -- David
Re: Capturing only STDERR with Capture::Tiny?
On Wed, Nov 30, 2011 at 5:24 AM, Gabor Szabo szab...@gmail.com wrote: Does a request for such feature in Capture::Tiny sound reasonable? https://metacpan.org/source/DAGOLDEN/Capture-Tiny-0.12/Changes -- David
Re: TB2::Mouse will be internal use only... with one hitch.
On Wed, Nov 30, 2011 at 12:39 AM, Michael G Schwern schw...@pobox.com wrote: There is a 3rd choice, which brings us full circle to where this discussion started. We write TB2::Object [1] to select load Mouse if its available, TB2::Mouse otherwise. That way TB2 classes are compatible with Mouse just in time. They also get a performance kick. The core gets the for internal use only marked TB2::Mouse as previously discussed. Without commenting on the rest of the plan, to which I want to give a little more thought, it occurs to me TB2 as released to CPAN could explicitly depend on Mouse, so that any TB2 installed (or upgraded) via CPAN client gets Mouse installed (or upgraded). Meanwhile, the TB2 in Perl core could rely only TB2::Mouse as a temporary shim until someone upgrades their TB2. That might be good or bad policy depending on how one wants to look at it as a precedent for core modules, but it does deal with this particular issue pragmatically. -- David
Re: Capturing only STDERR with Capture::Tiny?
On Tue, Nov 29, 2011 at 11:25 PM, Gabor Szabo szab...@gmail.com wrote: What is your suggestion to solve this? Does a request for such feature in Capture::Tiny sound reasonable? It's reasonable and has been requested before: https://rt.cpan.org/Public/Bug/Display.html?id=60515 I haven't found the tuits to dive back into the mess and implement it. To some extent, I'm trying to keep this Tiny and not implement every possible variation of capture that people might like. To another extent, I've not decided on an API that I like for doing the variations. (I cringed when I added 'capture_merged'). In the meantime, I think IO::CaptureOutput does what you want, albeit I don't think it necessarily does all things as well as Capture::Tiny (particularly anything dealing with layers) use IO::CaptureOutput qw/capture/; capture \subroutine, \undef, \$stderr; # pass through to STDOUT -- David
Re: TB2::Mouse will be internal use only... with one hitch.
On Wed, Nov 30, 2011 at 6:13 AM, Michael G Schwern schw...@pobox.com wrote: Unfortunately, that would be a circular dependency between Mouse and TB2 unless... Not a complete dependency, though, as one can function without the other. As long as the CPAN clients are smart enough not to follow the prereq chain endlessly, it might work. E.g. * install Mouse -- requires TB2 -- TB2 requires Mouse and client ignores repeat. TB2 passes test by falling back on TB::Mouse. Mouse tests find TB2 and pass. * install TB2 -- requires Mouse -- Mouse requires TB2 and client ignores repeat. Mouse tests fail (ugly!). TB2 fails to find Mouse, falls back and passes. That second case only happens if TB2 has *never* been installed on a system (or if it's too old to meet Mouse's needs). And possibly TB2's Makefile could be smart and only add the Mouse prereq if it finds a version of itself (or, really, of Test::More) sufficient to meet Mouse's prereq needs. (Bundling might still be easier.) Another idea might be for TB2 to use the new runtime/recomends prereq type in v2 CPAN Meta files. One of my Paris QA hackathon goals is to teach at least CPAN.pm (and maybe the other CPAN clients) to do something useful with the new prereq types, and useful might include something smart when a recommendation fails. (E.g. letting it happen without complaining). The definition of useful is one of the things I want to debate over meals there in the non-coding hours. :-) -- David
Re: [QA2012] IRC Meeting 2011-11-24
On Sun, Nov 27, 2011 at 4:29 AM, Philippe Bruhat (BooK) philippe.bru...@free.fr wrote: - the web site is live, DNS changes were fully propagated shortly after the meeting I've created a stub Attendees page on the wiki and added myself along with my proposed topics. http://2012.qa-hackathon.org/qa2012/wiki?node=Attendees -- David
Re: Relying more on Mouse
On Thu, Nov 24, 2011 at 4:23 PM, Michael G Schwern schw...@pobox.com wrote: I don't think cobbling together an OO system from bits and bobs, and then relying on it, makes more sense than using a fully baked one Of course, that pretty much describes Perl OO in a nutshell until Moose/Class::MOP came along. And it describes pretty much how every other core module does OO. (With the possible exception of CPANPLUS, which dragged Object::Accessor into core with it.) To be clear -- I'm not (yet) saying that TB2 should absolutely roll it's own OO, but I am concerned about hiding yet another copy of a perfectly decent CPAN module so that we can avoid saying that we're including another CPAN module in core. It's seductively easy to get the benefits *now* during the development of TB2, but there are downstream maintenance costs that we shouldn't discount lightly. Also one of the side benefits of shipping TB2::Mouse is now other Test modules can rely on it getting the benefit of a fully operational OO system without adding further dependencies. And potentially *other*, non-test modules could start doing so too. Is there a guarantee that TB2::Mouse will always be in core (short of a deprecation cycle)? Would it have to be available forever under use v5.18? I'm very leery of adding more core modules when the core policy around deprecation or lack thereof is in flux. [ Social hack: include TB2::Mouse but mark it deprecated from the start. :-) ] If I had my druthers, I'd like to see Stevan's core MOP work finished, then I'd like to see a powerful, minimalist object system on top of *that* built for inclusion in core, then I'd like to see core modules gradually migrating to that. And I'd like a pony, too. Maybe the related, fundamental question is whether TB2 itself needs to be in core -- or, if so, *when* it needs to be in core. Maybe core is just fine with Test::More as is and anyone needing more powerful testing frameworks can install TB2 from CPAN. (Ditto for any new test modules that are written that rely on TB2.) Eventually, when TB2 is well proven on CPAN and the core offers the kind of MOP that TB2 needs, then it can migrate in. -- David
Re: Relying more on Mouse
On Tue, Nov 22, 2011 at 2:02 PM, Eric Wilhelm enoba...@gmail.com wrote: Is there a way to remove some of the work Mouse is doing at startup? What is it doing? How much of Mouse is needed? Could Moo be used? (I ask without having read the details of the OO breakdown of TB2) -- David
Re: Relying more on Mouse
On Tue, Nov 22, 2011 at 10:49 PM, Michael G Schwern schw...@pobox.com wrote: The major Mo[ou]se features that Test::Builder2 uses heavily is roles and attributes. It also uses types, coercion, meta classes and method modifiers, but those could be removed if necessary. I've tried to keep it simple to avoid touching any actual or potential Mouse bugs. Attributes -- leaving aside types and coercion -- are you just talking about the sugar? And I wonder if Role::Tiny could be extended to do multiple roles. Or could you do a poor-man's version of roles using mix-ins? I understand the goal of decomposition and why Mo[ou]se rocks, but for a testing framework, maybe simpler is better. For example, Moose will detect method clashes with roles, right? (I assume Mouse does that.) Do you really need that? Or could you verify the design by hand rather than relying on a MOP to do the work for you? How much of it is fundamental to what you're trying to do and how much of it is just making life easier for you in implementing it? Granted, well tested Mouse might be less buggy than hand-rolled replacement, but a special-purpose class framework could possibly be simpler (thus faster and less buggy). [N.B. Not arguing not to use it -- trying to provoke discussion of preexisting assumptions.] -- David
Re: Relying more on Mouse
My bigger concern would be inclusion of Mouse in core as a dependency, since the direction of Perl seems to be to have fewer core modules, not more. I'd run that discussion by p5p/Ricardo before getting too tied to Mouse. Separately -- once the Perl core gets a MOP, maybe this gets easier/faster anyway. -- David On Sat, Nov 19, 2011 at 2:20 AM, Michael G Schwern schw...@pobox.com wrote: I did some profiling and easy optimizations which sped things up quite a bit, but it didn't translate into real world improvements. Turns out the real problem is startup time. use Test::More; 0.98 real 0m0.021s user 0m0.016s sys 0m0.004s 1.5 real 0m0.092s user 0m0.083s sys 0m0.008s That's going to be a tough one to cut down, simply because Test::Builder1.5 contains so many more individual files to load and attributes to set up. There's some things it's doing at compile time that can be hard coded (like setting up all the Result subclasses), but what shaves off a good chunk is using Mouse with XS. 1.5 with XS Mouse real 0m0.070s user 0m0.061s sys 0m0.007s So that's another point in its favor. -- Schwern What we learned was if you get confused, grab someone and swing them around a few times -- Life's lessons from square dancing
Re: Smoker setup to compare old vs new versions?
On Tue, Nov 15, 2011 at 8:28 AM, Michael G Schwern schw...@pobox.com wrote: I've never used the CPAN smoker software before and I'm hoping somebody has a recipe? Regression testing is less well developed. As far as I know, people have rolled their own. I have some poorly documented tools I used to use to regression smoke Module::Build: https://github.com/dagolden/cpan-testers-tools Steffen has another regression smoker (possibly with some parts adapted from mine) that he's been using to do Perl regression tests against CPAN: https://github.com/tsee/cpan_perl_branch_smoke The general approach is this: * Smoke configuration 1 and save reports to a directory * Smoke configuration 2 and save reports to a different directory * Compare directories and output HTML to browse differences Hope that helps point you in the right direction. -- David
Re: The end of 5.6 is nigh!
On Sat, Nov 12, 2011 at 6:50 PM, Michael G Schwern schw...@pobox.com wrote: By which I mean, the next major release of Test::More (aka Test::Builder1.5) will support 5.8.1 and up. ExtUtils::MakeMaker will probably go that way, too. This effectively cuts off most of CPAN from 5.8.0 and down. This will happen in the next few months. Thank Schwern! It's about time. I look forward to the jump to 5.10.1 in a couple years, so we can finally rely on configure_requires in the toolchain. (And it will be nice to stop bundling version.pm clones with toolchain distros, too.) -- David
Re: Perl QA Hackathon 2012
On Thu, Nov 3, 2011 at 8:25 PM, Philippe Bruhat (BooK) philippe.bru...@free.fr wrote: It seems we are picking up the gauntlet, and it will happen in Paris. :-) The current local team is very small: it's just myself and Laurent Boivin. Hooray! And thank you! We have two possible venues: ** Carrefour Numérique, Cité des Sciences et de l'Industrie ** Price: free ** La Cantine ** Price: we have a quote for 5700 EUR for the three days I vote for free, particularly since I would guess that hotels on the outskirts are less expensive as well. I don't mind the limited opening hours of Carrefour Numérique, as we can always reconvene at the hotel after dinner for evening hacking. We will have a weekly meeting on #perl-qa (irc.perl.org), every Thursday at 13:00, for 15 minutes. Time zone? --David
Re: Do we need subtests in TAP?
On Sun, Oct 30, 2011 at 3:23 PM, Michael G Schwern schw...@pobox.com wrote: On 2011.10.30 2:58 AM, Adrian Howard wrote: I prefer the current subtests system for a few reasons: * With the new system I would have to re-write TAP streams from other sources to match the numbering system of the current stream. This makes more work for folk who are pulling in TAP streams from other boxes/processes. If we do go this route you would, ideally, need to have a standard system for renumbering/counting streams. No, TAP::Harness would continue to parse subtest TAP. TAP from other tests could still be embedded by simple indentation. Michael, I haven't followed the T::B 2 work closely enough, so could I ask you to please step back and explain the benefits of T::B 1.5 that is worth stepping backwards in terms of capabilities? What I mean is that we have TAP::Harness now that processes subtest TAP and we have a T::B now that produces subtest TAP, so why stop? This quote from the TB 2 docs scares me a little: Test::Builder2 is very generic and doesn't do a lot of the work you've probably come to expect a test framework to do. This reduction of assumptions increases flexibility and ensures that TB2 can remain the core of Perl testing for another decade to come. That sounds an awful lot like second system syndrome. And -- hey, that's great if it works -- I mean no offense is saying that. But something that is *half* of a second system that loses a nice feature of the first system seems a suboptimal outcome. -- David
Re: Do we need subtests in TAP?
On Fri, Oct 28, 2011 at 3:23 AM, Ovid curtis_ovid_...@yahoo.com wrote:
Moving along, the *idea* of a nested TAP is so conceptually simple that if
the implementing code is struggling with it, perhaps it's a sign that there
are some design decisions which should be revisited? When I find conceptually
simple ideas hard to do, I find it a code smell. (note that I'm not saying
the actual design is bad. I haven't looked). I also find subtests so
incredibly convenient and opens up so many possibilities that I would hate to
lose them (and I use them a lot).
Without looking at the actual code, I would guess that the complexity
is implementing subtests while preserving the legacy procedural
interface that wraps calls to a global singleton.
Conceptually, this doesn't seem hard:
my $t = Test::SuperDooper-new;
$t-is(...);
$t-subtest( $label = sub {
my $t = shift;
$t-is(...);
);
$t-done_testing;
Tests would be recorded in an object and subtest subroutines get
passed a (new) test object.
But that's *not* the Test::More interface we all know and love so the
interface has to fake that with globals/local() that we can all
imagine (even without going to the code). So that design decision is
sort of forced due to the API.
I agree that given the existence of nested TAP now, I wouldn't want to
see it removed, so I'm not sure what complexity Schwern is looking to
eliminate -- whether it's existing complexity or potential new
complexity implementing new features.
-- David
Re: Do we need subtests in TAP?
On Tue, Oct 25, 2011 at 11:56 PM, Michael G Schwern schw...@pobox.com wrote: I keep looking at subtests and keeping thinking that if there wasn't a test count to manage, would we need subtests? Do we need all that complexity? If it's just about the test count, can it be managed a better way? I find several benefits: * Less visual clutter in non-verbose output * Indentation of verbose output * Test granularity I rarely care about test counts anymore -- the overhead of updating plan just isn't worth it to me. I understand wanting blocks of tests and the ability to make plans for just those blocks, but do we need a discrete test state for that? For example, Test::Class provides most of what subtests provide without special support. I would suggest looking at Test::Routine as well -- it has been a god-send to me for testing Metabase and uses subtests. It occurred to me because most other testing systems don't have subtests. They have test methods, but those aren't treated as a sub-state of the test. That might work under an assertion-type system, where tests within the method are assertions and some harness is catching exceptions within the test methods. The downside of assertions is that the test method dies on the first assertion and you get no information about whether subsequent tests would have passed or failed. In essence, a subtest is nothing more than a block of tests with a name and it's own plan. I wonder how many people are using subtests with a plan and how many are replying on the implied done_testIng feature. The special TAP formatting seems unnecessary. I guess that's the real question, do we need the special TAP formatting or do we just need named test blocks with their own plan? Because I really don't care about numbering, I don't really care if subtests have their own numbers or just continue the number sequence. I do like the visualization options of subtests and it would be weird if the numbering jumped when subtests were not verbose. -- David
Re: QA hackathon 2012
On Fri, Sep 30, 2011 at 4:44 AM, Philippe Bruhat (BooK) philippe.bru...@free.fr wrote: I thought I'd see people complain about having the hackathon start on a Friday, so I'll take the silence to mean doesn't really matter, so long as you have a good venue. :-) Yes. Friday is fine. -- David
Re: QA hackathon 2012
On Mon, Aug 22, 2011 at 7:07 PM, Lars Dɪᴇᴄᴋᴏᴡ 迪拉斯 da...@cpan.org wrote: To the usual suspects on this list, especially the ones who could not attend the last times: when do you have time around April? (Traditionally, the QA hackathon is from Saturday to Monday and avoids Easter.) My daughter turns 1 on March 22, but I would expect to be free the weekend that includes April 1. I would not be free the following weekend, which is Easter/Passover. Otherwise, April looks flexible. -- David
Re: [test-more] DBIx::Class mysterious fails (#146)
To clarify -- the tester is running Test::More 2.00_07. Whether that is advisable or not I won't take an opinion on, but it's a sign that 2.00_07 is breaking stuff on CPAN. I have several modules with similar issues and I've just started ignoring TB2 reports. -- David On Thu, Jul 21, 2011 at 3:25 PM, Ovid curtis_ovid_...@yahoo.com wrote: Hi there, I'm sorry to forward this to PerlQA, but with a baby at home, I've so little free time available that it's hard for me to really look into issues right now. I really would like your issue to be resolved and I'm hoping (beg, beg, beg) that someone on PerlQA might recognize this issue and be able to help. Cheers, Ovid -- Live and work overseas - http://overseas-exile.blogspot.com/ Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog - http://blogs.perl.org/users/ovid/ Twitter - http://twitter.com/OvidPerl/ From: ribasushi reply+i-1262479-ab3974188a293ecda596dd25eb86b133a62b1...@reply.github.com To: o...@cpan.org Sent: Thursday, 21 July 2011, 13:30 Subject: [test-more] DBIx::Class mysterious fails (#146) Granted I do some weird shit in these tests, but my TAP output/TB hooking is legit afaict. Please share your thoughts: http://www.cpantesters.org/cpan/report/f4703b28-b0d6-11e0-bffd-2a6e543d7ba4 -- Reply to this email directly or view it on GitHub: https://github.com/schwern/test-more/issues/146
Re: Suggestion regarding test suite design
On Wed, Jul 6, 2011 at 6:01 AM, Murugesan Kandasamy kmuruge...@ventechsolutions.com wrote: I have hundreds of similar scenarios and they are getting executed well and working as expected with test harness. But my doubt is I am having hundreds of .t files just with two lines of code i.e. loading the module, calling the subroutine Load_and_execute with the xml file as input. Is it not just a repetition/code duplication? But I need a way to test the whole suite with multiple scenarios like this. Is any other way possible. Maybe you need another level of abstraction. Create a file that contains all your cases (what you would put in the two-line .t file) and then have one .t file iterate through that file, calling each case. -- David
Re: stable and development releases on CPAN
On Sat, Jun 25, 2011 at 12:49 PM, chromatic chroma...@wgz.org wrote: For projects like Padre we would like to be able to upload versions and let the end user easily decide which version they would like to use. The best* approach I've seen so far is to add version numbers to names, as does perl5i--although you could add ::Stable or ::Development or ::Ancient::For::RHEL as suffixes easily enough. What would the interface look like in various contexts for selecting branches? I think the medium term answer is that we need to unbundle module/distribution indexes from CPAN mirrors. Then a project like Padre could publish versioned, custom indices and people could select them for an install: $ futurecpan --index http://... --install Padre Another alternative could be miyagawa's new carton project (see github) and you start publishing a list of things to be installed together rather than relying on a cpan client. My long term thinking is that CPAN clients as we know them are fundamentally limited and we'll see a round of innovation to find alternatives for different purposes, such as installing an application such as Padre or deploying web apps and so on. -- David
Re: RFC: Private CPAN In A Box
On Wed, May 25, 2011 at 1:19 AM, Jeffrey Thalhammer j...@imaginative-software.com wrote: I realize this doesn't solve the big problem of automatically building a particular CPAN mirror with a magical mix modules. But based on my own development patterns, I think this model could provide a workable framework for organically evolving application dependencies, while providing a stable and reproducible dependency stack at the same time. I hadn't quite taken it that far in my thinking, but I'm glad to have a use case to consider as I work. My mental model to date has been along these lines: * You're working off a particular version of perl (/opt/perl/5.XX/) which has no site_lib modules installed * Modules you're adding as dependencies get installed into your local::lib (possibly per application, if you need that separation) * When you're done with your application, you tar it up and inject it into a repository that can be accessed via a URL * You add your tarball to a local+BackPAN index (local supersedes BackPAN) * You run the dependency analysis tool against your tarball, using the perl as a base and using the local+BackPAN to resolve dependencies. * Tool spits out the ordered list of tarballs (probably as URLs) needed to install the application If you need to patch a module from CPAN, you treat it just like an app, publish locally and add it to the local index. To deploy, on some other machine, you run a deployment tool against the matching perl on that machine, giving it the ordered list of URLs. For instances where you might have App A and App B, the tool should be able to factor out a common set of dependencies, which could be deployed separately. Then with the right ordering of library paths, you could have App A use version X of a module, while App B uses version Y of a module and otherwise, they use a common set of modules. I hadn't considered the team development case much. My snap reaction is that you would regularly deploy to a shared development library path rather than install modules to a personal local::lib during development. As long as the local+BackPAN index can reverse your dependency (module name and version) to the right tarball (including locally patched ones), it all should just work. My though is to write a separate tool for this rather than try to make it work with either M::B or dzil. -- David
Re: RFC: Private CPAN In A Box
The problem is that there is no way for a single application to use multiple versions of a module. So it's easy to imagine that very specific module prereqs could be in conflict and require manual override. That's not the right answer. What we need is application level dependencies that are specific and non conflicting and then to make it easy to install into app specific lib directories. That's what im On May 23, 2011 5:22 PM, David E. Wheeler da...@kineticode.com wrote: On May 20, 2011, at 11:04 AM, Jeffrey Thalhammer wrote: I'm not attempting to build an all-encompasing dependency management system. I've tried to avoid thinking about it in terms of CPAN as we know it, since in theory, that is an implementation detail. But in practice, I'm will stipulate that a dependency is just a Perl module in some CPAN repository. Yeah, that's what's confusing me. To me, a dependency is a Perl module in some CPAN repository that the module I'm trying to install right now depends on. That is, it's listed in in the prereqs section of a META.json file. The trouble with CPANs is that they are a moving target -- modules are constantly updated, added, and removed[1]. This creates problems for developers that want to use the CPAN tool chain, but need to have a stable set of dependencies. At the same time, they need flexibility to evolve those dependencies in a systematic way. Well, the new version spec in CPAN Meta Spec 2.0 helps with this. You can specify very precise requirements. Thanks for bringing PGXN to my attention. The architecture is very similar to what I was thinking. I will dig deeper to see if we can leverage PGXN. [1] I think a lot of this trouble would go away if the CPAN tool chain simply permitted authors to express precisely which $VERSION of something they require. So I have to wonder if that is the right place to focus, rather than building something on top of the tool chain. You can. http://search.cpan.org/~dagolden/CPAN-Meta-2.110930/lib/CPAN/Meta/Spec.pm#Version_Ranges Best, David
Re: RFC: Private CPAN In A Box
On Monday, May 23, 2011, David Golden da...@autopragmatic.com wrote: The problem is that there is no way for a single application to use multiple versions of a module. So it's easy to imagine that very specific module prereqs could be in conflict and require manual override. That's not the right answer. What we need is application level dependencies that are specific and non conflicting and then to make it easy to install into app specific lib directories. That's what im The SEND button should be harder to press by accident... That's what I'm working on in June - trying to tie all the various lessons from other toolchain and meta tools into something that can be for Perl apps what cpan(p|m) is for Perl modules. I think people are trying to make cpan solve the application level problem and that's why custom CPAN mirror seems like the answer. David
Re: RFC: Private CPAN In A Box
On Mon, May 23, 2011 at 8:38 PM, Cosimo Streppone cos...@streppone.it wrote:
I say:
prereqs : {
...
runtime : {
requires : {
Net::Ping : = 2.35
}
},
...
},
Where current Net::Ping on CPAN is 2.36,
than this will install Net::Ping 2.35 as
prerequisite?
Or, if Net::Ping on the system is already 2.36
it will refuse to proceed?
I think the current CPAN clients will all refuse to proceed. This is
what I meant when I said that version ranges don't accomplish the goal
because the computer can't know what the right thing to do is.
-- David
Re: RFC: Private CPAN In A Box
On Tue, May 24, 2011 at 12:14 PM, David E. Wheeler da...@kineticode.com wrote: On May 24, 2011, at 3:04 AM, David Golden wrote: Or, if Net::Ping on the system is already 2.36 it will refuse to proceed? I think the current CPAN clients will all refuse to proceed. This is what I meant when I said that version ranges don't accomplish the goal because the computer can't know what the right thing to do is Ah, but sometime this summer the clients (and therefor the computer) will? I doubt it. I have plans for an application installer that will pre-compute dependencies and reliably set up per-application lib directories. Different applications could have different versions of a module, but a single application will only have one version available to it. This is not something that the CPAN clients are really designed to do (though they can be made to do the right things with INSTALLBASE and so on). A better specification and design for application dependencies and installation will avoid the need to create N custom CPAN views as a hack to make CPAN clients try to be application deployment tools. -- David
Re: RFC: Private CPAN In A Box
On Tue, May 24, 2011 at 12:25 PM, David E. Wheeler da...@kineticode.com wrote: Well, are you using the same version ranges spec for the META.json file? Seems to me that this might result in a stand-alone library that, given prereqs, could determine what should be installed. Then the existing clients could use it too, eh? That's the idea, though I've not worked out the details. But for example, I'd like to use the configuration output data (MYMETA.json) to determine three sets of modules (tarballs, really): - latest versions on CPAN - currently installed versions on developers machine - minimum versions on CPAN that satisfy the prereq specs It's possible that version ranges could conflict, in which case some or all of those might not be achievable. Ideally, the dependencies would be determined against a specific version of perl without additional libraries installed. Then, given one of those three ordered list of tarballs that satisfy all prereqs, it should be possibly to repeatably deploy an application with a known set of module versions, even as the latest on CPAN evolves. Of the three sets, users would have to decide which suit their specific needs/purpose, or could substitute versions, etc. That's my vision in a nutshell. I have some people in NY that I'm going to do some brainstorming with and then I'll try to get a proof of concept code out next month. -- David
Re: RFC: Private CPAN In A Box
On Tue, May 24, 2011 at 4:32 PM, Leon Timmermans faw...@gmail.com wrote: On Tue, May 24, 2011 at 10:17 PM, David Golden xda...@gmail.com wrote: Then, given one of those three ordered list of tarballs that satisfy all prereqs, it should be possibly to repeatably deploy an application with a known set of module versions, even as the latest on CPAN evolves. That could get a bit troublesome with the current best practice to delete older versions of module from CPAN. http://backpan.perl.org/ I don't remember when they started keeping the history -- I've seen stuff as far back as 1997. -- Davie
Re: RFC: Private CPAN In A Box
On Tue, May 24, 2011 at 5:27 PM, Jan Dubois j...@activestate.com wrote: AFAICT there is no good index for backpan though. I have to audit diffs to older module versions quite often, so I have my own tools, and found that even the links are not comprehensive; I have to crawl both backpan/modules/by-module and backpan/modules/by-authors/id to make sure I find *all* the different versions. I believe that brian foy created a pretty comprehensive one -- I think he may have actually investigated how module files changed from tarball to tarball. (E.g. Module $VERSION was the same in two tarballs, but the file MD5's are different). The hard problem is that PAUSE doesn't have any maintainer/co-maintainer history -- so given a historical DAGOLDEN/Foo-Bar-1.23.tar.gz and RJBS/Foo-Bar-1.23.tar.gz, there's no way to tell that one or the other was the official indexed version. You can look at who has maintainer rights *today* and assume from that who was authorized when the tarball was uploaded. I don't know (yet) how many collisions like that there are. I suspect it's a relatively small number that can be flagged and resolved with a bit of research and email. -- David
Re: Vague Testing
On Thu, Feb 17, 2011 at 3:52 AM, Ovid publiustemp-perl...@yahoo.com wrote: It's OK if I get back that data structure, but the 2 and 4 records are swapped or maybe the 5 isn't present. However, for any contained array reference its exact data can't change. However, if those came back in the order of 6,5,4,3,2,1, the test should fail (thus, I can't use bag tests). Do you have actual hard criteria? Or is it fuzzy/arbitrary? How do you know that one swap is OK, but a full reversal is not? Put differently, with the criteria you describe, I have a hard time seeing how this could actually be a meaningful test. Can you explain more about the problem domain? -- David
Re: Vague Testing
On Thu, Feb 17, 2011 at 9:09 AM, Ovid publiustemp-perl...@yahoo.com wrote:
For a *very* contrived use case, imagine that you're being introduced to your
daughter's boyfriend for the first time and you know his name is Alexander.
He might introduce himself as Alexander, Alex, Al, or even Xander and
you might not bat an eyelash. If he introduces himself as Sally or Bob,
it's times to start asking questions.
In my case, I have code which returns a list of items, but I'm pulling real
data (and it's very hard not to pull real data for this use case) and that
data will *usually* be in the order I expect, but subtle variations are
allowed and cannot be easily prevented. Unfortunately, I can't tell you more
than this.
Got it.
My impression is that if the real question is whether it's Alex and
not Sally, then I would try to avoid the ordering issue entirely if
possible and apply some domain-relevant sort to the data and then
iterate through, counting how many things are totally missing.
In your original example,
[
[ 1, 'North Beach', 'au', 'city' ],
[ 2, 'North Beach', 'us', 'city' ],
[ 3, 'North Beach', 'us', 'city' ],
[ 4, 'North Beach Hotel', 'us', 'hotel' ],
[ 5, 'North Beach', 'us', 'city' ],
[ 6, 'North Beach', 'us', 'city' ],
]
records 2, 3, 5 and 6 are identical. So when you say that 654321 is
not OK, is that because 1 ('au') should come before 2 ('us')? You
said that 2 and 4 could be reversed, so the ordering of 'hotel' and
'city' seems irrelevant.
So I think you've got to nail down what specifically about the order
is required, then sort in a way that preserves that important
dimension (country), but standardizes the rest (e.g. always putting
'hotel' after 'city'). Then you're just walking through it looking
for things that are mismatched or missing.
I realize your original example is highly simplified, but I hope my
suggestion about preserving the important ordering and sorting the
rest for comparison is clear enough and maybe even constructive. :-)
Regards,
David
Test::Harness fails tests on stock 5.6.2
Could someone please investigate this: http://www.cpantesters.org/cpan/report/334f5920-fbb8-11df-b9a1-c219e212a13c This prevents upgrading Test::Harness on a stock perl 5.6.2 system, which cascades into preventing CPAN.pm from upgrading likewise. I suspect it might be due to a bug in Getopt::Long as another 5.6.2 with upgraded Getopt::Long passes the tests with no problems. Thank you, David
Re: Depend on Imager w/ PNG feature?
On Sun, May 23, 2010 at 4:45 PM, Joshua ben Jore twi...@gmail.com wrote: Hi, My Runops-Movie depends on Imager with it's png feature. I'm not sure exactly how to accomplish this as a dependency. My module can depend on Imager but I have no way to communicate to the running CPAN harness that the Imager that's been delivered isn't sufficient. Right now the only abort message I know to send is exit(0) before Makefiie is created. I think I'd like to abort but I want to attempt to let the normal CPAN dependency resolution to work because I think it's likely that Imager will often have the right support and be built with png. Your suggetions? I might do it with a Task::Imager::PNG module that has Imager as a prereq and has tests for png capabilities. Test failures would get picked up by the harness. To be most helpful to users, you should probably have the tests fail with some sort of diagnostic message about how to install PNG support for Imager. -- David
Re: Toolchain issues
Quick thoughts: 2. ExtUtils::MakeMaker and Module::Build - Serious! HP-UX, and probably other OS's too, do not allow installation of shared objects when the object already exists and is in use. That means that all XS modules used by CPAN cannot be installed by CPAN. This is actually a problem with ExtUtils::Install, I think. The same problem exists on Windows, but EU::I uses a Win32 call to schedule the file for deletion on reboot before replacing it with a new one. I don't know what the answer is for HP-UX, but it's EU::I, not M::B or EU::MM -- which has the extra advantage of having only one place to fix it. 3. CPAN / CPANPLUS FUN! My HP-UX 10.20 only has 512 Mb memory :) When LWP and Archive::Tar are installed, large distributions - like DBD::SQLite - will cause an Out of memory! crash of cpan. When I use other fetch methods and build from source myself, there is no problem at all. I plan to get CPAN.pm using HTTP::Lite in 5.13, which will solve half the problem. David
Re: camels
On Mon, Jan 4, 2010 at 10:18 PM, Aristotle Pagaltzis pagalt...@gmx.de wrote: It’s unrecognisable at favicon size. The camel is distinctive down to a handful of pixels. And if you add a shell to it, so can a pearl be. In fact a pearl in a shell is what iX magazine in Germany has used as the masthead for their on-and-off Perl column for at least a decade. Or how about a string of pearls looped artistically in the shape of a P? My bikeshed is green, David
Re: qa.perl.org
For the record, I'll second Barbie's suggestion. David On Wed, Dec 30, 2009 at 4:16 AM, Barbie bar...@missbarbell.co.uk wrote: On Tue, Dec 29, 2009 at 10:10:48PM -0500, David Golden wrote: On Tue, Dec 29, 2009 at 4:54 PM, Leo Lapworth l...@cuckoo.org wrote: 2) Testing CPAN Shaun, would you mind looking at this? - Just needs a quick review/update The CPAN Testers stuff looks fine to me. The Daily Build section should really go to a new tab Testing Perl or Testing Perl Source or Testing Core Perl or something along those lines. Not sure it needs an extra tab, but perhaps renaming the current tab to TESTING CPAN PERL. There are CPAN Testers that submit test reports to CPAN Testers Reports for blead-perl as well as the Perl Smokers sending reports to the daily-build.reports archive, so although the two are different, and largely have different testers, there is still a slight overlap. Then you could have 2 H2 headings on the page, Testing CPAN and Testing Perl to differentiate the 2. Cheers, Barbie. -- Birmingham Perl Mongers http://birmingham.pm.org Memoirs Of A Roadie http://barbie.missbarbell.co.uk CPAN Testers Blog http://blog.cpantesters.org YAPC Conference Surveys http://yapc-surveys.org
Re: qa.perl.org
On Tue, Dec 29, 2009 at 4:54 PM, Leo Lapworth l...@cuckoo.org wrote: 2) Testing CPAN Shaun, would you mind looking at this? - Just needs a quick review/update The CPAN Testers stuff looks fine to me. The Daily Build section should really go to a new tab Testing Perl or Testing Perl Source or Testing Core Perl or something along those lines. David
Re: What's up with No targets specified and no makefile found?
You need to be asking on cpan-testers-discuss. Copying that list now. -- David On Mon, Dec 21, 2009 at 4:02 PM, Joshua ben Jore twi...@gmail.com wrote: The CPAN smokers regularly spit back UNKNOWN results where the text of the error is: Output from '/usr/bin/make': make: *** No targets specified and no makefile found. Stop. One example is http://www.nntp.perl.org/group/perl.cpan.testers/2009/07/msg4616138.html but there are many, many others. Does anyone know what's going on here? Josh
Re: Discourage use_ok?
(n Mon, Nov 9, 2009 at 5:11 AM, Ovid publiustemp-perl...@yahoo.com wrote: For the life of me, I can't really see any utility to use_ok() or require_ok(). Not only are both fragile and a source of strange action at a distance bugs, but the constructs they replace not only work correctly, but I don't see any problem with require_ok. I've found it useful as a cheap sanity check and don't see the action at a distance problems you imply. Or maybe I just prefer it over the ordinary death message of the module not loading. -- David
Re: standard for internal-only tests?
On Fri, Jul 31, 2009 at 2:32 PM, Jonathan Swartzswa...@pobox.com wrote: Wow, putting them in MANIFEST.skip - what a simple and great idea. :) I don't even need the environment variable in that case. Anyone who is running 'make test' in the git source will see the internal tests, as they should...anyone who has the published distribution won't see them. The downside is that anyone who works on your code from CPAN rather than git won't know you have them. -- David
Re: standard for internal-only tests?
A common standard is to put them in an xt/ directory and run them during development with 'prove'. Regards, David On Jul 31, 2009 2:12 PM, Jonathan Swartz swa...@pobox.com wrote: There are certain tests in my distribution that I don't want end users to run. I want to run them during development, and I also want anyone else contributing to the distribution to run them. These are typically related to static analysis of the code, e.g. perl critic, perl tidy and pod checking - it makes no sense to risk having these fail for end users. Is there a standard for signifying internal-only tests, and for make test to figure out when they should run? Jon
FYI: new 'patched' status on rt.cpan.org
I'd like to thank Ruslan (ruz) for adding a 'patched' status value on rt.cpan.org. I asked for the 'patched' status for tickets that I've fixed in the code repository, but have not yet released to CPAN. Tickets with 'patched' will appear on the active bugs list (e.g. from search.cpan.org) so that users can see the status and not make a duplicate report. -- David
Re: Making TODO Tests Fail
On Tue, Jul 14, 2009 at 5:43 AM, Ovid publiustemp-perl...@yahoo.com wrote: We have no diagnostics. We've never had diagnostics (the ad-hoc things going to STDERR don't count because they can't be synched or reliably parsed). Thus, I can't add diagnostics to the TODO tests until Schwern puts diagnostics in Test::Builder or accepts a patch. That doesn't look like it's going to happen any time soon, so telling me to add diagnostics to TODO tests doesn't help :( Thus, I'm trying to think of a way of solving my problem now, not at some hypothetical date in the future. I'm not sure how this is any different than finding failing tests, but if you can do that, here's one simple option: * Monkey patch Test::Builder::in_todo to return false. If Test::Aggregate does what I think it does, you can probably do the monkey patch in whatever file calls Test::Aggregate. (And make the monkey patch optional depending on some option or environment variable.) -- David
Re: Test module for tests in Perl module distro
On Sun, Jul 5, 2009 at 9:40 PM, Buddy Burdenbarefootco...@gmail.com wrote: Let's say I have some common functions that I want available to all my .t files. So I've created a module that all the .t files can include. But where do I put it? I don't want to put it in lib/ because I don't want it to get installed, right? But when I put it in t/, make test can't seem to find it. I tend to see three variations. (1) put it in t/lib/MyTest.pm and use lib 't/lib' at the start of your test files (2) put it in t/MyTest.pm and use lib 't' at the start of your test files (3) either of the above but use t::MyTest rather than changing @INC with use lib... I don't think any one is much better than the others. I tend to like the t::MyTest approach from a readability standpoint because it's clear within the *.t file that this is a test library and it's clear where to locate it. -- David
Re: Calling All Test:: Authors
On Tue, Jun 30, 2009 at 10:21 AM, Ovidpubliustemp-perl...@yahoo.com wrote: my $BUILDER = Test::Builder-new; I'm running visitcpan to generate a list of offenders now. Results posted soon. -- David
Re: Calling All Test:: Authors
On Tue, Jun 30, 2009 at 11:01 AM, David Goldenxda...@gmail.com wrote: On Tue, Jun 30, 2009 at 10:21 AM, Ovidpubliustemp-perl...@yahoo.com wrote: my $BUILDER = Test::Builder-new; I'm running visitcpan to generate a list of offenders now. Results posted soon. And here we go: http://echo.dagolden.com/~xdg/2009-06-30-test-builder-new.txt Some are certainly false positives, e.g. Test-Simple itself. -- David
Re: Fluent tests?
On Tue, Jun 30, 2009 at 1:17 PM, Ovidpubliustemp-perl...@yahoo.com wrote: Thoughts? Am I totally smoking crack here? If there's a clean way to shoehorn diagnostics on the Test::More-style interface, I guess that would be ok. Doesn't Test::Builder2 address this? I'd rather see more energy directed at getting that done. - David
Re: Fluent tests?
On Tue, Jun 30, 2009 at 2:15 PM, Ovidpubliustemp-perl...@yahoo.com wrote:
Also, I think playing around with more fluent interfaces is a good idea. If
my interface is great, why not? If it's bad, what would people *love* to see
in a test interface which allows them to naturally write tests?
Well, if you're doing interface design, one of the first things that
comes to mind is that the name of the test should come first, not
last.
I've often taken to writing tests like this:
is( $have, $want
label goes here
);
This means I can skim down the screen and all of the test labels line
up nicely. So I'm in favor of something along the lines of
test label goes here = is( $have, $want ) = diag $diag;
Which could work nicely for nesting, too
test a subtest label = nest {
# sub tests here
};
It would be test( $label, @objects ), where objects are executed in
some sensible order (e.g. diag objects only called if assertion object
returns false).
It could get quite involved, if one really wanted
test label = is( $have, $want ) = setup \setup = teardown
\teardown = diag \diag = note \note;
Then creating new tests is just a matter of creating assertion
classes. (Assertion classes could return diagnostic objects as well,
perhaps).
-- David
Re: [ANNOUNCE] Test::More/Builder 0.89_01 now with subtests
On Thu, Jun 25, 2009 at 5:49 AM, Ovidpubliustemp-perl...@yahoo.com wrote:
I understand where you're coming from, but different things should look
different. SKIP and TODO are relatively similar, but subtests are
significantly different from them.
The implementation is vastly different, but semantically, they just
provide context around a set of tests.
Though it does make me wonder how/whether subtests can be marked TODO.
(Whereas SKIP can be done through skip_all with plan). Does plan()
need a todo/todo_all option? Or would it be just nesting a TODO
block in the subtest?
In any event, I'm completely mystified why anyone has a problem with the
subtest $name, sub { ...} syntax. Honestly :)
I don't really have a problem with it, either, other than golf score.
Though I'd still possibly prefer something like this:
subtest Sanity check = sub {
plan 3; # or subplan()
pass for 1..3;
}
-- David
Re: [ANNOUNCE] Test::More/Builder 0.89_01 now with subtests
On Thu, Jun 25, 2009 at 12:08 AM, David E. Wheelerda...@kineticode.com wrote:
The how about:
subtest {
plan tests = 3,
for = 'my sanity';
pass; pass; pass;
}
As long as we're bike-shedding, a simplification:
subtest {
plan sanity check = 3;
pass for 1 .. 3;
}
Anything other than no_plan or skip_all is taken as if tests.
-- David
Re: Cpan Dist Layout Issue
On Fri, May 15, 2009 at 12:11 PM, Smylers smyl...@stripey.com wrote: That Perl has this QA thing which tells you which platforms a module will work on but that doesn't actually mean it'll install only adds to external perceptions of Perl being confusing and awkward. Of course CPAN Testers doesn't claim to check whether a module installs, only whether it passes tests. It doesn't even tell you whether there are good tests. It could be t/pass.t with a single pass(fooled you) test and it would still get a PASS grade on CPAN Testers. Automated testing doesn't remove to need for real people to find bugs and report them. -- David
Re: Cpan Dist Layout Issue
On Fri, May 15, 2009 at 1:29 PM, Smylers smyl...@stripey.com wrote: Most of the information on that page is likely to be bewildering or irrelevant to the user who's just stumbled upon the site, but it does say mention that it's a way to provide multi-platform testing for modules and provides valuable feedback for users. Yes, well, I agree with you there. Care to draft some replacement text and pitch it to Barbie? I believe the source is here: http://github.com/barbie/cpan-www-testers/tree/master Maybe we need some simple, web 2.0 interface like Adam did for Strawberry Perl and explain CPAN Testers in 140 characters or less. :-) Anyway, our sys-admin could easily end up deciding that Cpan Testers results are clearly wrong, and think less of Cpan Testers or Cpan or Perl or the Perl Community. Maybe in future when evaluating software he'll be more inclined to choose something written in another language, so he doesn't have to deal with the trials of Cpan again. So instead your sysadmin will go choose something in a language that doesn't have a code library repository of the breadth and depth of CPAN, one which doesn't bother to encourage a culture of testing, and doesn't bother to collect test results at all? Because it's simpler and less confusing than the 'trials' of CPAN? That doesn't say much for the quality of your sysadmin, does it? On the other hand, you're right -- people will often choose the lesser path because it doesn't require them to think as much. And that's fair enough. But possibly there are those in the Perl community who think this perception is unfortunate and are interested in doing something about it, making the Perl World a friendlier or more helpful place. Seen Alias's rants about Perl websites? Or the Iron Man blogging competition? I'm totally with you on the broad point, just not on the cheap shot at CPAN Testers. Do you have any idea how long it took to get the CPAN Testers websites handed over at all? (Years) Or how many flamewars there have been on how to make CPAN Testers less annoying? (At least a couple a year, though less since we stopped emailing authors directly.) Or how much work is being done to improve the quality and utility of the reports? Or to switch away from email as a transport mechanism? It's not that we don't know that parts of it suck, it's just that we're working on other things. Rant aside -- please do send patches if you think you can make it better. We could use all the help we can get. [*1] Though actually is there anything stopping somebody from checking whether modules actuall install and sending FAIL results to Cpan Testers for those that didn't? Surely they'd just display like other FAILs. Yes that'd be more involved than what current testers are doing, but I don't think it means it couldn't be done by somebody. Nothing is there to stop it, though I'd personally oppose it and push to drop those reports on the floor so it doesn't confuse matters. One of the reasons that we decided to have Build/make failures become UNKNOWN was to help authors distinguish them from test FAIL reports. I've supported expanding the grades in the past and would do so again if someone were to argue for an INSTALLFAIL grade or equilvalent for the case you describe. -- David
Re: masking installed versions when running tests
On Tue, May 5, 2009 at 7:44 PM, Jonathan Swartz swa...@pobox.com wrote: One solution, I guess, is to use a fresh install of Perl that contains just the requirements for CHI. But I'd need one of these for each CPAN distribution I'm working on! You only need one fresh perl. Set CPAN not to install build_requires. cpan o conf init /build_requires/ Then always test with that perl/CPAN and never install and each run will start clean. (This is, in fact, more or less what CPAN::Reporter::Smoker does) As a nice bit of sugar, CPAN recognizes the . argument as meaning the current directory, so you can do this: $ cleanperl -MCPAN -e 'test .' Hope that helps, David
Re: masking installed versions when running tests
On Wed, May 6, 2009 at 12:05 PM, David Cantrell da...@cantrell.org.uk wrote: That's true, but this isn't just about testing libs - it's any lib I might delete from the distribution. You want Devel::Hide. I think the original question was about catching a module deleted by mistake -- otherwise, Devel::Hide rocks. -- David (different one)
Re: test harness presentation layer plugins
On Tue, Mar 31, 2009 at 6:22 PM, Michael G Schwern schw...@pobox.com wrote: But it's really more useful for more complicated formatting. Something simple like making subtle whitespace differences visible should be a feature of the TAP consumer. Which is why it will be nice when the consumer isn't just getting text output on a filehandle. /me wants structured TAP capture and easy Harness plugins to change how things are presented. I hope I can finally get CPAN Testers 2.0 far along this year so that next year at a QA hackathon, I can spend more time with the TAP/Harness team. -- David
Re: Informal 'make test' on production poll
On Thu, Mar 26, 2009 at 2:59 PM, Ovid publiustemp-perl...@yahoo.com wrote: That's the main reason why our tests don't run on production right now. I would like, at the very least, to have a './Build sanity' target to ensure that guaranteed non-destructive tests are run, but the strange argument I'm facing is that production should be an exact copy of staging and thus tests on staging should be enough. The word should makes my trigger finger itch. So test the assertion that production is an exact copy of staging. E.g. md5sum of a manifest of files. If there's a concern about processor load, use a cheaper checksum and bump up if there is a collision. Or handle it via process -- who's on the hook if the assertion fails and make them sign something affirming it. Pass the buck. :-) -- David
Re: How do I respond to a CPAN Tester failure report?
On Tue, Mar 24, 2009 at 6:29 AM, Michael G Schwern schw...@pobox.com wrote: Like I said to Barbie, all I want to hear is something's in the works. Bring your persuasion stick and stop by whatever room the CPAN Testers junkies are in from time to time. Or bring carrots. :-) -- David
Re: How do I respond to a CPAN Tester failure report?
On Mon, Mar 23, 2009 at 3:56 PM, Michael G Schwern schw...@pobox.com wrote: I ask not for myself, I know the answers to this and they suck. I've bitched about this before and its not necessary to go into details again. I ask as Joe Newbie CPAN Author who just got his first pile of failure reports and has no idea what to do next. Michael -- I'm really tempted to create an auto-responder for any Perl-QA email that mentions CPAN Testers and point out *AGAIN* that there is a cpan-testers-disc...@perl.org mailing list where most of the CPAN Testers hang out as well as most if not all of the CPAN Testers module authors. (Yes, some of us monitor this group too.) (My abbreviated tangential rant: wouldn't it be great if someone would update lists.perl.org or hand over maintenance?) What there should be is simple instructions on how to get a message back to the tester. This should be a single link or email address. [1] This was discussed at length when we cut over from email direct from testers to authors to a periodic notification service. It's harder for authors to contact testers. The flip side is that authors don't feel spammed. The vocal minority of those who shouted for an end to spam were louder than those who shouted that they wanted to just be able to hit 'reply' and send to authors. And since the NNTP web archive hides email addresses, a kludgy workaround was developed. (N.B. I'm not dissing the tool, Barbie, just that we have to do it in the first place) In the meantime, for those who want to have easy replies to authors, see http://search.cpan.org/dist/App-CPAN-Testers-Remailer An immediate and relatively simple fix is to provide a single use mail address on the report. A hash of the reporter's name and some unique information about the report, perhaps something as simple as a UUID. The author mails that and its delivered to the appropriate things, probably the test reporter and maybe cpan-testers as well. Since switching from email to web-based submission is a major goal of the hackathon this weekend, and thus we'll be moving away from NNTP as an archive, we'll need to design a report view web page anyway, which could include a method to contact the author or otherwise reveal an email address in a spam-harvesting proof way. Can I put you down as volunteering to design something, get group consensus and then implement it? You can start by adding it to the roadmap and hackathon project pages here: * http://wiki.cpantesters.org/wiki/Roadmap * http://qa-hackathon.org/wiki/CPANTesters See you at the hackathon! :-) -- David
Please test Module::Build release candidate
(Apologies to those receiving this multiple times) Eric Wilhelm and I have put out Module::Build 0.31_04 as a release candidate for a stable 0.32 Module::Build. Initial CPAN Testers results look good. (So far, it's the first all PASS since 0.2804) However, before release (and before it gets pushed into blead), it would be great if it could be tested on more obscure operating systems and/or less common (older?) versions of perl than available from CPAN Testers. The current matrix of CPAN Testers coverage is here: http://bbbike.radzeit.de/~slaven/cpantestersmatrix.cgi?dist=Module-Build+0.31_04 If you have access to other platforms/perls, please give it a try and report back if you have problems. Module::Build 0.31_04 is available from CPAN as EWILHELM/Module-Build-0.31_04.tar.gz Thank you very much. -- David
Re: Testing scripts with expected STDOUT and STDERR in external files
On Sat, Feb 21, 2009 at 5:24 AM, Gabor Szabo szab...@gmail.com wrote:
I wonder if there are modules out there that already do this?
I could not find any that would fit my needs.
Test::Cmd? It's not built on Test::Builder but could be
adapted/wrapped. It uses shell redirection to capture output, which
should be OK as long as you don't need interactivity with the programs
being run.
I have something I wrote a long time ago for repetitive testing of a
command line program. I've considered putting it on CPAN but never
got around to writing much documentation for it. It works more or
less like this:
my $program = Test::CLI-new($path_to_program);
$program-stdin( $input );
$program-runs_ok( @args );
$program-stdout_like( qr/$expected/ );
$stderr - $program-stderr; # for custom examination
Right now, it assumes the program is invoked via perl, but that
could be stripped out. It also supports default arguments that are
added to every run. So you might use it like this:
my $interp = Test::CLI-new($path_to_interpreter, @default_params);
for my $c (@cases) {
$interp-stdin( slurp $c-{in_file} );
$interp-runs_ok( $c-{utility} );
$interp-stdout_like( $c-{stdout} );
$interp-stderr_is( $c-{stderr} );
}
If you want, I'll stick it in some boilerplate and publish it to CPAN
and github and then you're welcome to document it or improve it. Just
let me know if that would be helpful.
Or, if it would need a lot of adaptation for what you're looking to
do, you're welcome to browse it and take whatever is useful. It's
t/CLI.pm in the Pod::WikiDoc repo:
http://github.com/dagolden/pod-wikidoc/
Capture::Tiny is cute.
More than cute -- it works and works pretty universally for all
situations. I have a few more edge cases to code (e.g. if STDOUT or
STDERR are closed before it starts capturing), but my goal is to have
it replace the 20 or so other capturing modules out there. Then
things like Test::Output can be re-written around it and will become
much more powerful.
-- David
Re: build_requires, the xt and build_prereq_matches_use
On Fri, Feb 20, 2009 at 2:30 AM, Thomas Klausner d...@cpan.org wrote: I agree partly (it can look, but should not add any dependencies), but I'm very unlikely to find some time to fix this in the next weeks (http://use.perl.org/~domm/journal/38260). Understood. :-) But of course, patches welcome (eg I'd like to use Alias' new Find::File::Rule::Perl or take a look at Davids App::CPAN::Mini::Visit) FFRP may be more useful. visitcpan is mostly just handy to iterate though CPAN and execute arbitrary commands in each directory. -- David
Re: configure_requires and new META.yml keys
On Thu, Feb 19, 2009 at 2:05 PM, Eric Wilhelm scratchcomput...@gmail.com wrote: We do not currently have a plan for how to handle this. It would likely involve some mechanism of requiring compliance to a minimum version of the META spec. An old loader cannot know that it is too old. When we do define some sort of minimum META mechanism, an old loader with no mMm support will still never know that it is too old. This is similar to the configure_requires issue in that a tool not supporting c_r will fail in unknown ways. But the mMm could have cleanly forced c_r support had it been present from the get-go. That's a great point. I've added it for discussion on the Toolchain Roadmap page [1] [1] http://perl-qa.hexten.net/wiki/index.php/Toolchain_Roadmap -- David
Re: build_requires, the xt and build_prereq_matches_use
On Thu, Feb 19, 2009 at 4:49 PM, Marc Lucksch p...@marc-s.de wrote: Sorry, that previous post sounded way to agressive, this is probably because I'm not english speaker (Worst in my class and proud of it) and my Don't worry, I was not offended. Yes, but I kind of want to deliver the best module I can produce and Kwalitee is at least *a* way to improve my module. Except when it's not. For example, even if I wanted to have a perfect Kwalitee score, I can't because one of my modules *cannot* use strict and serve its intended purpose. Oh, well. It's important to recognize where Kwalitee is simple an opinion of quality and can be disregarded. Not always. You can have Foo.PL files that are run during make/Build and that require any module you want. True, but who does that? 138 distributions on CPAN use *.PL files. I didn't check how many use modules that need to be in build_requires. See http://gist.github.com/67173 for a list. (N.B. list created with App::CPAN::Mini::Visit) That's absurd. The Kwalitee analyzer shouldn't look in xt. Period. For this, yes. But before anyone goes fixing things, for uses_test_pod and uses_test_pod_coverage it has to look at 'xt' I'll be more specific -- build_prereq_matches_use shouldn't look at use in xt. If that means the analyzer needs to have @used_not_xt and @used_in_xt and look at both for pod/pod-coverage and only the first for build_prereq... then that's what needs to happen. -- David
Re: What's the common denominator in these NYTProf failures?
On Wed, Jan 28, 2009 at 8:50 AM, Tim Bunce tim.bu...@pobox.com wrote: I'm strugling to find a common denominator in these test results: http://bbbike.radzeit.de/~slaven/cpantestersmatrix.cgi?dist=Devel-NYTProf+2.07_94http://bbbike.radzeit.de/%7Eslaven/cpantestersmatrix.cgi?dist=Devel-NYTProf+2.07_94 The common denominator appears to be that they're all from BinGOs. Which means they're all CPANPLUS-based smoke tests running on virtual machines. I'm not sure what impact that has. He's pretty responsive so you can email him at his cpan.org address. (bin...@...) For future reference, these kinds of questions are better directed to the cpan-testers-discuss mailing list, which is regularly monitored by most of the high-volume testers. -- David
Re: Module::Build 0.30_01 - please test
On Thu, Dec 11, 2008 at 10:25 PM, Eric Wilhelm scratchcomput...@gmail.comwrote: Authors will notice that they now need to install Software::License Ick. How about a Task::Module::Build or Bundle::Module::Build to make that automatic? (I'll make one if people think it's a good idea.) Testers, it would be great if you could run through a sampling of distros out-of-band (to not annoy authors with our alpha bugs.) David, do you have a write-up from the procedure you used on this last time? http://www.nntp.perl.org/group/perl.qa/2008/09/msg11418.html Um. No write up. I seem to recall that I set up CPAN::Reporter to use Test::Reporter::Transport::File to save reports to a directory, then ran CPAN::Reporter::Smoker with a list of distributions to test. I did that for both versions of M::B and then I parsed the names of the resulting files to see how many distributions changed grades and how. I'll see if I still have my helper code somewhere on my smoke VM. -- David
Re: [rt.cpan.org #40976] New files installed without user-write permission
My understanding is that EU:MM and M::B chmod everything 0444 (maybe plus executable) when copying them to blib and ExtUtils::Install just picks them up from there, no? -- David On Sun, Nov 16, 2008 at 7:34 PM, demerphq [EMAIL PROTECTED] wrote: Can anyone help me out with this? Is this EUI's fault or something else? Is anything to do with this permission issue that we have been discussing? Cheers, yves -- Forwarded message -- From: Guillaume Rousse via RT [EMAIL PROTECTED] Date: 2008/11/16 Subject: [rt.cpan.org #40976] New files installed without user-write permission To: undisclosed-recipients Sun Nov 16 13:41:21 2008: Request 40976 was acted upon. Transaction: Ticket created by GROUSSE Queue: ExtUtils-Install Subject: New files installed without user-write permission Broken in: 1.52 Severity: Normal Owner: Nobody Requestors: [EMAIL PROTECTED] Status: new Ticket URL: http://rt.cpan.org/Ticket/Display.html?id=40976 All new files installed through ExtUtils::MakeMaker-generated makefiles are installed without user-writable permission bit set. As a consequence, all attempt to modify them during installation with a non-privilegiated user (typically, package building) fails. PDL installation is a good example: couldn't open /home/guillomovitch/cooker/perl-PDL/BUILDROOT/perl-PDL-2.4.4/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi/PDL/Index.pod at Doc/scantree.pl line 44. [EMAIL PROTECTED] perl-PDL]$ ll /home/guillomovitch/cooker/perl-PDL/BUILDROOT/perl-PDL-2.4.4/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi/PDL/Index.pod -r--r--r-- 1 guillomovitch users 10763 2008-11-16 19:23 /home/guillomovitch/cooker/perl-PDL/BUILDROOT/perl-PDL-2.4.4/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi/PDL/Index.pod Removing 1.52 version, so as to use perl-5.10 provided 1.44 version, fix the issue. -- perl -Mre=debug -e /just|another|perl|hacker/
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
On Thu, Nov 13, 2008 at 3:39 AM, Shlomi Fish [EMAIL PROTECTED] wrote: What I was expressing is that the CPAN shell can do the twiddling to strip flags at the point of extraction, rather than PAUSE stopping it at the gate. Archive::Tar already does this (see $Archive::Tar::INSECURE_EXTRACT_MODE). Archive::Tar does, but Archive::Extract (which CPANPLUS uses) doesn't. It was a bug. Addressed in 0.28 as a result of these discussions. The next non-development release of CPANPLUS will use the new Archive::Extract and close the security hole under discussion. -- David
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
On Wed, Nov 12, 2008 at 3:17 PM, demerphq [EMAIL PROTECTED] wrote: I rather strongly object to this change. I totally understand -- but keep in mind that this was in response to someone flagging this as a potential (if highly unlikely) security hole, forwarding it to some security-watchdog site, etc. So the rapid response was close the hole so no one can say CPAN creates a security risk. (Other than the usual, obvious one of running arbitrary code...) So it causes some pain, but in my view, it's in the interest of the Perl community to be seen as vigilant. this silly test. What really gets me going tho is I WASNT TOLD THIS ABOUT 1.51_01 or 1.51_02 or 1.51_03 or (do you detect a pattern here?) 1.51_04 or 1.51_05, all of which i uploaded in the last few days in the exact same way!!! That's kind of a loophole, since development versions aren't indexed. I think any upload that fails a security test should probably be rejected, whether development or full release. IMO if the toolchain is to work this should happen at PAUSE (if it can detect this problem IMO it should just damn well fix it itself) or at extraction. It *is* being fixed at extraction. But it requires people to upgrade CPAN and CPANPLUS (maybe Archive::Extract as well). It was a faster fix to close the PAUSE indexing door than to get those fixes released. Whats going to happen next, stuff rejected because they don't have *nix line endings? Or *nix style shebangs? Or use perl-qa's preferred indentation style or something? H?! Maybe instead, at a minimum, every distribution should be run against Perl::Critic at severity level 4 and anything that doesn't pass should be rejected as well. ;-) (THAT'S A JOKE, PEOPLE!) /g Right there with you, except my /grrr was back when the security alert got sent off to the watchdogs while the discussion was still going on as to whether this was a significant risk in the first place. -- David
Re: What are CPAN tester N/A reports?
On Thu, Nov 6, 2008 at 8:15 AM, Ovid [EMAIL PROTECTED] wrote: Questions: 1. How can I find out why a CPAN testers report is N/A? I'm getting test failures on Test::Most which are N/A and I don't know why. N/A should only be for situations where the distribution isn't compatible with the platform or perl. It used to be used for missing prerequisites but testers should have upgraded. However, CPANPLUS based testing sometimes gives funny results depending on what version of CPANPLUS is used, whether Module::Build support is available, and so on. Bingos will have to investigate. 2. To debug the problem, I've rewritten my t/00-load.t to show the version number of all of the dependencies I specify. Not all CPAN test reports have this information. Should I bug CPAN reporters to upgrade or something? I don't know enough about what they do to be sure of the right action here. CPAN+CPAN::Reporter based tests include this information. CPANPLUS based testers *may* include this information, or may not. (I think the exceptions are CPANPLUS vs Module::Build related, still, but I could be wrong.) To be quite frank, CPANPLUS based testing is such a crapshoot at times that I mostly ignore anything weird from it about my own CPAN distributions. Of course, I might be biased in that view, since anything wierd from CPAN::Reporter about my own CPAN distributions means that I have a bug to fix one way or another. :-) -- David
Re: What are CPAN tester N/A reports?
Additional note as a reminder to all -- please send CPAN Testers related questions to [EMAIL PROTECTED], not the Perl QA list. You're more likely to get a greater number of helpful responses on that dedicated list. David On Thu, Nov 6, 2008 at 11:50 AM, David Golden [EMAIL PROTECTED] wrote: On Thu, Nov 6, 2008 at 8:15 AM, Ovid [EMAIL PROTECTED] wrote: Questions: 1. How can I find out why a CPAN testers report is N/A? I'm getting test failures on Test::Most which are N/A and I don't know why. N/A should only be for situations where the distribution isn't compatible with the platform or perl. It used to be used for missing prerequisites but testers should have upgraded. However, CPANPLUS based testing sometimes gives funny results depending on what version of CPANPLUS is used, whether Module::Build support is available, and so on. Bingos will have to investigate. 2. To debug the problem, I've rewritten my t/00-load.t to show the version number of all of the dependencies I specify. Not all CPAN test reports have this information. Should I bug CPAN reporters to upgrade or something? I don't know enough about what they do to be sure of the right action here. CPAN+CPAN::Reporter based tests include this information. CPANPLUS based testers *may* include this information, or may not. (I think the exceptions are CPANPLUS vs Module::Build related, still, but I could be wrong.) To be quite frank, CPANPLUS based testing is such a crapshoot at times that I mostly ignore anything weird from it about my own CPAN distributions. Of course, I might be biased in that view, since anything wierd from CPAN::Reporter about my own CPAN distributions means that I have a bug to fix one way or another. :-) -- David
Re: http://www.nntp.perl.org/group/perl.cpan.testers/2008/10/msg2399796.html
On Thu, Oct 9, 2008 at 2:52 PM, Pete Krawczyk [EMAIL PROTECTED] wrote: Third, there was no reason to cc the perl-qa mailing list. None at all. This list is for discussion of testing methods and applications, not for chastising the result of an automated smokebot. Pete makes a good point. Complaints about automated smokebots should be directed to [EMAIL PROTECTED] which many of the CPAN Testers subscribe to and where such complaints are considered on-topic. -- David
Re: Archive::Tar does not behave like gnu tar
On Wed, Sep 24, 2008 at 10:13 AM, David Cantrell [EMAIL PROTECTED] wrote:
NetBSD
$ tar --help | grep no-same-permissions
tar: unknown option -- help
usage: tar [-]{crtux}[-befhjlmopqvwzHOPSXZ014578] [archive] [blocksize]
[-C directory] [-T file] [-s replstr] [file ...]
Where's the DWIM option for the DCANTRELL module? ;-)
Looks like Archive::Tar, while slower, it more consistently
controllable everywhere. Otherwise, we need experts on all systems
and versions of tar to figure out the right flags. And preferably a
module to abstract all that away. Ugh.
-- David
Re: Archive::Tar does not behave like gnu tar
On Tue, Sep 23, 2008 at 3:06 PM, Eric Wilhelm [EMAIL PROTECTED] wrote: So, something needs to set $Archive::Tar::CHMOD = 1; to make it behave like gnu tar. $Archive::Tar::CHMOD = 1 or 0? The default is 1. From the Pod: By default, Archive::Tar will try to chmod your files to whatever mode was specified for the particular file in the archive. In some cases, this may not be desired. In that case, set this variable to 0 to disable chmod-ing. So I think 0 would leave it to the umask, or am I reading this wrong? If it's that easy, I can go patch CPAN.pm right away. -- David
Re: Archive::Tar does not behave like gnu tar
On Tue, Sep 23, 2008 at 3:53 PM, Eric Wilhelm [EMAIL PROTECTED] wrote: Yeah, sorry. 0. It has a default which is opposite of gnu tar. If it's that easy, I can go patch CPAN.pm right away. Done: CPAN.pm is patched in trunk. And CPANPLUS I guess. I'm not sure what else happens in Archive::Extract and whatever other code is involved, I've only dug deep enough to determine that shlomi's problem is caused by something other than gnu tar. It looks like Archive::Extract attempts to do the right thing, but sets the wrong variable: local $Archive::Tar::Constant::CHOWN = 0; I'll file a bug report on RT. And are there other tar programs? .zip? -- David
Re: Archive::Tar does not behave like gnu tar
On Tue, Sep 23, 2008 at 7:27 PM, David Cantrell [EMAIL PROTECTED] wrote: If people can point me at some simple tests (sorry, I've not been following this thread), I can see what these tars do: NetBSD FreeBSD Irix Solaris They all appear to be different - at least, they all respond differently to 'tar --help'. This would tend to indicate to me that the solution is use Archive::Tar. Please see if they all honor a --no-same-permissions flag. $ tar --help | grep no-same-permissions --David
Re: [RFC] Dealing with World-writable Files in the Archive of CPAN Distributions
On Mon, Sep 22, 2008 at 8:40 AM, Shlomi Fish [EMAIL PROTECTED] wrote: My suggestion for resolving this is to modify the smoking modules so, after the archive is unpacked (with a proper umask and arguments to tar), they will traverse the directory tree and look for any world-writable files. If any are found, they will report the smoking of the module as FAIL, and delete the unpacked directory tree, without doing the perl Makefile.PL/Build.PL ... dance. This isn't just a smoking problem, right? A normal CPAN/CPANPLUS install would trigger the same warning? We could give an option for doing this, if it bothers you. But I'm tired of finding these files in the msec report and reporting them manually. Now I volunteer to implement this. I think that CPANTS is probably the better place for this kind of analysis, particularly because it's static and because the reason for the Kwalitee point is clear. It sounds like exactly the kind of thing that fits among the core Kwalitee metrics. There are some reasons I think that CPAN Testers is *not* the right place for this: * The CPAN Testers grades relate only to the ability to build/test a distribution. Unless world writable files prevent that, FAIL or UNKNOWN are not appropriate * Someone would have to read the FAIL and pay attention to understand that the problem is a world-writable file (whereas it's obvious on CPANTS what the problem is) * CPAN::Testers is no longer notifying authors directly anyway -- David
Re: [RFC] Dealing with World-writable Files in the Archive of CPAN Distributions
On Mon, Sep 22, 2008 at 9:24 AM, Shlomi Fish [EMAIL PROTECTED] wrote: Well, it does. However, hardly anyone pays any attention to CPANTS, and it's out there in the background, and hardly influences the general perception of the module. As an aside, if the core Kwalitee metrics are sufficiently tight, I'd support seeing core Kwalitee on search.cpan.org. I think all the controversial stuff has moved to the optional metrics, now. * The CPAN Testers grades relate only to the ability to build/test a distribution. Unless world writable files prevent that, FAIL or UNKNOWN are not appropriate World-writable files are a security risk and the CPAN shell should refuse to test the distribution if they exist. A security conscious admin won't install such modules if they generate world-writable files. As such, one should not proceed to the build/test stage and fail immediately. These are orthogonal problems. CPAN Testers is intended to determine whether a distributions tests pass. The project recently returned to the original definition of UNKNOWN to include build failures, meaning that FAIL is reserved only for failing tests. World-writable files are unrelated to the success of either building or testing. That doesn't mean that they aren't potentially important. However, I'm a little confused as to the nature of the security risk and the level of risk involved. If the risk is that someone might replace my Makefile.PL in the exact moment between untarring a distribution and running 'perl Makefile.PL' then I think that the overall threat level is pretty low. After all, any PL or test file can do anything it wants to your system -- this is why modules shouldn't be build or tested as a privileged user. And even if you skim the PL file, how often do you read every test file to make sure it doesn't call system(rm -rf /)? If the risk is that a world-writable file could be installed, it seems like most things in blib seem to be set read-only -- but I just noticed that some things are not (e.g. scripts man pages). That seems like it might be a security bug in EU::MM or M::B and worth fixing. In any case, my point is that if there is a real security risk of a non-trivial magnitude, then we should fix it. If it isn't of significant magnitude, then I question whether anything needs to be done about it at all if the only impact is Mandriva users getting annoying warnings. In either case, I don't think CPAN Testers is the right project to notify authors about world-writable files. If you don't think CPANTS is too passive, it would be a fairly simply perl program to monitor CPAN uploads, pull down and extract the tarball, check for world-writable files and email the author. E.g. one could use POE::Component::SmokeBox::Uploads::RSS, LWP, Archive::Extract, Email::Simple and POE::Component::Client::SMTP and just add a bit of glue. It's actually a harder problem for CPAN Testers tools because of the need to tap into CPAN or CPANPLUS between the extraction and the initial call to 'perl Makefile.PL' or 'perl Build.PL', which they currently don't do. -- David
Re: [RFC] Dealing with World-writable Files in the Archive of CPAN Distributions
[Copying Andreas, Jos, Schwern and the Module::Build list]
Well, I'm not sure that escalating to Securiteam at this point was
necessary given the low overall risk of the threat, but let's set that
aside and find some agreement on closing the hole. Here are my
thoughts on some of the problems/options:
Problem 1: race condition between unarchiving and execution if
Makefile.PL or Build.PL is world writable (ditto test files as well)
(a) Have CPAN and CPANPLUS refuse to run 'perl *.PL' if the PL in
question is world writable.
(b) Have CPAN and CPANPLUS not preserve mode permissions even for
root; that's --no-same-permissions) for tar or $Archive::Tar::CHMOD
= 0 for Archive::Tar. I presume there's a comparable thing for zip
archives. That leaves it up to the users umask setting.
(c) Both
(d) Something else
(e) Ignore it
Personally, I lean towards (b) as that seems relatively sane and
minimally disruptive.
For (a), I worry about edge cases for operating systems that don't
have unixish permissions. E.g. on Win32, an administrative always has
write-permission, even on files set to be read-only. A less
aggressive option than (a) is just to issue warnings about
world-writable files.
For completeness, there's a possible problem 2: An insecure umask can
lead to world-writable files, including not only the unarchived files,
but also Makefile (or Build) and some files in blib [1]:
(a) Ignore this -- insecure umask isn't Perl's problem to worry about
(b) Set an appropriate umask before generating Makefile, Build or
copying things to blib.
For this one, I lean towards (a).
-- David
[1] EU::MM and M::B have a race condition when then copy files to blib
before calling chmod. Also, shebang fixes seem to undo the chmod and
leave script files with the default permissions from the umask
On Mon, Sep 22, 2008 at 1:37 PM, Shlomi Fish [EMAIL PROTECTED] wrote:
Hi all.
Note to Securiteam: there's a link to the possible security problem report
at the bottom.
On Monday 22 September 2008, chromatic wrote:
On Monday 22 September 2008 08:41:31 Michael G Schwern wrote:
Shlomi Fish wrote:
Let's suppose Makefile.PL is world-writable. While the distro is being
unpacked, a malicious user writes something like:
{{{
system('rm -fr $HOME');
}}}
to it, and after you come to the perl Makefile.PL stage - you lose
your home-directory. ;-)
Run that by me again how the Makefile.PL being world-writable has any
effect on that? If a Makefile.PL does an rm -rf $HOME and you run it,
it doesn't matter what permission flags are on the file. Your home
directory is gone.
There's a race condition attack between the time the CPAN client *writes*
the world-writeable file and the time the CPAN client *executes* the
world-writeable file. During that time, anyone on the system can write
anything to the file, replacing its legitimate and safe contents with
malicious contents.
That's completely orthogonal to the problem of the Build.PL/Makefile.PL
containing malicious code.
Right. I decided that it was a major problem with how CPANPLUS handles such
situations (regardless to whether we are smoking or just installing) and
reported it here:
http://rt.cpan.org/Ticket/Display.html?id=39516
Please don't keep it more public than it is already until there's a good fix.
Regards,
Shlomi Fish
-
Shlomi Fish http://www.shlomifish.org/
My Aphorisms - http://www.shlomifish.org/humour.html
Shlomi, so what are you working on? Working on a new wiki about unit testing
fortunes in freecell? -- Ran Eilam
Re: [RFC] Dealing with World-writable Files in the Archive of CPAN Distributions
On Mon, Sep 22, 2008 at 6:23 PM, Eric Wilhelm [EMAIL PROTECTED] wrote: Yes. Would someone please explain to me how this issue is not already made a mostly non-issue by having a proper umask and running CPAN as non-root? Someone in the thread (sorry, forget who and I'm not going to search for it) gave the example that access to Makefile.PL allows arbitrary additions to the Makefile install target that a non-root user might well run with sudo make install -- thus a Makefile.PL compromise could be used to execute arbitrary code as root. I'm not saying it's a big threat. Risk level is the combination of likelihood of an event and the severity of the event and the first is low. Nevertheless, stopping archive extraction from creating world-readable files is probably a good idea. -- David
Re: App::Prove::History
On Fri, Sep 19, 2008 at 5:21 AM, Ovid [EMAIL PROTECTED] wrote: I have the basics of App::Prove::History working (not yet releasable), and it's time to ask what people want in it. Should I just store the results of individual test programs or also the results of individual tests? TAP streams? Something else entirely? Store the most granular thing you can, I'd say. Then people can write tools to filter it down. -- David
Re: PerlUnit
On Tue, Sep 16, 2008 at 8:13 AM, Nicholas Clark [EMAIL PROTECTED] wrote: So everyone gainfully employed because they know what they are doing is automatically disqualified? I don't think they limit votes the way PM does, and it only takes two upvotes to get qualified to vote up, so we just need a known question posted that a bunch of us answer in and anyone with +15 slacker points votes up, then once you get your 15 slacker points you can vote everyone else up. Easy slacker points for all assuming the site gods don't nerf us. :-) -- David
CPAN Testers remailer program available
I've just released POE::Component::Client::NNTP::Tail to CPAN. In the examples directory, see the remailer.pl program. It will follow the NNTP perl.cpan.testers group, pick out an author's distributions and remail certain grades directly to the author. E.g: $ perl remailer.pl --author=DAGOLDEN --grade=FAIL --grade=UNKNOWN There are also options to set SMTP server and CPAN mirror. If anyone wants more specific filtering (e.g. OS), you'll want to edit the new_header subroutine with your own logic. This is very, very alpha and is intended to help authors who want CPAN Testers by email until a centralized all reports email service is built. I'll take suggestions for improvements only with patches. :-) git://echo.dagolden.com/git/POE-Component-Client-NNTP-Tail/ -- David
Re: CPAN Testers remailer program available
To make life a bit easier with dependencies, I've packaged the remailer in its own distribution (and fixed a bug or two already). App::CPAN::Testers::Remailer Usage: $ cpantest-remailer --author=DAGOLDEN --grade=FAIL --grade=UNKNOWN -- David On Tue, Sep 16, 2008 at 12:39 PM, David Golden [EMAIL PROTECTED] wrote: I've just released POE::Component::Client::NNTP::Tail to CPAN. In the examples directory, see the remailer.pl program. It will follow the NNTP perl.cpan.testers group, pick out an author's distributions and remail certain grades directly to the author. E.g: $ perl remailer.pl --author=DAGOLDEN --grade=FAIL --grade=UNKNOWN There are also options to set SMTP server and CPAN mirror. If anyone wants more specific filtering (e.g. OS), you'll want to edit the new_header subroutine with your own logic. This is very, very alpha and is intended to help authors who want CPAN Testers by email until a centralized all reports email service is built. I'll take suggestions for improvements only with patches. :-) git://echo.dagolden.com/git/POE-Component-Client-NNTP-Tail/ -- David
Re: CPAN Testers - Author Notification System
On Thu, Sep 11, 2008 at 5:17 PM, Michael G Schwern [EMAIL PROTECTED] wrote: The way it looks right now, I want my CC's back. :( Well, I guess we win some and we lose some. Just brainstorming for the moment, but if we had an option to forward all your FAIL or UNKNOWN reports to the distribution's RT queue, would that serve? You'd get the RT email notification, plus you'd have the senders email with it, plus you'd have it in a tracking system where you could decide priority, merge it to existing bug reports or just close it out if you determine it's bogus. Possible variation (a) -- just forward the *first* FAIL/UNKNOWN report for a distribution/perl/osname tuple so your RT queue isn't deluged Possible variation (b) -- forward from the central bot back to your author email or arbitrary email (e.g. project mailing list) The batch is a temporary fix to keep info flowing while we get things centralized. Once it's centralized, we can continue to innovate smart ways to redistribute reports. -- David
Re: CPAN Testers - Author Notification System
On Thu, Sep 11, 2008 at 6:51 PM, Michael G Schwern [EMAIL PROTECTED] wrote: They'll be sent out early morning (~2am) UK time, so should be waiting in you inbox first think in the morning. Ok, I have to wait until the beginning of the next day to fix the problem. FYI, I have 90% of my POE nag bot to hassle (oops, I mean remind) CPAN Testers to upgrade Test::Reporter. It build on a POE::Component to tail the NNTP feed. Once I put that POE::Component on CPAN (soon), it should be a pretty trivial matter to whip up a script to reflect emails for your distributions to you. I might even write that for the examples directory. Then you can fire it up on a server of your choice and get your CC's back -- but you'll be in control, you can filter to your heart's content, etc. -- David
Re: CPAN Testers - Author Notification System
On Thu, Sep 11, 2008 at 7:32 PM, brian d foy [EMAIL PROTECTED] wrote: Just brainstorming for the moment, but if we had an option to forward all your FAIL or UNKNOWN reports to the distribution's RT queue Ouch. Some people might want that, but only as an opt-in thing please. Purely opt-in. CPAN Testers is going to be an opt-in service very, very soon now. Today I considering writing a bot to listen to cpan-testers and simply resend individual reports to me, depending on how quickly the opt-in stuff gets set up and if it actually works. Give me until next week, OK? I'll write one that you can download from CPAN. -- David
Re: Sub::Uplevel vs Test::More
On Wed, Sep 10, 2008 at 10:01 AM, Aristotle Pagaltzis [EMAIL PROTECTED] wrote: The correct solution (which also doesn't require changes to CPANTS) is called warnings::compat. Assuming you actually care that much… Assuming I don't mind adding a dependency that is only needed for perl 5.6. Or customizing my *.PL to only add the dependency on perl 5.6. All for a version of Perl I no longer care about since Schwern declared it dead. And since I just got beat up over suggesting adding exit 0 to *.PL... I'll invoke the same logic to say no thanks. :-) David
Re: Sub::Uplevel vs Test::More
On Wed, Sep 10, 2008 at 1:30 PM, Eric Wilhelm [EMAIL PROTECTED] wrote: If CPANTS can find the -w in the tests or whatever and the META.yml says 5.6... (Because enabling warnings in *everyone else's* code is a good way to placate a static kwalitee scanner?) I give up. I don't think people realized that my CPANTS comment was a joke. David
