Re: Test::Benchmark ??
* Michael G Schwern [EMAIL PROTECTED] [2003-12-04T16:51:03] On Thu, Dec 04, 2003 at 05:26:07AM -0500, [EMAIL PROTECTED] wrote: I'd say a lot of the trouble comes from the fact that you're using the automated test framework for something that isn't an automated test. But it could be. It would be nice to have a test like make sure the hand optimized version is faster than the unoptimized version or make sure the XS version is faster than the Perl version. If this isn't what he meant, it's what I thought he meant. A T::B that allowed these kinds of tests would be really sweet for a number of applications. Even without the perlmip (which seems cool but further off), this would be great. -- rjbs pgp0.pgp Description: PGP signature
rt.cpan.org
Alright, I know this is tangential to QA, but I'm trying to access rt.cpan.org to update a bug on a module I'm working on. I can't log into rt.cpan.org using my PAUSE credentials. I emailed [EMAIL PROTECTED] a month ago, and again this week, and received only an automated reply. I was assigned request numbers 4655 and 4288, but I can't find these on fsck.com anywhere. Can someone tell me: (a) what I'm doing wrong (b) where to find information on these reports (c) whether there's a much more appropriate list for these questions Thanks. -- rjbs pgp0.pgp Description: PGP signature
Re: rt.cpan.org
* Michael G Schwern [EMAIL PROTECTED] [2003-12-06T20:04:24] On Sat, Dec 06, 2003 at 07:46:24PM -0500, Ricardo SIGNES wrote: Alright, I know this is tangential to QA, but I'm trying to access rt.cpan.org to update a bug on a module I'm working on. I can't log into rt.cpan.org using my PAUSE credentials. Stupid question: can you log into PAUSE using your PAUSE credentials? https://pause.perl.org/pause/authenquery?ACTION=add_uri Yup. I can also log into books.perl.org -- I don't use any other PAUSE-authing sites. If you're feeling your oats you can hop onto irc.perl.org #perl and track down obra (aka Jesse). I went on joined #rt, but it was asleep. I'll try #perl. Thanks. -- rjbs pgp0.pgp Description: PGP signature
Re: Style question
* Francisco Olarte Sanz. [EMAIL PROTECTED] [2004-05-25T13:58:21] ? Which aproach is better, have a single independent huge test file or several interdependent smaller ones ( w/ notes in the readme stating test dependence ) ? The better approach is the one that makes it most likely for you to write enough tests, use them, and maintain them. ;) Given the situation you're describing, I'd use a number of smaller files that need to work together. Look at Test::Manifest for a way to run scripts in order. -- rjbs pgpWgA4XX7UlE.pgp Description: PGP signature
Re: Adding analysis to Devel::Cover reports
* Michael Carman [EMAIL PROTECTED] [2004-06-07T23:13:02] On 6/7/2004 9:20 PM, Andy Lester wrote: The ALT attribute as tooltip thing isn't portable, though. I don't use ALT, I use TITLE. That's the right way according to the W3C and supported by at least IE and Mozilla-based browsers. Or did you mean something else by isn't portable? Also by KHTML browsers, like Safari. -- rjbs pgpRqSa7TErYp.pgp Description: PGP signature
Re: Phalanx: What if full coverage isn't possible?
* Pete Krawczyk [EMAIL PROTECTED] [2004-07-09T13:10:52] Devel::Cover will always see that as a partial test, and never a full test: [ ... ] Is that a bug, then? Or is it something else? And how should I notate that, keeping in mind the goals of Phalanx, so that it's clearly visible that no test of that condition's failure can truly take place in a regular make test run? This came up just a few weeks ago, and I think the general feeling was something like: cover everything, if possible; make everything coverable, if practical; don't write stupid code to avoid uncoverable situations. So, if you know that branch isn't covered, but can't (really) be taken, it's up to you whether that's a problem or not. Deming, Who Knows Quality, would tell us to avoid arbitrary goals of 100% coverage... -- rjbs pgplEChA9MEik.pgp Description: PGP signature
is_deeply and overloaded obejcts
I was going nuts recently, trying to figure out why I couldn't use is_deeply to compare objects. I've finally determined that it's only an issue (as far as I see) when comparing objects that overload dereferencing to their implementation type. The attached code should fail all three tests; at no point are the two structures identical. Instead, the third test -- when the structures are blessed into an implementation-type overloaded class -- passes. This should be simple-ish to fix, I'd think. 1. two objects are different if their implementation types differ 2. two objects are different if their blessed-into-temp-class structures differ 3. otherwise two objects are the same So, given two Garbage::Overloaded=HASH objects, we'd bless them into Test::More::TempClass and then compare those. Thoughts? -- rjbs obj_deep.t Description: Troff document pgpoF89BZ6dWy.pgp Description: PGP signature
Re: Test-Regex-0.01.tar.gz
* Rafael Garcia-Suarez [EMAIL PROTECTED] [2004-08-02T10:12:22] What kind of useful diagnostics could this module emit in case of failure? IMO they have to be precisely detailed. # failed comparisons follow: # expected $1: (610) # got $1: 691 # expected $4: x258 # got $4: 258 That's just off the cuff, but I think that's probably as reasonable as it gets, unless you want to (ugh!) diag nothing. -- rjbs pgpCLu0QZe1r1.pgp Description: PGP signature
Re: Test::Harness/prove: printing the test name when a test fails
* Andrew Savige [EMAIL PROTECTED] [2004-08-31T04:24:55] Is there a way to make Test::Harness do this? If nothing else, in the given case, it would have made more sense to use is() is(0, 1, Zero shouldn't equal one.); That will print got/expected values on error, even when not verbose. -- rjbs pgptKuQoc0HaU.pgp Description: PGP signature
Re: [Devel::Cover] @INC at Runtime?
* David Wheeler [EMAIL PROTECTED] [2004-09-17T00:51:22] So, what's to be lost by having the inc directories default to the contents of @INC when you load Devel::Cover rather than at install time? Presumably the problem is that by runtime, lib and blib directories are already in @INC, so the things in your ./distro/lib/ won't be covered, defeating the point. As to whether there is a simple way to resolve this, I haven't thought about it. -- rjbs pgpE0NUl8lZlb.pgp Description: PGP signature
Re: Uncle Bob on Coding Standards
* H.Merijn Brand [EMAIL PROTECTED] [2004-12-14T11:28:19] About spaces, another thing springs to mind, for which I would gladly kill the responsible people to allow it (I bet M$ was the first to push it): Spaces in database table and field names. DON'T! NEVER! Once you start it, you will never be able to escape the quicksands of (incompatible) quotation and unportability of your scripts, be that in sql, sh, perl/dbi, hli, or e/sql SELECT rsUniqueId FROM [SQL-ABCdatabase2K]..tblsTABLE as t JOIN [SQL-ABCdatabase2K]..tblwPRODUCT p ON p.ABC P/N = t.rsProductCode WHERE p.id IN (SELECT ItemIdentifier FROM tblSomeIds) Yup. -- rjbs pgpzduHkk6UnF.pgp Description: PGP signature
Re: Why a scoreboard?
* David A. Golden [EMAIL PROTECTED] [2005-04-02T05:27:18] Andy Lester wrote: Why is there a scoreboard? Why do we care about rankings? Why is it necessary to compare one measure to another? What purpose is being served? Why is there XP on perlmonks? Or Karma on Slashdot? Or for that matter, why do we grade students' exams (particularly, why do we often grade them on a curve)? This is not a good analogy to Kwalitee, because XP and Karma are primarily awarded by humans who can make judgements based on reason. Kwalitee is a judgement based on statistical analysis. What if XP on perlmonks was awarded by automated grammar analysis, use of strict in code snippets, and the finding of dictionary words in acrostic interpretation of paragraphs? That wouldn't be so good. I think the advantage of a scoreboard system is that metrics like this are a motivator. Absolutely! Do we want to motivate people to go back and add pod-coverate.t when they already use Devel::Cover? Or would we rather they improve clarify of documentation? Et cetera. I think the scoreboard is fun and cute, but I think that there is an extent to which people are going to let it distract them from actual improvements. -- rjbs pgpuWya72Y1x8.pgp Description: PGP signature
Re: Test::Expect
* Adrian Howard [EMAIL PROTECTED] [2005-04-14T15:37:07] On 14 Apr 2005, at 11:36, Leon Brocard wrote: Oh, I forgot to mention to perl-qa that I wrote Test::Expect: http://search.cpan.org/dist/Test-Expect/ It's nice. Already used it :-) Does anyone who has used both Test::Expect and Test::Output feel like giving a simple comparison? -- rjbs pgpB2LbrsGf1o.pgp Description: PGP signature
Devel::Cover and HTTP::Server::Simple
Yesterday, hide gave me some sweet example code to use HTTP::Server::Simple and Test::WWW::Mechanize to test Rubric's CGI bits. I've started working with them, and they make me happy. I've realized that the server, which is forked from the test script, doesn't have its usage show up in Devel::Cover. I can see why that would happen but... I'm so addicted to the awesome utility of coverage reports that I am loathe to have to cope with half my code being so hard to cover. Does anyone have a good idea on how to make this work? -- rjbs pgpQAYYbSZOtt.pgp Description: PGP signature
Re: Self-testing kwalitee
* Adam Kennedy [EMAIL PROTECTED] [2005-07-11T10:10:31] Note: The last kwalitee test, the one related to Devel::Cover, is considered dangerous by a non-trivial percentage of the community, and there's been a lot of debate on whether it should be removed. Sorry, I should have said Pod::Test::Coverage. Regardless, it executes code. I wonder how it deals with testing Acme::BadExample. Remind me to put in a hard infinite loop. I don't think you understand the test. CPANTS did not check for POD coverage, it checked that you seemed to have a pod coverage test. You could create one that did nothing, and that caused the test to pass. CPANTS does not evaluate your code, it just looks at it. The objection was that some people don't want to distribute that kind of test, because its results are for the developer's benefit, not the user's. -- rjbs pgpCRZP8rGREC.pgp Description: PGP signature
Re: Test::More behavior issue with Devel::Cover + patch
* Pete Krawczyk [EMAIL PROTECTED] [2005-11-03T12:46:48] The solution I see is to make sure the object can() isa(), thus avoiding the die in the process: It was using -isa instead of UNIVERSAL::isa because isa might be overridden. Surely the same could apply to -can. -- rjbs pgpF9z1fSpvrD.pgp Description: PGP signature
Re: CPANTS: has_signature, has_pod_index
* Tels [EMAIL PROTECTED] [2005-11-06T09:44:14] * has_signature: a SIGNATURE file exists, and is a valid signatur. That seems reasonable, even though I dread signing all my dists. I feel like it will be a big hassle, but maybe I'm just afraid of change. * has_pod_index: The POD contains at least one X keyword that helps POD indexers. Whether only one is usefull is open for debate, because at least the license (Xgpl), your CPAN ID under authors (xtels), and some generic keyword what your module (Xfoo) is about can probably added even for the most minimal module. Can you give an example of how this has any practical impact on anything? -- rjbs pgp8S7CW0B3F2.pgp Description: PGP signature
Re: TODO test paradox: better TODO test management?
* demerphq [EMAIL PROTECTED] [2006-02-01T03:26:55]
And I think you've conveniently sidestepped my main point which is
that TODO tests passing are errors. Consider you have two TODO tests,
both of which depend on a common set of functionality. Both should
pass or both should fail.
I just don't agree. I prefer the current model.
I defend your right to disagree, however, and support it with this idea:
strict_todo {
frobulate_ok(someone implemented frobulate()!!);
};
sub strict_todo() {
local $Test::Builder::OPPOSITE_WORLD = 1;
$_[0]-();
}
...where OPPOSITE_WORLD causes ok to be not ok, not ok to be ok, and cucumbers
to taste better pickled.
(I wouldn't exactly implement it that way, but you get the idea, and I bet that
a patch to provide test result reversal would not be unwelcome.)
--
rjbs
pgp3u61k8JOSm.pgp
Description: PGP signature
Re: Module requirements (was: Module::Build and installing in non-standardlocations)
* H.Merijn Brand [EMAIL PROTECTED] [2006-04-04T10:40:39] And then still people make more of the same. Take Getopt::Long. A perfect and very functional module. Full of features, matured, and actively maintained. Now go look at CPAN, and see how many people either do not like it or find other reasons to write their own. The problems arise when authors of big modules start prefering non-core versions of good core modules. IMHO something like that should give you a (very) negative score on CPANTS. Could you elaborate on this? As stated, it seems pretty ludicrous to me. It reads like this: You should not use module B that is like module A, if A is in the core distribution. This is true regardless of the fact that B may be better optimized for your current needs, planned needs, programming style. This can be further distilled to: There's more than one way to do it, but most of them will get you dirty looks. -- rjbs
Re: Module requirements (was: Module::Build and installing in non-standardlocations)
* H.Merijn Brand [EMAIL PROTECTED] [2006-04-05T02:39:20] I'll just mention two things, both very different A. CORE modules are tested on all supported architectures, while CPAN modules do not give that guarantee. The smoke system still causes all possible combinations to be tested on various architectures in various configurations. Absolutely; this, I cede. B. Let's just name YAML. Up until 0.38 it was not to difficult to install a module that is very useful, but now in 0.58, it uses a different test suite, that needs Spiffy, that needs .. Ugh, yeah, but you're picking a very specific and specifically troubled distribution. You also didn't mention (maybe you've been spared learning) that YAML used to export :all but now doesn't. That said, I don't dispute the point that it can be wildly obnoxious when Something::Trivial requires DBD::MySQL and Data::Dump::Streamer when it could use neither -- or at least rely on AnyDBM and Data::Dumper. It will just meant that fewer people want to use that trivial module because it takes longer to install and is less well-tested by virtue of its dependencies. My opinion only. I don't think your opinion, as stated, is insane. I think it's better to vote with your installer, though, and avoid using those modules that have more dependencies than you think is right. I think that Kwalitee is an over-rated concept, but not useless -- but weirder things keep getting proposed to make it even less useful. I think too many non-core deps would be a nail in its coffin as a global metric. -- rjbs
Re: Module requirements (was: Module::Build and installing in non-standardlocations)
* demerphq [EMAIL PROTECTED] [2006-04-05T10:23:45] On 4/5/06, Ricardo SIGNES [EMAIL PROTECTED] wrote: That said, I don't dispute the point that it can be wildly obnoxious when Something::Trivial requires DBD::MySQL and Data::Dump::Streamer when it could use neither -- or at least rely on AnyDBM and Data::Dumper. It will just meant that fewer people want to use that trivial module because it takes longer to install and is less well-tested by virtue of its dependencies. Interesting example. What module do you mean? I wasnt aware that there were very many modules that have dependencies on DDS. OTOH, if it has to do with proper serilization (of closures, or cyclic data structures) then Data::Dumper just wont cut it. I was making a random example of something that used something it probably didn't need, and that used something that might be emulable by core modules. -- rjbs
Re: Non-Perl TAP implementations
* Andy Lester [EMAIL PROTECTED] [2006-04-16T23:08:26] I'm adding a section to Test::Harness::TAP on non-Perl TAP. http://svn.perl.org/modules/Test-Harness/trunk/lib/Test/Harness/TAP.pod If you know of one, please send me some text to add. It's not really ready to be publicized, and I haven't touched it in a little while, but I'll mention PyTap: http://svn.codesimply.com/projects/pytap I'll get it in a useful state soon. Comments from more regular producers of Python code are welcome. -- rjbs signature.asc Description: Digital signature
Re: Non-Perl TAP implementations
* Ricardo SIGNES [EMAIL PROTECTED] [2006-04-16T23:33:19]
It's not really ready to be publicized, and I haven't touched it in a little
while, but I'll mention PyTap: http://svn.codesimply.com/projects/pytap
I got a request, off-list, for more info, so here is some:
PyTap will, when it's done, provide a simple, assertive
(Test::Builder/More-like) interface for writing tests in Python. It will
output TAP and will include the functionality found in Test::Builder and
Test::More. It will try to make it easy to add more test code (so you can
write your own TAP.StringDiff, for example.
Right now, it's got a fair bit of the basics needed to emulate Test::More, and
I think it's easy to add more stuff -- just like Test::Builder, there's a
singleton that you can get at easily.
I need to better identify and finish implementing the most basic tests. I am
not a Python guru, I just use it from time to time, so my aim may not be true.
I need to write tests for it, which means either relying on Perl for the tester
tester, or writing one in Python.
Here's a sample test, as found in my Subversion:
from TAP.Simple import *
plan(15)
ok(1)
ok(1, everything is OK!)
ok(0, always fails)
is_ok(10, 10, is ten ten?)
is_ok(ok, ok, even ok is ok!)
ok(id(ok),ok is not the null pointer)
ok(True, the Truth will set you ok)
ok(not False, and nothing but the truth)
ok(False, and we'll know if you lie to us)
isa_ok(10, int, 10)
isa_ok('ok', str, some string)
ok(0,zero is true, todo=be more like Ruby!)
ok(None, none is true, skip=not possible in this universe)
eq_ok(not, equal, two strings are not equal);
--
rjbs
signature.asc
Description: Digital signature
Re: Non-Perl TAP implementations (and diag() problems)
* Ovid [EMAIL PROTECTED] [2006-04-19T04:02:31] From a parser standpoint, there's no clean way of distinguishing that from what the test functions are going to output. As a result, I really think that diag and normal test failure information should be marked differently (instead of the /^# / that we see). I strongly agree. This came up when TAP was being named, and also during a previous Harness upgrade. Unfortunately, Harness and Straps discard comments and, as mentioned before, the STDOUT/STDERR makes it hard to associate diagonstic output with tests. How many things rely on which stream the various output methods in Test::Builder use? I know there was trouble last time that things changed, but wasn't that entirely because of the disconnect between Test::Builder and Test::Builder::Tester? Since they're now together, isn't that particular issue solved? There are other things that test test output, like Test::Tester. Will they break? To find out, I downloaded a pristene copy of the latest Test-Simple and Test-Tester and changed Test::Builder to use STDOUT for failure_output. The only test that failed in the whole set was one that checked whether failure_output defaulted to STDERR. I think that a real investigation into the impact of using one stream by default is in order. -- rjbs signature.asc Description: Digital signature
Re: CPANTS is not a game.
* Michael G Schwern [EMAIL PROTECTED] [2006-05-23T12:46:13] So I guess its down to this: pick a goal. Either drop the gaming aspects or drop any remaining pretense that its a measurement of module quality. Since the whole kwalitee thing is pretty flimsy to begin with, I'd go with just making it a distribution improvement game. That's what it seems to do best, what people like to use it for and games are fun! I've always seen it as a distribution quality tool; I just don't think it can really measure module quality in any useful way. I'd vote, were I the voting type, for continued work in that direction. -- rjbs signature.asc Description: Digital signature
Re: Unintended consequences
* Adam Kennedy [EMAIL PROTECTED] [2006-05-27T23:48:43] The questions that are being asked are for the user's benefit. That is NOT being a freeloader. Freeloading is taken something from the user and providing nothing in return. She's providing her free code in return. -- rjbs signature.asc Description: Digital signature
Re: Running individual unit tests with Test::Class???
* Adrian Howard [EMAIL PROTECTED] [2006-05-31T10:55:20]
In short
* Yes running one test method at a time is a sensible things to do.
* No - there currently isn't a simple way of doing this
* Good news - Ovid has submitted a patch to make it easy
* Bad news - I've been too bone idle to apply it
Potentially too clever idea:
# In your test unit class:
sub import {
my ($class, @tests) = @_;
# startup;
$test-$_ for @tests;
# teardown;
}
Because then:
perl -I lib -I t/lib -MTest::Your::Code=basics,extra,weird -e0
This might end up being too clever and too much typing, anyway.
--
rjbs
signature.asc
Description: Digital signature
Re: Proposal Suggestion - Test::Run [was Re: [Israel.pm] Fwd: Call for proposals -- Perl Foundation Grants]
* Shlomi Fish [EMAIL PROTECTED] [2006-07-05T15:28:28] The grant is about Test::Run, which is a fork of Test::Harness that aims to greatly refactor and modularise it. I've already revamped and re-written a lot of code for it, but there's still a lot that needs to be done. [...] Some of the code in Test::Run is derived from the code of Test::Harness, which is GPL+[Artistic 1.0]. Thus, it should be kept this way. However, some code was written from scratch and due to my personal preferences, it was licensed under the MIT X11 Licence[1]. I prefer that all new code will be also MIT X11 licence, and that existing code will retain its original licensing terms. [...] TAP::Harness is a rewrite of Test::Harness by Michael G. Schwern and others. So, you want to have someone (other than you) work on a project that largely seems to duplicate the functionality of existing code. Among the existing code whose functionality is being duplicate is code still under development. The other project is being done with no TPF funding and will be capable of becoming part of the core in future Perls, largely superseding the existing test framework. You want funding for a parallel project that will be not be license-compatible with the core. I don't see how this would be worth TPF's money. It might be a very useful project for people who use the code or for people who want to steal ideas, but I don't see how it fills a really critical void in the language or its toolset. -- rjbs signature.asc Description: Digital signature
Re: TAPx::Parser can't dup STDOUT: bad file descriptor
* Ovid [EMAIL PROTECTED] [2006-07-30T06:19:17] After struggling with this for over an hour, I think I finally found the problem. If I *don't* localize the filehandle, this problem goes away. Apparently, localizing that typeglob caused the underlying descriptor to disappear once the localized typeglob went out of scope. I wondered why the FILE typeglob in Test::Harness::Straps::analyze_file() wasn't localized and now I know. A comment in there might be useful. In cases like these, I've encountered enough grief that now I always use Symbol.pm's gensym. -- rjbs signature.asc Description: Digital signature
Re: prove support for Test::Class?
* Ovid [EMAIL PROTECTED] [2006-08-03T09:20:50] prove t/test_class_tests.t path/to/test/class.pm I want to do that so I can use Test::Harness, but the path is considered an argument to prove and not my Test::Class driver wrapper script. My first reaction would be to use =, like -M does. prove t/test_class.t=t/lib/OvidTests/Major.pm,t/lib/OvidTests/Minor.pm -- rjbs signature.asc Description: Digital signature
Re: Test::WWW::Mechanize - submit form with invalid values
* Gabor Szabo [EMAIL PROTECTED] [2006-08-19T10:40:37] During testing sometimes I would like to submit a form either with values missing from a selection list or with illegal values supplied. I recently released HTML::Form::ForceValue, which lets you do this. I believe this feature should be avaialble in HTML::Form, but this is a crude way to cram it in for testing. I submitted a basic patch to add this feature to HTML::Form. Similarry ocassionally I would like to supply fields and values that are not in the real form. ForceValue does not do this... yet! -- rjbs signature.asc Description: Digital signature
Re: Terrible diagnostic failure
* Ovid [EMAIL PROTECTED] [2006-09-18T13:18:19] Anyone have a Windows box and is willing to test this out for me? I will gladly run tests under WinXP + Strawberry if you tell me where to get it and what you want to know. Drop me an IM or email. -- rjbs
Re: Don't 'rm -fr /' when testing
* Ovid [EMAIL PROTECTED] [2006-09-26T09:19:46]
It would be nice if I could just write 'use My::Test::More' in my
test scripts and have that provide what I need
Side note: yes, it's trivial for me to write an extra module which provide
an environment variable or something similar for this, but I like the idea of
hooking this onto Test::More since we rely on this module for *every* test
script and thus can't forget to include it.
I thought this could be done in a nice simple way by exporting curried methods,
but then I remembered that using methods would chomp prototypes. The attached
code has a few jagged edges (doesn't use exported_to, etc), but I gave up on
polishing them when I realized the prototype issue; maybe B::something can fix
that.
This would have made it possible to subclass Test::More, but maintain the
interface, so you could write:
package Test::Most;
use base qw(Moog);
sub ok {
my ($class, $bool, $comment) = @_;
for (1..10) { $class-SUPER($bool, $comment); }
}
This was probably more a way to waste a couple minutes than a good path
forward, but YMMV.
--
rjbs
package Moog;
use strict;
use warnings;
use Test::More ();
use Sub::Exporter ();
use Sub::Exporter::Util ();
use base qw(Test::Builder::Module);
BEGIN {
no strict 'refs';
for my $sub (@Test::More::EXPORT) {
*{$sub} = sub { shift; {Test::More::$sub}(@_); };
}
Sub::Exporter::setup_exporter({
as = 'import_code',
exports= [
map { $_ = Sub::Exporter::Util::curry_class($_) } @Test::More::EXPORT
],
groups = { default = [':all'] },
});
}
sub import_extra {
my ($class, $things) = @_;
my %ok = map { $_ = 1 } @Test::More::EXPORT;
$class-import_code(
{ into_level = 2, },
grep { $ok{$_} } @$things
);
}
1;
Re: Comment about BAIL_OUT
* Ovid [EMAIL PROTECTED] [2007-01-04T16:34:31] I guess the reason I have never used BAIL_OUT is because if I have a bunch of tests failing, they fail quickly and I don't have to wait for them :) I suppose it's not that big of a deal, but I noticed it this evening and thought I would toss it out there in case anyone has any comments. The problem with some things failing quickly is that you might do some simple tests way up front in, say, 00-am-i-online.t, and then have your lots and lots of tests start failing only after slow setups -- so you can either realize what's happened and interrupt, or go have a coffee. Worse, if your slow setups will need teardown, and your tests don't properly handle interrupts... and if you're running THAT under Devel::Cover to get things really nice and slow? Anyway, I don't use BAIL_OUT in everything, but having it as an option for really big, obnoxious test suites is lovely. -- rjbs
Re: ExtUtils::MakeMaker, and t/ sub-directories
* Adam Kennedy [EMAIL PROTECTED] [2007-02-08T19:17:39] Yep, although M::I has some capacity to add a but of extra magic if you could come up with a workaround (like having File::Find locate them all and provide a complete list of TESTS). I do this in http://search.cpan.org/src/RJBS/Rubric-0.142/Makefile.PL The annoyance for me is that EUMM's normal behavior means that if I add .t files to ./t/, they get run the next time I make test, because they're globbed then. With the M::I trick, they aren't run unless I regenerate the Makefile. I've never been annoyed enough by this to do anything about it, though. -- rjbs
Re: Custom extensions to META.yml
* David Golden [EMAIL PROTECTED] [2007-02-28T22:39:01] Is there a de facto standard for custom extensions to META.yml? (I didn't see one in the spec.) An example might be fields beginning with a capital letter or X-foo style extensions. E.g. Why not: extensions: CPAN::Reporter: cc_author: 0 (PLEASE let's avoid 'yes' and 'no' as booleans.) If all the extensions are under the module (or, hey, dist) that uses them, then the X-nonsense is avoided. -- rjbs
Re: Custom extensions to META.yml
* brian d foy [EMAIL PROTECTED] [2007-03-03T13:31:15]
In article [EMAIL PROTECTED], Ricardo
SIGNES [EMAIL PROTECTED] wrote:
extensions:
CPAN::Reporter:
cc_author: 0
I think in some cases this might work, but I can imagine options that
I'd want, such as cc_author, that future modules or tools might use. I
won't know of them a priori, but it's likely that my answer to them
will be the same.
If you don't know that CPAN::Reporter uses extensions/CPAN::Reporter/cc_author,
you won't know that it uses cc_author, either. While you might implement the
same solution, I think it's less likely that you'll settle on the same name,
and thus benefit from happy accidents.
...but maybe the benefit of the middle-level namespace there isn't so great,
and extensions/cc_author is good enough. I suppose that the chance of
collision of extensions is fairly low. This could be helped by having CPANTS
(or something) produce a listing of extensions actually used in the CPAN's
code, so you could see what else uses what. Collisions would be reduced to
times when modules introducing new extensions are uploaded at the same time or
when one author is careless.
If a feature becomes useful enough to be part of the core spec, then tools need
to know to look for -{extension}-{yourface} or for -{yourface}, which is
also annoying.
A solution might be to use purpose-named namespaces owned by other individuals
to manage common configuration, including a generic per-tool namespace.
test_reporting:
also_cc: [ [EMAIL PROTECTED] ]
indexer:
index_all: 1
extensions:
Module::Started: { built_by: '1.602' }
This lets people who will probably have common needs work together (hopefully)
without the META.yml spec maintainer worry about much besides does this
deserve a namespace and lets everyone use a namespace for their own little
tools.
I don't know, I'm just throwing the idea out there. These things are always a
pain.
--
rjbs
Re: a safer way to use no_plan?
* Eric Wilhelm [EMAIL PROTECTED] [2007-03-03T21:59:32]
In fact, so much eyewringing that I've taken to this lately:
use inc::testplan(0,
+ 3 # use
+ 199 # those others
);
What is that ... for?
I often do this:
use Test::More;
my @test_data = load_data('t/whatever.dat');
plan tests = 3 + @test_data * 3;
I'm not sure if your snippet is doing something else.
--
rjbs
Re: Custom extensions to META.yml
* brian d foy [EMAIL PROTECTED] [2007-03-04T12:09:26] I'm not talking about the particular field name, but the idea that I'd want to say in META.yml Don't send me mail, or whatever setting I want. Instead of having to disable (or enable) CC for every new tool, I'd want a setting that new tools could look at without me having to change the META.yml in all of my distributions then re-uploading them all. So, for some subset of META.yml settings, you could consult the module's author settings, found at (say) $MIRROR/authors/.../RJBS/METAMETA.yml That would be on your local mirror, be it minicpan or CPAN-over-HTTP. Something like that? I feel a potentially irrational sense of dread. -- rjbs
Re: The most important TAP tasks.
* Michael G Schwern [EMAIL PROTECTED] [2007-03-09T18:17:57]
*) TAP diagnostic format
http://perl-qa.yi.org/index.php/TAP_diagnostic_syntax
There is no way to output diagnostics in TAP. The stuff Test::More spits
out to STDERR are unparsable comments indented for humans. Its not TAP.
This is the single biggest TAP deficiency.
Diagnostics need not be only for failures. There's nothing which says you
can't output them for passes.
So, under what circumstances do we display the diagnostics data regarding test
line/file? Something like either:
* if other diagnostics were given, we add these and display it
* if the test fails, we add these and display it
Assuming that diagnostics are passed as a hashref, and someone passes line or
file entries (or whatever the standard name is), presumably we croak. Or, we
make the standard name obnoxious enough that by using it you are clearly
stating that you know you're about to shoot your foot. ('TAP/testline')
--
rjbs
Re: .t files as specs
* James E Keenan [EMAIL PROTECTED] [2007-06-19T21:33:11] But, guess what? To the extent that I've been able to determine my own approach to development, I have increasingly moved in the direction of doing step 1 first: documentation-driven development. Or, perhaps more precisely, specification-driven development. I advocate another form of this: synopsis-driven development. I write a five to ten line hypothetical use case (or longer, for modules whose most basic use would require it), and then say, Wouldn't it be totally awesome if that /worked/? If I still feel that way in a few minutes/hours/days/years, I turn it into a test (by writing all the boring filler that needs to be written around it) and then make it pass. The bulk of the documentation can come later. -- rjbs
Re: podcoverage vs pragmatism
* Eric Wilhelm [EMAIL PROTECTED] [2007-04-09T16:31:31] # from Ricardo SIGNES # on Monday 09 April 2007 05:10 am: I need to finish/test/release my PC subclass that looks at @PKG::POD_COVERAGE_TRUSTME. I saw that in rt, but I really think pod is the place for it. Why clutter the code with variables which are only used by a tool that reads pod? Isn't that what '=for' is for? Has anyone written a means to embed this sort of thing in POD? -- rjbs
new test module, Test::BinaryData
http://search.cpan.org/dist/Test-BinaryData/lib/Test/BinaryData.pm I won't rehash the whole documentation here, but the gist is that I really hate getting test reports that say: not ok 1 # Failed test in demo.t at line 8. # got: 'foo # bar # ' # expected: 'foo # bar # ' So this can outuput: not ok 2 # Failed test in demo.t at line 10. # got (hex)got expect (hex) expect # 666f6f0a6261720a foo.bar. ! 666f6f0d0a6261720d0a foo..bar.. There are some tweakable options. A future version will probably make the whole chunk-diff-displayer pluggable, but this is the one that *I* wanted *now*. -- rjbs
Re: new test module, Test::BinaryData
* Michael Peters [EMAIL PROTECTED] [2007-07-21T09:07:42] Ricardo SIGNES wrote: http://search.cpan.org/dist/Test-BinaryData/lib/Test/BinaryData.pm Did you know that Test::LongString (despite the name) can handle binary information too? Probably not as detailed as your diagnostics, but it works fairly well for seeing the differences in binary stuff. Yeah, I knew that, and have used it for this in the past... but my brain, I think, has become clouded by extensive use of perl -I lib t/file.t | xxd for this sort of testing, which is pretty silly, I guess. What I should really do is make a little wrapper so that I can toggle between is and is_string and is_binary automatically based on environment variables or compared data. -- rjbs
Re: Parsing TAP into TAP
* Andy Armstrong [EMAIL PROTECTED] [2007-12-10T11:52:33] On 10 Dec 2007, at 16:49, Ovid wrote: Seems Ricardo Signes likes this idea, too: http://use.perl.org/~Ovid/journal/35076 Who? :) :'( I like this idea so much that if you point me at a repo and failing tests, I will try to be a little helpful. :) -- rjbs
Re: phalanx - Getting involved
* Miguel Pignatelli [EMAIL PROTECTED] [2008-01-08T10:10:00] Is there a possibility of getting involved in the phalanx project?. I'm a regular Perl user trying to improve his knowledge and being able to help the community. Absolutely. A very simple way to do this is to pick a distribution from the Phalanx 100 and help write more tests for it. http://qa.perl.org/phalanx/100/ ...assuming, of course, that you talk to the maintainer and he or she is interested. I can tell you that the maintainer of Mail-Audit would be, and you could get involved by joining the PEP list http://emailproject.perl.org/wiki/Getting_Involved to get commits and start writing tests! -- rjbs
author tests with Module::Install::AuthorTests
I thought I'd relay my journal post here:
http://use.perl.org/~rjbs/journal/35746
Basically, it lets you say author_tests('xt') in your Makefile.PL (using
Module::Install) to have a directory (or directory tree) of tests run only by
the module's authors.
--
rjbs
Re: author tests with Module::Install::AuthorTests
* Eric Wilhelm [EMAIL PROTECTED] [2008-02-25T12:53:03]
# from Ricardo SIGNES
# on Monday 25 February 2008 04:59:
Basically, it lets you say author_tests('xt') in your Makefile.PL
(using Module::Install) to have a directory (or directory tree) of
tests run only by the module's authors.
Now we only have to determine 'author'ness. I was thinking something
along the lines of a capabilities/willingness registry for testers
would be the way to handle it. That doesn't address the on/off switch
for extra so much as which bits to test (do I have a chroot, ramdisk,
etc. available.)
But: I dislike the 'authortests' naming. These are simply extra
tests, so whether you're the author only matters in terms of your 'make
dist' default -- that is: a volunteer tester should be able to run as
many of the extra tests as desired, while an author must run them all.
The name AuthorTests makes sense and should not be objectionable given the
mechanism used. It uses the is the current directory an author's development
workspace, or a dist from the CPAN? mechanism built into Module::Install. I
know that there was talk about PERL_TESTER_RUN_EXTRA_TESTS=1 and so on, but I
don't care about that. In fact, I don't ever want anyone who isn't hacking on
the module from a git checkout to run those tests, so it works very well for
me. Obviously, YMMV.
--
rjbs
ExtUtils::FakeMaker 0.001 uploaded
Lately, we're doing a lot of work on some for-now internal tools that operate on CPAN archives as a whole. They're sort of like CPAN::Mini and CPAN::Mini::Inject, writ large. Doing stuff like unpacking an entire CPAN mirror to analyse its contents and prereqs is a real drag, though. We also need a lot of specific test cases, like Do we get things right when the file is Foo-Bar-0.01.tar.gz but META.yml says the version 0.001, but no module actually has either of those versions? (What is the right behavior in this scenario? I'm afraid I can't disclose that information.) ExtUtils::FakeMaker builds distributions out of various kinds of input. The only sort of useful input right now is a META.yml file. You describe a dist in META.yml format and it builds it for you, using a lot of assumptions if needed. Here's an example META.yml used in the test cases: --- #YAML:1.0 name: Mostly-Auto abstract: nothing to see here This produces a dist with a Makefile.PL (using EUMM), a manifest, a useless test, files in lib, etc. It also has its own META.yml which is NOT the same as the input, but rather a complete file. Right now this code is pretty slapdash, and requires 5.10. In the future, I might make it work on 5.8 and I will certainly add way, way more test cases. That's all! I hope someone else finds it useful. -- rjbs
Module-Faker and CPAN-Faker (RIP EU::FakeMaker)
First, ExtUtils::FakeMaker is now Module::Faker. Schwern suggested that ExtUtils should no longer be used, and gave excellent reasoning, summed up here: http://www.perlfoundation.org/perl5/index.cgi?extutils So, that's done. There haven't been any changes worth mentioning, yet, in its documentation, tests, or featureset. What's much more important, as far as I'm concerned, is the upload of CPAN::Faker. Given a set of META.yml files like this: ./Fake-Distro.yaml ./Fake-Another.yaml ./Yet-Another-Bogus-1.yaml ./Yet-Another-Bogus-2.yaml ./Yet-Another-Bogus-3.yaml ./Quirk-WonkyVersion.yaml CPAN::Faker will build a complete-enough-for-CPAN.pm CPAN instance, containing: ./modules/02packages.details.txt.gz ./modules/03modlist.txt.gz ./authors/id/L/LO/LOCAL/*.tar.gz You're not really limited to LOCAL yet, but I need to make it clearer how that will work... well, it all needs more docs. Anyway, this will make it really easy to build a small CPAN-alike with known quirky dists, authors, structure, and so on, excellent for testing all kinds of things. I will probably use it for testing CPAN::Mini sometime soon, and hopefully some other things also. Next, I think I will add a bit more documentation and more tests. If you think this tool will be useful, please let me know if you have feature requests while I can still refactor things in all their undocumented glory. Oh, and another totally sweet future use for Module::Faker? The basis for a much more powerful engine for Module::Starter! -- rjbs
Re: Hackathon logistics
* Michael G Schwern [EMAIL PROTECTED] [2008-03-26T04:15:47] Not trying to cause repo-wars, but I think I'll need to use git actively for something before I really start to grok it. I had the same thoughts. My concern is that we'll be spending time futzing with git rather than hacking on QA stuff. We all know SVN/SVK and can stand to use it for a little while longer. I agree on all fronts. I will gladly give a quick demo of git stuffs before we get started, but unless everyone working on Thing X really wants to use git, I bet the average fluency in svn is much much higher. -- rjbs
new module, Pod::Coverage::TrustPod
After having this task languish in my todo for years, at least, I have finally
reduced my goal to the important 90% and applied some JFDI.
Pod::Coverage::TrustPod acts like Pod::Coverage::CountParents, but accepts
non-whitespace lines inside Pod::Coverage POD targets as trustme
instructions.
In otherwords, given this module:
package Foo;
=begin Pod::Coverage
bar
=end Pod::Coverage
=head2 baz
This routine does stuff.
=cut
sub foo { ... }
sub bar { ... }
sub baz { ... }
sub _xyzzy { ... }
Pod::Coverage::TrustPod will return 2/3 coverage. _xyzzy is private, so
uncounted. There are docs for baz, bar is trusted (because it's in the
Pod::Coverage =begin block) and there are no docs for foo.
The basic use case is that now my xt/pod-coverage can now read:
all_pod_coverage_ok({ coverage_class = 'Pod::Coverage::TrustPod' });
and the information on what code is written but undocumented can be put where
it belongs -- near the code that is written but undocumented. This will help
eliminate both (A) marking a symbol trustme/also_private in all of a dist's
packages so that I can use all_pod_coverage_ok and (B) having to write a
bunch of pod_coverage_ok lines to avoid problem A.
--
rjbs
Re: Friday afternoon in Oslo
* David Golden [EMAIL PROTECTED] [2008-03-31T10:07:57] On Mon, Mar 31, 2008 at 9:36 AM, Andy Armstrong [EMAIL PROTECTED] wrote: I'm getting in on Thursday night - so I'll be around. Just planning to wander around taking pictures but could be persuaded by something more structured :) Wandering around taking pictures is cool, too. I've got a guide book that has a 1-day must see section, so I planned to go look at a couple of those, perhaps. Yup, I arrive at OSL, Friday morning at 9:30, along with Michael Peters. Assuming we can find the hotel, I (at least) am all for forming a touring company. Since I have NFI what the cell situation will be, shall we establish a time to try and meet up and depart the Anker? -- rjbs
Re: User Supplied YAML Diagnostic Keys: Descriptive Version
* Michael G Schwern [EMAIL PROTECTED] [2008-04-11T07:01:19] Here's the descriptive way to specify how the diagnostic keys work. 1) We reserve every key which begins with a lower case letter 2) We say nothing about anything else 3) All keys are optional I thought this had been the resolution. I hope it *is* the formalized resolution. It is simple and easy and leaves plenty of room on both sides of the velvet rope. If isLower isn't clear enough, I think you're right: [a-z] in Latin-1 is fine. I think it's clear enough, from where I'm sitting, but I only speak American. -- rjbs
Re: New CPANTS metrics
* brian d foy [EMAIL PROTECTED] [2008-06-10T12:27:29] I'd like to see the metircs that only talk about the quality of the distribution, and leave everything else alone. If it's something I can fix by doing something to the distribution itself, measure it. If it's anything else, leave it out. :) Given that CPANTS has been discussed as a tool to help authors write better dists, I think this is a very, very good suggestion. Gathering other information is great. I'd like to know if there Debian is packaging my code. It just isn't as much about your code seems to be well-produced. -- rjbs
AndyL reports HARNESS_TIMER broken
His email is bustified. http://rafb.net/p/vD8hRk81.html FWIW, I get the same results. -- rjbs
Re: Have/Want
* David E. Wheeler [EMAIL PROTECTED] [2008-08-07T00:31:27] On Aug 6, 2008, at 20:12, Michael G Schwern wrote: So I encourage folks to use have and want in the future. I'll be using them in Test::Builder2. Good call. Change committed to pgtap and Test.Builder. and PyTAP, which might actually get overhauled in the next month. -- rjbs
Re: Have/Want
On Aug 6, 2008, at 20:12, Michael G Schwern wrote: So I encourage folks to use have and want in the future. I'll be using them in Test::Builder2. Test.php updated and released. -- rjbs
Re: JSON TAP Diagnostics?
* Ovid [EMAIL PROTECTED] [2008-08-18T06:50:00]
JSON is fairly well implemented and new implementations are trivial. This is
not true for YAML. Trying to define a minimum standard of YAML for extended
TAP is a quagmire. With JSON, we can punt and just point to a fairly
well-established JSON spec.
I would absolutely love to use JSON instead of YAML. There are no good,
well-documented implementations of YAML for Perl, period. YAML::Tiny has less
of a reliable spec than YAML.
I've been doing a lot of work lately with both YAML and JSON, and in every case
dealing with JSON is very easy and dealing with YAML is very frustrating. I
love the idea of YAML, but in practice, beyond writing simple config files, it
ends up really driving me nuts.
It is slightly less readable, and I think that bears noting:
YAML:
neck: 17
chest: 38
feet: { left: 9 1/2, right: 10 }
JSON
{
neck: 17,
chest: 38,
feet: { left: 9 1/2, right: 10 }
}
...but I don't care, because I trust that the presentation layer can do
something nice, if I come to hate just viewing the TAP stream. It's still
readable enough.
JSON makes numbers/strings clearly distinct, which is very nice when in
non-Perl lang. It also makes booleans clear. Seriously, I will gladly fight
this fight. I thought it was lost in Oslo, but if Ovid is willing to bring it
up again, bring it on.
To be honest, this does not block the IETF proposal at all because we'd
merely be describing the areas of agreement which need to be reached.
However, we do need to discuss this and going from I has an implementation
to I has a spec is rather like going from I has Perl to I has PPI if
we stick to YAML.
This also really helps us keep the spec simple, as we can say ...emitted as
JSON data [RFC 4627].
--
rjbs
Re: JSON TAP Diagnostics?
* David Golden [EMAIL PROTECTED] [2008-08-18T09:27:57] What's the latest consensus on the best pure-perl JSON module? And ditto for JSON via XS? JSON and JSON::XS, most likely. Certainly JSON::XS. -- rjbs
Re: JSON TAP Diagnostics?
* Ovid [EMAIL PROTECTED] [2008-08-18T11:17:25] Oh, definitely agreed. I cannot assert that non-Perl implementations of JSON are any better, but JSON is simple enough that I'm pretty damned sure they are. However, YAML is so problematic that I *CAN* state that non-Perl versions are often as problematic as Perl's. I can. There are basically three good implementations of YAML. PyYAML, libYAML, and the Haskell YAML. libyaml is kinda sorta available in some non-C langauges with bindings. In Perl, you can use it with default settings, with none of the features (beyond prettiness) that make YAML more useful than JSON. There are many, many useful implementations of JSON in other languages. Some of them have edgey bugs, but those are easy to fix because the spec is simple. For example, PHP's JSON does not understand 0e0 as the number 0. Meanwhile, YAML 1.2 continues to ambiguously imply that 1:23:45.6 is a floating-point sexagesimal number. Which are more people going to be able to implement correctly, JSON or YAML? -- rjbs
Re: JSON TAP Diagnostics?
* Michael G Schwern [EMAIL PROTECTED] [2008-08-18T12:26:54]
YAML types can be little more than local tags which only have meaning to that
particular document.
name: !customer Evil Business Guy Made Of Butter
Yeah, that's neat and everything, but there aren't any Perl implementations
that let you USE that feature.
When last I tried, I had to use YAML::Syck, which caused them to be blessed
into 'customer' and then rewrite and rebless them with a visitor.
libyaml offers this feature, but YAML::LibYAML does not, and I don't have much
reason to believe it's coming in the near? future.
A TAP producer need not use a full YAML library, they can happily output JSON
or even hand roll it. Only a TAP parser need concern itself with having a
full YAML library. One of the complexity principles of TAP is to push
complexity down from the human and the TAP producer into the TAP parser.
There need only be a handful of parsers and there need not be one for each
programming language.
...but there IS no full YAML library in Perl. YAML::LibYAML still blesses
things into a package named after their tag. It also doesn't properly parse
JSON...
my $json = q[ { this-is-null: null } ];
use Data::Dumper;
print Dumper(Load($json));
$VAR1 = {
'this-is-null' = 'null'
};
...so, no. You can't just use a JSON emitter until someone writes or bugfixes
a Perl YAML parser.
YAML has lots of cool features. One of its chief authors is a prolific Perl
author. We still have no high-quality implementation of YAML for Perl. What
does that tell us about the future of YAML in other languages?
JSON is a subset of YAML::Tiny, would you have had those problems with JSON?
What were the problems?
No, YAML::Tiny is a DIFFERENT subset of YAML than JSON is.
use YAML::Tiny ();
print Dumper(YAML::Tiny-read_string($json));
$VAR1 = undef;
Also failing: true, false.
Remember, YAML::Tiny can be extended. JSON can not.
That's a selling point for JSON, in my estimation, and a selling point for
begin document; type={json,jpeg,zmachine,whatever}. Just mandate that the
only mandatory type for a compliant parser is JSON.
--
rjbs
Re: Should MANIFEST go in the repository?
* Jeffrey Thalhammer [EMAIL PROTECTED] [2008-08-20T12:42:49] But there is some debate about whether the MANIFEST and other metafiles should be put in the source code repository. My gut feeling is that MANIFEST is a generated file, therefore it does not go in the repository. Instead, the MANIFEST.SKIP file should be carefully crafted to ensure that the MANIFEST is correct. However, this raises a question: How do you ensure that *unexpected* cruft files don't end up in the MANIFEST? I don't store MANIFEST, I store MANIFEST.SKIP. In fact, on my latest dists, I only include the MANIFEST as a couresty to stupid software that expects it. I don't see any value in having a MANIFEST file in the dist, let alone in my repo. I have other tools for managing what ends up in the tarball; I configure those instead of MANIFEST, which I find to be a fairly anemic tool. -- rjbs
Re: JSON TAP Diagnostics?
* chromatic [EMAIL PROTECTED] [2008-08-20T13:59:14]
Aren't these two separate concerns, human versus machine readability? The
latter rarely respects ambiguity.
Yes.
Right now, there seem to be two pro-YAML arguments.
(1) It's easier to for humans read.
Sure. I will admit that. It is easier for humans to read in probably every
likely case but one. It's harder for machines to read, though, so that cost
has to be weighed.
Also, given how few languages have reliable, conformant YAML emitters, I
imagine that it will often be easier to use a JSON emitter. Even if your
language has a YAML emitter, the JSON library will probably be in heavier use,
you're more likely to be familiar with it, etc. I admit that this is not a
strong argument, but I think it's pretty likely to be the case for a good long
time yet. (Even Perl doesn't have much by way of YAML libraries.) The one
case in which YAML is definitely not easier to read than JSON is when it is
formatted in the subset of YAML that is JSON.
So, how often, in non-Perl TAP emitters, will you see pretty YAML instead of
JSONic YAML? I predict: plenty.
So, yes, languages that can emit pretty YAML can do so, to make the protocol
stream easier for humans to read. That will, of course, limit the number of
consumers than can parse it.
So: how often is YAML going to realistically give this benefit outside of
Test::Builder and TAP::Harness?
(2) YAML is better suited for complex serialization than JSON.
This is pretty clearly just true. The ability to tag structures as
representing data more complex than YAML's native kinds allow, or just has
having greater significance than some raw data, is really great.
Your language won't need (say) pickle if it has YAML and a schema for mapping
complex native types to YAML. Many languages have their own serialization
system, though; how many will implement another one that stores in YAML,
especially in languages (like Perl) where YAML support for making use of tags
is paltry, at best.
It would be almost as easy, if not as easy or easier, to instead make your
language serialize to JSON. After all, these two things could easily mean the
same thing:
---
account: !yourcorp.biz!account 1234
..or...
{
datatype: yourcorp.biz/account,
payload : 1234
}
So, definitely it is cool that there is a build in way of saying datatype and
payload in YAML, and I'm glad it's so compact. How often will people be able
to reap this benefit if YAML is chosen over JSON?
I am really, really worried by statements like we'll just keep pushing on YAML
implementors to get it implemented more places and better so that we can rely
on it for TAP.
--
rjbs
Re: Test::Harness Output Change
* Ovid [EMAIL PROTECTED] [2008-08-21T05:36:11] Both of us found this much cleaner. However, this might have unexpected consequences. It also highlights the issue of Test::Harness's long-standing practice of stripping the .t extension from filenames. Why? If we want other extensions, stripping them is probably bad. I'd definitely like to always see the .t, as I'm now working on more things that have multiple extensioned tests. I'd rather not need to know whether or not the item is a filename or munged filename based on some logic. It wasn't clear to me whether this is now the case, from AndyA's post, so I'm just chiming in. -- rjbs
Re: JSON TAP Diagnostics?
* Eric Wilhelm [EMAIL PROTECTED] [2008-08-21T12:46:59] # from Ovid 1. YAML is prettier. 2. JSON, unlike YAML, is stable. Let's not forget that the debated requirement for diagnostics is that the generators and consumers speak the same language Does it have to be just one? Now and forever? Some people want the spec to say that diagnostics (not free-form additional data blocks) are always readable by any TAP consumer, which means they need a declared format. It needs to be at least one declared format. Ovid (and I) would like it to be JSON, pending any better idea (that we agree is better). Schwern would like it to be YAML (a superset of JSON), with the phrasing consumers MUST understand JSON and SHOULD understand YAML. Is the specification for either of these going to be part of the TAP spec? It will probably have a reference to the relied-upon spec. JSON has an RFC. YAML has published, versioned specifications. -- rjbs
Re: Ignoring Non-Failures
* David E. Wheeler [EMAIL PROTECTED] [2008-09-03T13:27:08] http://cpantesters.perl.org/author/DWHEELER.rss Now that there's a new maintainer, I should send another email... this file, for me is so large (6,680,062 bytes) that my RSS reader times out trying to retrieve it. Ugh. -- rjbs
Re: App::Prove::History
* Andy Armstrong [EMAIL PROTECTED] [2008-09-19T05:24:40] I'd lean towards epoch for easier comparison on SQLite. I tend to use epoch in SQLite, too, but it's worth nothing that iso8601 is much easier for humans, and SQLite can compare either one pretty well, as 8601 strings are sortable. The benefits show up with things like date math, which is something I end up needed only rarely (since you can compute the right date string to put in your query, in Perl). -- rjbs
Re: Public Humiliation and Kwalitee
* Salve J Nilsen [EMAIL PROTECTED] [2008-10-28T14:54:07] David Cantrell said: On Mon, Oct 27, 2008 at 01:40:03PM +0100, Salve J Nilsen wrote: I think _some_ kind of shaming should be allowed. Carrots are good, but sticks work too when applied in a respectable fashion. They might, but a hall of shame ain't respectable. If I were on the list, then it would just make me think cpants is run by a bunch of cunts, so i'll just ignore them. What other list name (that still explains the purpose of the list) would you prefer then? Get rid of the hall of shame. Change the pagination on http://cpants.perl.org/highscores/many to look like standard pagination: | 1 2 3 ... 100 101 202 | People who want to see the lowest scores can click 202. People who want to see what they're doing wrong don't want to find out by clicking, Hall of Shame to see if they're there. They want to click the See my report page and see Here are your green and red boxes. Maybe it will say, This puts you at rank 102, the 33rd percentile. The hall of shame does not serve any productive purpose that the above would not do better. I imagine that hall of shame immediately sprang to mind after a hall of fame was made and seemed like a good idea, given the game metaphor. I just think it's not a great idea, all things told. -- rjbs
test harness presentation layer plugins
I'm mostly sending this email because I had an idea, and it's late, and I don't
want to forget.
Today it occured to me that many Test:: extensions are more about better
diagnostic output than better test comparison.
This stinks:
want: foo bar
have: foo bar
So you write some plugin that does the same old comparison but shows:
want: foo\N{SPACE}bar
have: foo\N{KLINGON MOMENT OF SILENCE}bar
What if display types could be provided informationally in the TAP stream. The
stream could include:
want: foo bar
have: foo bar
presentation: Some::Plugin
With the plugin installed, the presentation layer could fix the output to be
useful. Without it, you still have the data displayed and could presumably
re-display the TAP stream later with the presenter plugin in place.
I am not sure whether this is a good idea, or already well-established as a way
to deal with this annoyance. Good night.
--
rjbs
Re: test harness presentation layer plugins
* Michael G Schwern schw...@pobox.com [2009-03-31T18:22:50] What if display types could be provided informationally in the TAP stream. The stream could include: want: foo bar have: foo bar presentation: Some::Plugin With the plugin installed, the presentation layer could fix the output to be useful. Without it, you still have the data displayed and could presumably re-display the TAP stream later with the presenter plugin in place. Your annoyance is valid, but you've reversed the presentation responsibility. The test should not be dictating presentation. It isn't dictating. It's suggesting a useful default. Until cpan testers are submitting TAP harness, which would allow the author to redisplay the TAP, this would lower the prereqs for installing a distribution (you don't need a presentation-like test plugin to test the module). The whole point of the structured diagnostics is to allow the user of the test to decide how it's to be displayed. You should simply tell your TAP reader how you want diagnostics formatted. For a textual reader you have to do this before the test is run, but imagine a GUI reader where you can use a contextual menu to change the display of a single assertion after the fact. I understand that -- but we don't have those yet, and even if we did, we're not getting the real TAP back, yet, from smokers. -- rjbs
Re: Nested TAP
* Ovid publiustemp-perl...@yahoo.com [2009-04-06T04:35:41] Wondering if anyone's played with nested TAP yet and has any comments/requests/questions? Lot's of people have asked for it, so it would be good to make sure we've got it right before it's pushed out the door. I've given it a bit of a look-over, and I think it's fine. I'm probably not the most likely to want crazy things out of it, but really all I've wanted is a way to say now start a nested group. This gets me that. I thought it might be nice to give the group a description, but really just diagging a description should be good enough for me. I can wait for TAP 15 with preludes, envelopes, and the SWAK marker. -- rjbs
Re: Nested TAP
* Ovid publiustemp-perl...@yahoo.com [2009-04-06T10:52:33]
I thought it might be nice to give the group a description, but really just
diagging a description should be good enough for me. I can wait for TAP 15
with preludes, envelopes, and the SWAK marker.
Actually, you do a get description:
plan tests = 2;
ok 1, 'some test';
subtest this is a description = sub {
How did I miss that?? Thanks!
The summary test name is whatever name you give the test and comes out in
that trailing line. Does that cover what you're needing?
Yes, although I might still use diag. After all:
ok 1 - just did x
1..2
ok 1 - foo
ok 2 - bar
ok 2 - tested foo and bar
Is clear, but when that subtest is 1..50, it'd be more helpful when reading to
see:
ok 1 - just did x
1..50
# here begin the tests of the blort subsystem
ok 1 - foo
ok 2 - bar
...
ok 50 - metaquuxbaz
ok 2 - tested foo and bar
--
rjbs
Re: Calling All Test:: Authors
* Ovid publiustemp-perl...@yahoo.com [2009-06-30T10:21:24]
The latest developer release of Test::More allows subtests. Subtests are
great in that they solve a lot of problems in advanced Perl testing, but they
have required a change in Test::Builder. Previously you could do stuff like
this:
I updated my Test:: libraries to Test::Builder-new in their test routines,
instead, as that's what I thought the original wisdom was. Is that still
okay? (I did not add subtest-specific tests.)
That is, I turned:
my $TEST = Test::Builder-new;
sub is_baloney { $TEST-ok('delicious') }
into:
sub is_baloney {
my $TEST = Test::Builder-new;
$TEST-ok('delicious');
}
--
rjbs
Re: Fluent tests?
* David Golden xda...@gmail.com [2009-06-30T14:46:47]
Well, if you're doing interface design, one of the first things that
comes to mind is that the name of the test should come first, not
last.
...I basically liked everything you had to say here:
test label goes here = is( $have, $want ) = diag $diag;
test a subtest label = nest { [ ... } ]
test label = is( $have, $want ) = setup \setup = teardown
\teardown = diag \diag = note \note;
So, test is a fixture that expects to be handed:
* an assertion, required
* optionally a bunch of other components
... and it knows how to assemble components and execute them.
I really like this. You can add more known components as time goes on. You
can make new fixtures that do things differently. You could import a foo-style
one here and a bar-style one there, each called test, or you could mix and
match by renaming them:
(foo.t)
use Tester::Fooish;
test ...;
(bar.t)
use Tester::Barish;
test ...;
(baz.t)
use Tester::Fooish test = { -as = 'test_foo' };
use Tester::Barish test = { -as = 'test_bar' };
test_foo ...;
test_bar ...;
...so there's not much penalty for getting the specifics wrong to start with.
Since components just perform some known role, Fooish and Barish can both
expect a Tester::Diagnostic, but only Fooish wants a Tester::Metafixation. And
so on.
I am excited by this idea.
--
rjbs
Re: standard for internal-only tests?
* Jonathan Swartz swa...@pobox.com [2009-07-31T14:57:04] Justin Devuyst kindly pointed me to Module::Install::AuthorTests which appears to have the desired behavior if one is using Module::Install (which I happen to be). Thanks again. ...as long as you're looking at AuthorTests, have a look at its successor, ExtraTests. http://search.cpan.org/~rjbs/Module-Install-ExtraTests-0.006/lib/Module/Install/ExtraTests.pm It's better. -- rjbs
Re: camels
* Pedro Figueiredo m...@pedrofigueiredo.org [2010-01-03T07:06:19] On Sun, Jan 3, 2010 at 11:40 AM, chromatic chroma...@wgz.org wrote: If the design of perl.org had been up to me, I'd have spent much more time promoting the Perl brand instead of the proprietary brand of a privately held corporation. For better or for worse, the Perl brand *is* the camel. Either get O'Reilly to give up their copyright on it, or live dangerously. It would be a shocking display of benevolence on the part of O'Reilly to give up the camel. And... live dangerously? Do you mean: piss off the publisher of many useful Perl books, opening ourselves to lawsuits and ostracism? That's not a good plan. -- rjbs
Re: camels
* Shawn H Corey shawnhco...@gmail.com [2010-01-03T07:22:01] Leo Lapworth wrote: Maybe someone could get a grant to hire someone/a company with design skills to come up with a better logo than the onion? I always thought that something to do with pearls would be nice. The problems with pearls include: (a) promoting mispeling Perl as Pearl and (b) a pearl reduces, in its simplest depiction, to a circle. It's not very visually distinctive. An onion can be pretty pared down before you lose sight of what it is. -- rjbs
Re: Test::Differences and utf8
* Michael Peters mpet...@plusthree.com [2010-02-02T11:13:26] Maybe have the option to UTF8 stdout/stderr and just send the characters and let the display handle it. But I don't think it should be the default. Unicode is tricky and lots of unicode characters can be combined in ways that look identical to humans when displayed, but are not the same at all when the bytes are shown. I've been noodling around on a test routine that will smartly decide how to display the diagnostics between standard is, eq_or_diff, or using Test::BinaryData. This is a pretty common itch, I think. Suggested heuristics welcome. -- rjbs
writing tests in an OO paradigm with Test::Routine
I've recently uploaded my new Moose-based system for writing reusable test behavior. I'm very happy with it so far and hope that it can be useful to others, too. I have written about the system and how it works, with links to further material, here: http://rjbs.manxome.org/rubric/entry/1858 -- rjbs
Re: QA hackathon 2011: date poll
* Lars Dɪᴇᴄᴋᴏᴡ 迪拉斯 da...@cpan.org [2011-01-21T15:40:37] this year's prospective date for the QA hackathon in Amsterdam is 16-18 April. Please reply until Monday morning European time whether this is suitable for you, or you would rather have a different date. Those dates are great for me. My only hope would be that we can avoid Apr 8, since I am likely to have difficult-to-shirk plans. Apart from that, I am extremely excited and hopeful that I can make it. -- rjbs
Re: Vague Testing
* Ovid publiustemp-perl...@yahoo.com [2011-02-17T07:43:16] It would be nice if this was a custom comparator for Test::Deep, then you would be apply the almost to lists of arbitrarily complex items and also conduct that test at any level of the data structure (including nesting if you feeling really fruity), Ooh, I would love to see that! After feedback from folks on that blog entry, I'm sure something could be whipped up. I'd be happy to merge such a comparitor into Test::Deep, which needs a next release soon, anyway. ;) -- rjbs
Re: QA hackathon 2012
* David Golden xda...@gmail.com [2011-08-22T22:23:42] On Mon, Aug 22, 2011 at 7:07 PM, Lars Dɪᴇᴄᴋᴏᴡ 迪拉斯 da...@cpan.org wrote: To the usual suspects on this list, especially the ones who could not attend the last times: when do you have time around April? (Traditionally, the QA hackathon is from Saturday to Monday and avoids Easter.) My daughter turns 1 on March 22, but I would expect to be free the weekend that includes April 1. I would not be free the following weekend, which is Easter/Passover. Otherwise, April looks flexible. Apr 8 is both Easter and my wedding anniversary. I believe that other than that, I'm also quite flexible. -- rjbs
Re: Do we need subtests in TAP?
* Michael G Schwern schw...@pobox.com [2011-10-29T05:20:07] [ What if subtests stop indenting? ] Sorry, I'm quite late to the party. I really like the isolated planning of subtests, and the visual indenting, and (least of the three) the potential for building a better visualizer that works with the subtest organization. I would be quite sad to see it gone. -- rjbs
Re: Perl QA Hackathon 2012
* Philippe Bruhat (BooK) philippe.bru...@free.fr [2011-11-03T20:25:25] Here are some news about the Perl QA Hackathon 2012. I have very little to offer other than: Good luck, this all sounds fantastic, and I hope I can make it! As others have said, the very expensive venue sounds very nice, but even if somebody wanted to give us €6000, I bet we'd find other things we'd rather fund than a different venue. But I'm not very invested in either choice, and am sure we'll have an excellent environment no matter what, just because of the team. -- rjbs
Re: TB2::Mouse will be internal use only... with one hitch.
Please stop cc'ing my personal email address on this mailing list mail. I'm already a subscriber. It just pollutes my personal mail from friends folders. -- rjbs
Re: TB2::Mouse will be internal use only... with one hitch.
re: internal use What does internal define here? What are the boundaries of the space to which TB2::Mouse use is internal? In what scenarios do I have to write something that uses it? If I do that, am I likely using Mouse sugar? -- rjbs
Re: TB2::Mouse will be internal use only... with one hitch.
* Michael G Schwern schw...@pobox.com [2011-11-30T15:22:51] On 2011.11.30 7:20 AM, Ricardo Signes wrote: re: internal use What does internal define here? What are the boundaries of the space to which TB2::Mouse use is internal? Like any other internal use only .pm file in a CPAN module, it can only be used by the Test-Simple distribution (Test::Simple, Test::More, Test::Builder, TB2::Blah...). In what scenarios do I have to write something that uses it? You're patching Test-Simple. Okay. So, I want to write a third-party test output formatter that emits test results in the binary language of my water vaporators. I subclass some core formatter with use parent and add some methods or something? I want to write a better Test::Foo library. I'm not using extends 'Tester' or something? The only case when the programmer needs to know about the Mouse underpinnings is when hacking on the dist itself, not not ever reasonable extensions? -- rjbs
Re: TB2::Mouse will be internal use only... with one hitch.
Sorry for the long delay. Lately I'm task-switching at a slow pace. Also, I'm quoting you out of order. * Michael G Schwern schw...@pobox.com [2011-11-30T17:16:47] RJBS wrote, The only case when the programmer needs to know about the Mouse underpinnings is when hacking on the dist itself, not not ever reasonable extensions? Yes, unless you consider writing a Formatter and altering the behavior of the whole Test process to be reasonable. I feel like it's weird if Test::Builder v2 is delivered with the notice, any additional formatter classes must be made internal to Test-Builder, as they must rely on a for-internal-use library. Writing additional formatters is not a reasonable extension. That's what it sounds like, if you're saying that: a) TB2::Mouse is for internal use only b) nothing outside the dist should be using it c) it is unreasonable to write a new Formatter I mean, if you are saying those things, okay! I'm just surprised. TB2 will use Mouse if Mouse is available. [1] Otherwise it will use TB2::Mouse, its bundled and namespace altered copy of Mouse::Tiny. As far as any extensions are concerned, TB2 uses Mouse since must depend on Mouse. This behavior worries me more than anything else. Everything depends on Test::Builder. I'm too lazy to do the legwork right now, but I think we can agree that the vast majority of CPAN distributions with tests use Test::Builder for their tests. The opportunistic loading of Mouse means that nearly *every dist* has the possibility to change its test-time behavior based on an undeclared optional library. [ extremely long, boring, elaborate example written then removed ] I'm concerned about the action at a distance. A change to Mouse can cause a change in Test::Builder, if it really uses Mouse and not just a CP of it. If a distribution at one point in someone's massive dependency tree starts requiring Mouse, and is installed early, it could change the behavior of the subsequently installed distributions, and the change will probably be completely baffling to the user. I am only a little bit comforted by But the Mouse team will be testing TB2 before any release! The biggest reason is that they won't be testing every extension to the Test::Builder framework that might be in play. If writing those extensions is unreasonable for anyone outside of the Test-Builder dist to do, then maybe folks get what they deserve, but if that's the case, nobody needs to write use Mouse::Role for a TB2 role, so the only reason to load Mouse is to get Mouse::XS, which doesn't seem, to me and at first glance, like the right trade-off. If speed is the issue, and people know that their tests are too slow because of the lack of Mouse XS, I wonder whether TB2_USE_MOUSE=1 isn't the answer. Then at least you're opting in to the risk. -- rjbs
Re: Need suggestions for terminology
David Golden xda...@gmail.com writes: I plan to expand further on this idea of separation at the QA Hackathon in the spring (if I don't start working on it sooner). I'd like to get all CPAN clients able to use an index completely separate from a given repository, where the index could, for example, provide URLs to different repositories, such as internal or external, or CPAN and BackPAN, etc. We have a tool to do this, at work. It isn't really in a state that I'd suggest others use it, but I would love to see something better than it become popular, as it has been *very* useful. So, I will be happy to help, at least by telling you what we learned. https://github.com/pobox/xpan -- rjbs
Re: TPF Devel::Cover grant report June 2012
* Paul Johnson p...@pjcj.net [2012-07-12T13:36:26] Feature request: some way of easily seeing just my dists. TODO I can't promise when I'll get around to it, but I do have plans for extending cpancover. However, this work is fairly well decoupled from the core of Devel::Cover, so if anyone has any free time and is looking for a project I'd be very grateful for assistance. (Yeah, yeah, I know, but it's worth a try.) I'd love that feature, too. Where's the repo again? -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report July 2012
* Paul Johnson p...@pjcj.net [2012-08-01T11:48:43] In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering July 2012. Thanks Paul, +1. -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report August 2012
* Paul Johnson p...@pjcj.net [2012-09-07T10:36:52] In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering August 2012. +1, thanks very much, Paul! -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report Weeks 30-32
* Paul Johnson p...@pjcj.net [2013-01-02T09:36:45] In accordance with the terms of my grant from TPF this is the report for weeks 30-32 of my work on improving Devel::Cover. [...] Total 8:50 Thanks, Paul. Interesting report, and good luck! +1 -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report Weeks 30-32
* James E Keenan jk...@verizon.net [2013-01-02T22:39:21] On 1/2/13 9:36 AM, Paul Johnson wrote: Last week I mentioned the pull requests I had received. I this week I merged a pull request from Steffen Schwigon adding some important modules to cpancover.com. I'm always happy to add modules that people would like to see on cpancover, so if there's anything you think should be on there, let me know. I looked over these coverage reports and was shocked, shocked at the low test coverage of many important modules -- including modules which are part of the Perl 5 core distribution. I have set myself the goal of improving the test coverage for Data-Dumper: https://github.com/jkeenan/Data-Dumper. In about four hours of work I've not only improved the coverage substantially; You guys rock. This was a great email to read right before bedtime. Thanks and good luck! -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report January 2013
* Paul Johnson p...@pjcj.net [2013-02-11T15:53:10] I'm pleased to be able to report that we've agreed to continue with the second half of the grant. Excellent! I'd like to publicly thank Ricardo Signes and Florian Ragwitz for their help and support in managing the first half of the grant. I'm well aware that neither of them has really got anything better to do, but I'm grateful nevertheless. It's true, if it weren't for spending 30 minutes a month reviewing these grant reports, I'd be returning to a life of crime! With Florian! Total hours worked on grant: 221:30 +1 -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report February 2013
* Paul Johnson p...@pjcj.net [2013-03-07T17:38:54] In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering February 2013. Great, thanks, +1! Finally, David Golden politely observed that the coverage reporting for ||= operators in his Path::Iterator::Rule module was somewhat less than optimal. So I got hold of the module's source, pared the module down to the minimum required to reproduce the problem, and fixed it, along with another problem that showed up too. Especially for that! -- rjbs signature.asc Description: Digital signature
Re: Smoke Test::More 0.98_05 please.
* Michael G. Schwern schw...@pobox.com [2013-04-25T13:35:49] 0.98_05 is a release candidate for 0.99. I would say this is the last stable release of Test::More before 1.5.0 but hahahaha I won't say that. https://metacpan.org/release/MSCHWERN/Test-Simple-0.98_05/ Test::Exception fails, which breaks a lot of my basic toolchain from installing. -- rjbs signature.asc Description: Digital signature
Re: Smoke Test::More 0.98_05 please.
* Adrian Howard adri...@quietstars.com [2013-04-27T20:09:09] On 27 April 2013 15:19, Ricardo Signes perl...@rjbs.manxome.org wrote: Test::Exception fails, which breaks a lot of my basic toolchain from installing. T::E 0.32 just uploaded that kills the false failure. Thanks! With that, the next problem is Test::Moose::More, which appears to be testing exact isa_ok output. -- rjbs signature.asc Description: Digital signature
Re: TPF Devel::Cover grant report March 2013
* Paul Johnson p...@pjcj.net [2013-05-01T16:38:26] In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering March 2013. +1, thanks, Paul! -- rjbs signature.asc Description: Digital signature
