Re: Revert use_ok() change to allow lexical effects?
Aristotle Pagaltzis writes: > Ah d’uh! Now I feel stupid. Please don't! You're still ahead many of us ... > This could so easily be packaged in a handful of lines: Thanks for doing that. > my $reason = 'Tests must succeeded'; Grammar correction if anybody is going to publish this in a module: "succeed", rather than "succeeded". Cheers Smylers
Re: camels
Ovid writes: > customer perception is important. ... From a marketing perspective, > the camel wins hand-down. From a legal perspective, what are the pros > and cons? This definitely is important -- but as something which affects the Perl community as a whole, this aspect of the discussion seems no longer specific to Perl QA; perhaps the perl-qa list doesn't need to included on Perl-wide logo discussions? Smylers -- http://twitter.com/Smylers2
Re: Cpan Dist Layout Issue
David Golden writes:
> On Fri, May 15, 2009 at 12:11 PM, Smylers wrote:
>
> > That Perl has this QA thing which tells you which platforms a
> > module will work on but that doesn't actually mean it'll install
> > only adds to external perceptions of Perl being confusing and
> > awkward.
>
> Of course CPAN Testers doesn't claim to check whether a module
> installs, only whether it passes tests.
You say "of course", but I'm not sure. _I_ know what Cpan Testers
does[*1], but I don't think it's so obvious to those outside (or new to)
the Perl community.
Suppose somebody finds a distribution on Cpan Search:
http://search.cpan.org/~jzawodny/DBIx-DWIW-0.49/
There's a mention of "CPAN Testers", without explanation. "PASS (46)"
doesn't say what it has passed. 'View Reports' goes to:
http://www.cpantesters.org/show/DBIx-DWIW.html#DBIx-DWIW-0.49
Again there's nothing there which says what is being tested. There's a
promising looking 'Help' box, which starts "So you've stumbled upon this
site and are wondering what it all means ..."; that links to:
http://www.cpantesters.org/help.html
Most of the information on that page is likely to be bewildering or
irrelevant to the user who's just stumbled upon the site, but it does
say mention that it's "a way to provide multi-platform testing for
modules" and provides "valuable feedback for users".
So I'm struggling to see where our sys-admin would learn that Cpan
Testers don't actually test that modules install on those platforms.
Remember, he isn't a Perl programmer. In fact, he isn't a programmer at
all: he's only involved in this at all because he wishes to run a
program, and that program happens to be written in Perl, with some Cpan
dependencies.
> It doesn't even tell you whether there are good tests. It could be
> t/pass.t with a single pass("fooled you") test and it would still get
> a PASS grade on CPAN Testers.
Again, I know that but I'm not sure it's obvious to somebody who doesn't
(though it's probably easier to deduce than not checking installation).
Anyway, our sys-admin could easily end up deciding that Cpan Testers
results are clearly wrong, and think less of Cpan Testers or Cpan or
Perl or the Perl Community. Maybe in future when evaluating software
he'll be more inclined to choose something written in another language,
so he doesn't have to deal with the trials of Cpan again.
One possible response to this is that he's wrong: he's made an incorrect
presumption, and if he goes off elsewhere that's his loss.
And that's fair enough. But possibly there are those in the Perl
community who think this perception is unfortunate and are interested in
doing something about it, making the Perl World a friendlier or more
helpful place.
[*1] Though actually is there anything stopping somebody from checking
whether modules actuall install and sending FAIL results to Cpan Testers
for those that didn't? Surely they'd just display like other FAILs.
Yes that'd be more involved than what current testers are doing, but I
don't think it means it couldn't be done by somebody.
Smylers
Re: Cpan Dist Layout Issue
Graham Barr writes: > It is documented in ExtUtils::MakeMaker > > PMLIBDIRS > Ref to array of subdirectories containing library files. Thanks. > So for this modules it will scan lib/ and DWIW/ neither of which > exist. It also scans . Having never tried to use a directory layout other than ./lib/ I hadn't thought about where this came from, and having seen the correct filename mentioned in somewhere in Makefile.PL presumed that meant MakeMaker knew about it. So would it make sense for MakeMaker to warn if a file mentioned in VERSION_FROM (or ABSTRACT_FROM -- any others?) is in a directory that isn't in PMLIBDIRS? Smylers
Cpan Dist Layout Issue
I recently encountered a Cpan module which doesn't install from its
distribution yet passes all its tests, and I was wondering whether the
glitch is a generic one which should be addressed.
This particular module is not my concern (its maintainer responded
speedily and is now looking at fixing it), but it illustrates the issue:
http://cpansearch.perl.org/src/JZAWODNY/DBIx-DWIW-0.49/
Note the module source is in the tarball at ./DBIx/DWIW.pm, and that is
reflected in Makefile.PL. Running make creates the ./blib/lib/DBIx/
directory but doesn't copy the module to it (nor generate the manpage).
It appears to work fine with the source at either ./DWIW.pm or
./lib/DBIx/DWIW.pm (and a corresponding Makefile.PL).
* Is this a known limitation of MakeMaker or a bug? If a bug then I'll
report it. If a limitation, would it be possible for MakeMaker to
detect this condition and bail out (on both make dist and make),
rather than giving the impression it's succeeded?
* make test succeeded because perl puts the current directory at the end
of @INC. Because DBIx::DWIW isn't in ./blib/, the one at the top of
the tarball is found and used for the tests.
Obviously this doesn't happen with other tarball layouts because
./lib/ isn't in the path for make test (and ./DWIW.pm can't be in the
path, because the parent directory isn't called DBIx; that would be an
issue for single-level modules though).
* That the module doesn't install at all makes a slight mockery of the
many Cpan Testers reports giving it a PASS:
http://www.cpantesters.org/show/DBIx-DWIW.html#DBIx-DWIW-0.49
Yes, I know why that is, but that doesn't make it any better. It was
a sys-admin colleague with little Perl experience who encountered this
problem, while trying to install a program which uses this module.
That Perl has this QA thing which tells you which platforms a module
will work on but that doesn't actually mean it'll install only adds to
external perceptions of Perl being confusing and awkward.
* Even if MakeMaker were modified so that this distribution would
install, it's still concerning that modules with this distribution
layout get picked up by make test: there are other sorts of mistakes a
module author could make which would result in a bogus test pass
because of this.
* So it'd be nice if when running tests the current directory weren't in
@INC. Putting the following line in the Test and Test::More modules
would achieve this:
BEGIN { pop @INC if $INC[-1] eq '.' }
But that's so non-backwards-compatible that I'm guessing it would
cause problems elsewhere.
* An alternative would be to have make test do the above (for example by
putting the above line in a module and having make test -M that module
when running a test script). That limits the scope of the change, but
it's still backwards-incompatible -- and it means anybody running an
individual test directly still risks including a file they didn't
intend to.
* Or make test could cd into the blib/lib/ directory, such that the
current directory is one that should be in the path. But Michael
Schwern a few days ago argued that tests should be run from the parent
directory:
http://www.nntp.perl.org/group/perl.qa/2009/05/msg12215.html
Any thoughts?
Smylers
Re: TAP::Data or TAPx::Data?
Andy Armstrong writes: > On 11 Feb 2009, at 13:50, Smylers wrote: O > > > TAP::Struct? > > +0 - in the sense that it's fine by me - but I can still imagine a > better noun even if I can't think what it is :) +1 to Andy's comment on my suggestion! Smylers
Re: TAP::Data or TAPx::Data?
Steffen Schwigon writes: > For this purpose I need a reliable data structure that containes > parsed TAP. TAP::Struct? Yeah, I know "struct" is a C-ism, but it's quite well known (it has a Wikipedia page!) and at least makes the point it's structured data rather than just "data". Smylers
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
Michael G Schwern writes: > Smylers wrote: > > > > I have lying around a prototype for the CPAN shell to warn the user > > > when they run it as root and offer to reconfigure itself to only su > > > for the install. That would help plug the hole. > > > > Yeah, that sounds good. > > > > But only for users running CPAN, not anybody who is manually un-tar-ing > > a distribution. I have no data for this, but I suspect those who do > > manual installs in this way are also more likely to do the whole thing > > as root, and less likely to be involved in the Perl community (such as > > knowing much about Cpan) -- and therefore most likely to get hurt by > > this, or to pick up a bad impression of Perl or its community as a > > result. > > Since the perl build process is directly analogous to the autoconf build > process... I think the difference is that autoconf-generated software is generally created on Unix-like OSes, with actual tar commands. Cpan is different, in that it's quite reasonable for distributions to've been created on a system without tar or Unix-like permissions. > perl Makefile.PLsh Configure > makemake > make test make check > sudo make install sudo make install Or: $ su # ./Configure # make # make install I don't know how common that is any more. Most of the people I've seen doing it were installing Unix software well before sudo gained widespread popularity, and seem to've got into the habit of running as root for sys-admin tasks. But I take your basic point: > ...this is not a Perl problem but a general lack of basic security > problem. An admin should know to run as little as possible as root, > this is dead basic security. Anyone who blames Perl for the admin's > mistake is just looking for someone to blame, so there's little bother > in trying to convince them otherwise. > > We can only keep an ignorant admin from blowing off their foot for so > long. The longer we protect them from their own ignorance the bigger > the boom is likely to be. Smylers
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
Michael G Schwern writes: > Smylers wrote: > > > you're talking about Cpan being something morally equivalent to a > > common carrier, rather than an actual common carrier in the legal > > sense? > > Yes, because we are not lawyers I don't even want to approach arguing > about the legal definition. But there is utility in the idea, as a > line. The idea that the carrier is not responsible for the content. Makes sense. But I don't think rejecting certain uploads for > > The Debian manpage for GNU tar documents this option: > > > > --no-same-permissions > > apply umask to extracted files (the default for non-root > > users) > > > > So umask would be ignored for Andreas above because he un-tar-ed as root > > (and I'm guessing you tried it as you, thereby not triggering the > > behaviour). > > Yes, I don't even give myself root or su access to avoid accidentally > forgetting I'm logged in as root. Everything is through sudo. Me too. It just seems much saner all round. > > Requiring root privs for the last step of installation is common, so > > I guess it's fairly common for some people to do all the steps as > > root (however inadvisable that is). > > Well then those users are fucked another dozen ways. I've seen an alarming number of people do this, and generally get away with it. > I have lying around a prototype for the CPAN shell to warn the user > when they run it as root and offer to reconfigure itself to only su > for the install. That would help plug the hole. Yeah, that sounds good. But only for users running CPAN, not anybody who is manually un-tar-ing a distribution. I have no data for this, but I suspect those who do manual installs in this way are also more likely to do the whole thing as root, and less likely to be involved in the Perl community (such as knowing much about Cpan) -- and therefore most likely to get hurt by this, or to pick up a bad impression of Perl or its community as a result. Smylers
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
Michael G Schwern writes: > I use the term "common carrier" [1] because it has a very special > meaning. > > [1] "common carrier" is a legal idea from common US/UK law. I don't > want to get into the legal mumbo jumbo because we're not lawyers, but > invoking the idea is useful and powerful. OK, so you're talking about Cpan being something morally equivalent to a common carrier, rather than an actual common carrier in the legal sense? > It is the difference between just transporting sealed packages and > not. Once you peek inside them and get involved in their business, > for whatever reason, you are no longer a common carrier. This is a > whole different ball game. Indeed. But if Cpan is only _like_ a common carrier, then it never actually had common carrier protection in law in the first place -- so surely it can't lose it? Smylers
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
Aristotle Pagaltzis writes: > * Michael G Schwern <[EMAIL PROTECTED]> [2008-11-13 04:15]: > > > I really, really, really don't want PAUSE modifying my stuff after > > it's uploaded. > > Count me in this camp. That's my instinct as well. > I do think that PAUSE could fix this, but it *MUST* require author > consent. Given how few instances of this have been counted, is it worth the effort for this, rather than just rejecting problems? > (Needless to mention, once the toolchain is appropriately patched, the > won’t-index mail should also include the hint that if on Windows, one > might want to upgrade one’s toolchain to avoid having to deal with > this hassle.) Nah, this is Windows -- Pause should just use a rootkit to gain access to the developer's computer and upgrade the toolchain for him ... > > Until now CPAN has been a common carrier. Pretty much anything was > > allowed, stuff was only rejected for extreme reasons and always on a > > case-by-case basis and always by human judgment. > > The filtering does not change this. It doesn’t cause the upload to be > rejected. Also I believe that even proper legal-entity common carriers are permitted to reject certain classes of things. The Wikipedia page mentions postal services declining to tranport money; Royal Mail also refuses aerosols, batteries, counterfeit currency, dry ice, obscene publications, foreign lottery tickets ... Rejecting certain undesirable uploads meeting well-defined criteria is a long way from modifying stuff in other people's names. Smylers
Re: [PATCH] ExtUtils::MakeMaker and world writable files in dists
Michael G Schwern writes: > Andreas J. Koenig wrote: > > > # umask > > 002 > > # tar xzf > > /home/ftp/pub/PAUSE/authors/id/Y/YV/YVES/ExtUtils-Install-1.51.tar.gz > > # ls -la ExtUtils-Install-1.51 > > total 1104 > > -rwxrwxrwx 1 544 5131765 Mar 3 2008 Build.PL* > > Your tar is not honoring umask. ... What tar is that, btw? I've > tried out both BSD and GNU tar. The Debian manpage for GNU tar documents this option: --no-same-permissions apply umask to extracted files (the default for non-root users) So umask would be ignored for Andreas above because he un-tar-ed as root (and I'm guessing you tried it as you, thereby not triggering the behaviour). Requiring root privs for the last step of installation is common, so I guess it's fairly common for some people to do all the steps as root (however inadvisable that is). Smylers
Re: Generic test database
David Cantrell writes: > Smylers wrote: > > > It sounds scary to me. If I'm just installing a Perl module from > > Cpan on a newly installed OS, which happens to still have default DB > > connection permissions, I wouldn't expect the module's tests to > > start making use of the DB without asking. > > Of course, a module author can do that already, Indeed (and she could also do the other things I mentioned, such as grabbing images or e-mail addresses). But I don't thing we should encourage such things as being a good practice. > > Further, if the module install fails the database may be left > > behind. > > Having a well-known module specifically for creating, accessing, and > tearing down the test database, written carefully and maintained, > instead of people just hacking up their own nasty solutions to the > problem whenever they need it - this will *reduce* the chances of that > happening. End result - less cruft left on your system *and* better > testing for databasey modules. Yes, that's good. Connecting using environment variables which the user (or tester) has explicitly set up for the purpose is also good. I like this proposal overall, and the current plan for implementing it -- it was only the 'let's grub around on the system and see if the user happens to have left a DBMS unprotected that we can use' but which scared me. Simon
Re: Generic test database
Greg Sabino Mullane writes: > > ... I suppose it could also be somewhat easy to try out a few series > > of basic/default credentials on localhost for engines like MySQL and > > Postgres, and try to setup a test database from there. > > That sounds more interesting. It sounds scary to me. If I'm just installing a Perl module from Cpan on a newly installed OS, which happens to still have default DB connection permissions, I wouldn't expect the module's tests to start making use of the DB without asking. Further, if the module install fails the database may be left behind. Which is going to be very surprising when I then connect to the DBMS and see there's already this weird database there. (Similarly, if a test needs an image, please don't just search users' home directories for one; if a test needs to send an e-mail, don't just look in the user's address book for somebody to mail. Just cos a test is running as a user which has permission to do certain things outside the current directory doesn't mean it's reasonable for a test to do so.) Simon
Re: [RFC] Dealing with World-writable Files in the Archive of CPAN Distributions
Ovid writes: > --- On Tue, 23/9/08, Shlomi Fish <[EMAIL PROTECTED]> wrote: > > > The default Mandriva umask appears to be 0002 . > > That surprised me In general 0002 (aka u=rwx,g=rwx,o=rx) is the right choice of umask on a sytem where each user has their own group -- that is, where the user ovid has a default group of ovid (which doesn't have any other members). So files by default are created as ovid:ovid rw-rw-r--, and nobody but ovid can write to them. The each-user-has-a-private-group set-up is quite common. It facilitates having shared directories which members of a group can all write to. For example, if a group of developers are all in the dev group then root could create a directory that is root:dev rwxrwsr-x. A umask of 0002 means any files you create in that directory will be ovid:dev rw-rw-r-- (or rwxrwxr-x, if appropriate), so all members of the group can write to them, as intended. Whereas a umask of 0022 would leave the files not group-writeable, so you would have to specifically chmod them each time. That's irritating. Having 0002 is more convenient, and, so long as your default group is one with only you in it, not a security risk. Smylers
Re: TODO Tests
Bram writes: > At the moment foo() returns 3. > > Time passes and code changes. > > Now there are 3 options: > > foo() returns 1, this will result in 'unexpected todo test passed' > being outputted; > foo() returns 3, no special output is produced; > foo() returns 4, no special output is produced; > > To me the last one seems like a limitation. > The result of foo() is now different then when the TODO test was > written but there is nothing in the output that indicates this. If you believe that (until the TODO is done) foo will consistently return 3, and you wish to be alerted if it suddenly starts returning 4, then surely you can do that with a non-TODO test which checks for its being 3? Then, when the TODO is done you can remove that test entirely (while also removing the TODO marker from the check for 1)? Smylers
Re: Test::Most::EXIT()
Eric Wilhelm writes: > What I wonder is how Ovid ended up with a module setting a global die > handler that wasn't under his direct control Why would it being under his control mean he didn't want to test for it? I've been in situations before where a co-worker has inserted a die handler into a library, which then broke something which used that library. The code was completely under our control; we were able to fix it. But it'd've been good to have a test; perhaps that would've stopped the die handler being committed like that in the first place. Smylers
Re: [tap-l] User Supplied YAML Diagnostic Keys: Descriptive Version
Michael G Schwern writes: > David E. Wheeler wrote: > > > On Apr 18, 2008, at 10:50, chromatic wrote: > > > > > My argument was complex: solve the real problem or don't solve it. > > > The in between position is silly and won't make anyone happy. > > > > Yes. The choices, as I see them, are: > > > > 1. Do we start out conservative, and loosen things up as the cow > > paths indicate (Ovid's position)? An example of this would be saying: We promise that all future 'official' keys will start with a lower-case letter. It is prohibited for you to start an in-house key you create with a lower-case letter. > > 2. Do we start out liberal, allowing anything, and formalize > > whatever turns up in the cow paths (what I'm now suggesting)? An example of this could be saying: All current 'official' keys are lower-case (and start with a letter) and we expect future ones will be as well. If you choose to create an in-house key which starts with a lower-case letter then you risk a clash in future should we happen to introduce a key with that name. In practice, what's the difference between those two? The first is apparently stricter, but the prohibition can't actually be inforced: if a user chooses to ignore it then we can't send him to prison or anything; it's just that there's a (small) chance of his system later failing. Similarly if the future Tap maintainers bizarrely do create a key named in some other way, whether they are breaking a promise or a mere expectation doesn't make any difference: it isn't like users can sue over a broken promise. So since choice 1 can't be enforced, it seems to behave exactly like choice 2 anyway. > The prefixing solution sucks, but it's all we have... and that's a bad > place to be. Rather than arguing about a sucky solution, does anyone > have another solution to offer? In practice users can name keys however they want. Namespace clashes will be a bigger issue for some users than others. Issue advice on how to mitigate the problem and let users choose whether to follow it: It's possible that a key you pick will also be used by somebody else for a different purpose, which could cause problems if your systems later get used together. You can reduce the risk of name clashes by starting all your keys with a standard prefix or your organization or project name (or something else likely to be unique). Smylers
Re: Keeping tests for execution at a later point in time
David Golden writes: > On Thu, Apr 10, 2008 at 7:14 AM, Gabor Szabo <[EMAIL PROTECTED]> wrote: > > > So let's see what needs to be done in order to be able to keep > > the test files and run them later. > > > > There are two concerns I could immediately see. > > 1) Tests might assume a certain directory structure, they might say > > use lib 'lib'; > > use lib 't/lib'; > > or other things. > > 2) Tests might use other files outside the t/ directory. > > 3) What else do you think might be problematic? > > Seems like a lot of headache. Why not just save the tarballs > somewhere? Were there tarballs in the first place? > Then you're certain to recreate the environment that the distribution > expected for its tests when run by CPAN.pm. Suppose a Fedora or Ubuntu user is wanting to check that upgrading the OS hasn't caused any test regressions, it's likely that at least some of the existing Perl modules were installed as RPMs or Debian packages, not by CPAN.pm from tarballs. Smylers
Re: User Supplied YAML Diagnostic Keys
Aristotle Pagaltzis writes: > * Ovid <[EMAIL PROTECTED]> [2008-04-07 10:35]: > > > By requiring user keys to begin with 'X-', it's visually distinct, > > immediately clear to the user, it follows conventions used in mail > > and HTML headers, and if it's wrong, it's easy to change. > > A full `X-` would be a bit heavy, visually. How about requring > punctuation as the first character? > > _want: ... > _secure: y Would that mean that _secure, -secure, ~secure, +secure, !secure, and so on are all distinct private keys? I'm not sure that's advantageous; picking a single way of denotating privacy would seem less confusing. Smylers
Re: CPANTesters considered harmful
Brad Oaks writes: > On Mon, Mar 3, 2008 at 9:19 AM, Smylers <[EMAIL PROTECTED]> wrote: > > > demerphq writes: > > > > > It turned out the problem is that when the tests are root it seems to > > > be not possible to create a directory that is not writeable by root. > > > > I think that can be reduced to: It isn't possible to create a directory > > that is not writeable by root. The whole point of root is that as the > > super-user it can do anything! > > (I know this is not the point of the conversation, but file this away > as it might be useful to you some day.) > As part of being able to do anything, root *can* make a directory that > it cannot write to. > > Enter the immutable flag: > > [EMAIL PROTECTED] ~]# chattr +i unwritable/ $ chattr -bash: chattr: command not found > for more info: > man chattr > man lsattr $ man chattr No manual entry for chattr $ man lsattr No manual entry for lsattr That's on a FreeBSD server. Let's try Linux. Ah, yes: $ whatis chattr chattr (1) - change file attributes on a Linux second extended file system So you can make this test work even when running as root, so long as you don't mind it now failing on Unix-like OSes that aren't Linux. I'm not sure that's a nett win! Smylers
Re: CPANTesters considered harmful
demerphq writes: > It turned out the problem is that when the tests are root it seems to > be not possible to create a directory that is not writeable by root. I think that can be reduced to: It isn't possible to create a directory that is not writeable by root. The whole point of root is that as the super-user it can do anything! > Im not really sure how to tackle this better than simply skipping the > tests as root which is what the most recent release does. That's plausible. It could also temporarily drop privileges to be some other user for running that test, but I don't know how you'd work out which user to do it as. Smylers
.perldb (Was: Re: Debugger: stopping in caller)
Andy Armstrong writes: > That should be s/return 1/return/ at the end. Sorry, I edited from my > real ~/.perldb (which contains a load of other stuff) ... Oooh, what other stuff? As somebody who doesn't have a .perldb at all (I'm not even sure I was aware of it before this thread) I'm intrigued as to what's possible. I use the debugger only infrequently, but perhaps with some tweaked config I'd be inclined to use it more. Any chance of a 'Me and My .perldb' (lightning) talk somewhere? Smylers
Re: What should it's name be?
Ovid writes: > Forget all the silly debates. I don't want to wait or sit around > while we argue over this silly stuff. You mean you don't want flaming for the apostrofly in the subject? OK then, I won't mention it. > Discussions involving ... piping through less ... Keep that thought; I'm going to repurpose it below ... > ... what name do you want? Test::Even::More? That'd create a Test::Even:: namespace; I'm not sure if anything else would use it, but it'd be a bizarre cateogry if it did. (Though it does suggest we could have a Test::Odd:: counterpart, which given some of the recent discussion on this list may have its uses!) Names normally descend in specificity from left to right, but Test::More::Even doesn't really make sense. > Test::Extended? Something else? Your mentioning the less pager above reminded me that it's so-named as the opposite of more, an earlier pager it improved upon. So I wondered about Test::Less, but it appears Ingy's already using that for something. But somewhere that train of thought led me to Test::Most -- which implies "more than 'more'", and could work if you feel like claiming this module will meet most testing needs, or be useful in most test programs or something? Smylers
Re: Why not run a test without a plan?
A. Pagaltzis writes: > Hi Michael, > > * Michael G Schwern <[EMAIL PROTECTED]> [2007-12-04 03:35]: > > > use Test::More; > > pass(); > > plan tests => 2; > > pass(); > > > > Why shouldn't this work? > > ... this should be allowed: > > pass(); > plan 'no_plan'; > pass(); > > Whereas this should not: > > pass(); > plan tests => 2; > pass(); Why not? That's exactly what Schwern was suggesting should work. > > It also makes it technically possible to allow the test to change > > it's plan mid-stream > > Without some hypothetical future version of TAP this is only possible > if you have run tests before declaring a plan at all, because > otherwise the plan will already have been output as the first line of > the TAP stream. Wasn't there general agreement only a week or so ago to now allow plans to be specified at the end rather than the start? I was presuming that Schwern's suggestions were in the light of this other change. Smylers
Re: Providing command-line arguments to tests run via 'prove'
Andy Lester writes: > As much as I like the non-quotedness of the -- or --testargs idea, I > really think it needs to be --testargs='--foo --bar'. If we have just -- then its straightforward what to do with the arguments. After everything up to and including -- has been shifted off @ARGV, then you can just do: my @test_args = @ARGV; and if $test_file is the test being run you can then do the moral equivalent of: system $test_file, @test_args; But with --testargs you read in a single string so you get something like: $test_args = '--foo --bar'; And suppose we're trying to run $test_file. Obviously this is wrong: system $test_file, $test_args; For the above arguments you'd want to do: system $test_file, split / /, $test_args; But what if one of the arguments needs to have a space in it. With -- you can do: % runtests whatever -- --foo "kapow blam" How do you write that with --testargs? Perhaps: % runtests whatever --testargs='--foo "kapow blam"' Hmmm, but now this is wrong: system $test_file, split / /, $test_args; We need to only split on the unquoted whitespace, and we need to remove the quotes from the outside of what will then be the second argument. Obviously it's possible to do that. But it's hard to get it right, and to meet users' expectations. In particular, different OSes have different quoting rules. With -- it's still the local shell doing the quoting, so the user obviously just uses their normal quoting rules. With --testargs, runtests would be doing the parsing. Should it emulate the local OS's rules (but then behaving differently on each platform), or should it pick one set of rules and implement them everywhere (but then imposing, say, Unix-style quoting on Windows users)? The convention of using '--' to mean 'that's the end of my own arguments' neatly avoids all of these issues. Smylers
Re: Providing command-line arguments to tests run via 'prove'
Michael G Schwern writes: > There's nothing about '--' which indicates "pass the rest through to > the tests". It did to me, honestly! As soon as I opened David's e-mail at the start of this thread the "--" jumped out of the paragraph of text, and I guessed what his words would say before I read them. > In fact, it normally means "stop processing the following as switches > and instead treat them as files". Or possibly, it means "stop processing the following as switches". Which is what it does mean in this case. Smylers
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong writes: > On 28 Nov 2007, at 21:06, Andy Lester wrote: > > > Which is why I said --testargs > > > > prove -v t/fetch.t --testargs http://wakeupandya.com/ > > I thought --testargs was a metasyntactic name for a test argument :) > > Sorry folks. I'm being even more dense than usual tonight. Well I did exactly the same until Andy's second post! Borrowing the established meaning of -- means it can be understood (by at least some people) straight away; making anything new up especially for the purpose means everybody has to learn whatever is chosen. Also, because -- looks different from other options, it works well as a divider, making it more obvious that _everything_ that follows is distinct from that which went before, rather than it just being like a 'normal' option, where what follows are also options. Smylers
Re: Providing command-line arguments to tests run via 'prove'
Andy Armstrong writes: > > > > Could we not add a feature to prove and/or runtests such that, any > > > > arguments after a bare "--" will be passed on to the scripts it > > > > runs? I've often wanted this myself, and --exec seems like > > > > overkill to me. > > Agreed. I think the '--' is redundant But is it doing harm? There are quite a few commands which use -- to mean 'end of options; what follows is data' (where "data" can (and often does mean) options to pass opaquely to another command being invoked). Smylers
Re: New proposed CPANTS metric: prereq_matches_use
David Cantrell writes: > On Thu, Nov 22, 2007 at 12:25:49PM +0100, David Landgren wrote: > > > do I really declare pragmas like 'strict' and 'vars'? How many ISPs > > remove *them*? > > Excluding pragmata that have been in core since at least 5.6 would be > a good idea, I think. Really? Personall I think it would be a _terrible_ idea for ISPs to exclude such modules from their installations ... (Oh, I see what you mean. Sorry!) Smylers
Re: Running CPAN as a regular user, installing as root
Eric Wilhelm writes: > Yep. And if your sudoers file is more restrictive, you might have > something like this (relative paths aren't allowed in sudoers) > > ewilhelm ALL=(ALL) NOPASSWD: /usr/bin/make install That does of course mean that any user can run any command at all as root, passwordlessly: all she has to do is create a makefile in the current directory which has 'install' as a target and the command in question as the action. This may be acceptable (it's something you're very likely to do by accident), but it isn't suitable in environments where the purpose of the sudoers restrictions is because you don't completely trust all of your users. Smylers
Re: Win32:GuiTest
James E Keenan writes: > Gergely Brautigam wrote: > > Does anybody know why is that so that sometimes when trying to set a > > focuse to a window the window does not come into forground but it > > stays on the menubar and just blinks...? Is that a windows problem > > or a Win32 module problem? > > Hi Gergely. Glad to have you aboard. But please let me suggest that > there are probably better lists on which to pose this question. > perl.qa is very focused on use of Perl in testing and Quality > Assurance. So questions on Win32 are off topic here. The subject of this thread is Win32::GuiTest -- I know nothing about it, but given it's a module for use with testing it seems on-topic for this list. Smylers
Re: a "standard" place for extra tests
Eric Wilhelm writes: > # from Smylers > # on Thursday 16 August 2007 11:40 pm: > > > > I am certain that more than one 'extra tests' directory is needed, > > > > Why are you certain of this? > > Because I already have a use for three 'profiles' in one project and I > can name a few others from elsewhere: > > 1. author (kwalitee, pod, etc) Yep. > 2. gui Why can't gui tests simply be in t/ and be skipped if the appropriate environment isn't available? That way all users who are in that particular environment run the tests, rather than only those who've discovered the xt/gui/ directory. > 3. network Ditto -- why isn't skipping enough? > 4. you must have an account/password on $external_service > 5. postgres/mysql/whatever availability/setup/permissions There are existing modules user tests which cope with those, either by prompting or environment variables -- defaulting to skipping the tests, of course. Possibly we need a better (or standardized) way of getting information from users running tests in these situations, and for safely skipping by default. (When trying to get a bunch of Cpan modules to install unattendedly I discovered they collectively had many ways of seeking attention; having none of them waiting at a prompt involved all three of setting an environment variable, passing an option to Makefile.PL, and piping yes '' into it.) > 6. no modem attached to /dev/ttyS0 That could probably be treated similarly to the website case, defaulting to presuming no modem is there unless explicitly told so. > What I would ultimately like to see happen is that these > subdirectories fall into some form of standard(ish) naming scheme > which would allow testers and qa efforts to run particular subsets. > That is, the smoke box (or rigorous end-user upgrade verification > process) could set variables to test the author, 'gui', 'network', and > 'postgres' tests[1] without stumbling into e.g. "must have an X.com > web-services account". That requires a central standardized repository of permitted subdirectories and what needs to be done to run the tests there. Surely it's better to let each distribution keep control of this, by specifying in code what's needed to run those particular tests (and skipping otherwise)? Smylers
Re: a "standard" place for extra tests
Ovid writes: > --- Smylers <[EMAIL PROTECTED]> wrote: > > > > child directory I'm fairly certain of. I am certain that more > > > than one 'extra tests' directory is needed, > > > > Why are you certain of this? It seems far from obvious to me. > > for example: > > xt/smoke > > The latter of which is designed to run smoke tests against a wide > variety of Perl versions. That should not be in the t/ directory, but > it's also clearly not an author test. Just to be clear (because this is an area I'm ignorant about, not because there's anything wrong with your explanation) this is for the situation when testing a module with several Perl versions requires something other than merely running the standard tests on each Perl version, and as such these tests wouldn't routinely be run by the author (because he's just developing with one version of Perl to hand) but are available for use by people specifically doing this kind of smoke testing -- have I got that right? > Regardless of the examples we come up with, I'm terribly leery of > starting with the assumption that "we will only ever need one extra > test directory and therefore should not plan for more". Sure. I didn't say I was _against_ this level of hierarchy, nor that I was certain we wouldn't need. I merely asked why whoever-it-was-whose- name-you-snipped-from-the-above was certain we would -- so as to provide guidance to those of us who haven't yet worked it out. > Enforcing arbitrary limits at the beginning is not a good idea. Indeed not, but nor is creating lots of structure that isn't going to be needed; it's a balance. > > > As for the 'xt' name: > > > > Nobody is intuitively know what "xt" means > > Because 't/' is self-explanatory? Of course not, but that's no reason to make things worse when there are plenty of other more descriptive potential directory names out there. Smylers
Re: a "standard" place for extra tests
Eric Wilhelm writes: > Returning to something that came up in the 'testing pod' thread... > > # from Adam Kennedy on Tuesday 31 July 2007 08:23 pm: > > > A small nit, if we end up converging around the use of a relatively > > standard directory name, can we at least use something relatively > > self-evident? For example "author/"? I'm definitely with Adam on this one. Nobody is intuitively know what "at" means, and it isn't obvious that there's a need for the directory name to be short. > I would like to suggest that we adopt, recommend, (and support with > tools, etc) a "standard" (convention) for storing additional tests. > > MyModule-v0.0.1/ > xt/ > author/ > network/ > > Or something along those lines. At this point, 'author' is the only > child directory I'm fairly certain of. I am certain that more than > one 'extra tests' directory is needed, Why are you certain of this? It seems far from obvious to me. Anything under this directory isn't going to get run with 'normal' tests, which is an unusual requirement. > As for the 'xt' name: Nobody is intuitively know what "xt" means, and it isn't obvious that there's a need for the directory name to be short. > I think anything starting with 't' would get annoying as far as > tab-complete goes. You'd only have to type 2 characters for the new directory name, and "t" itself would obviously remain a single character that doesn't even require pressing Tab. Smylers
Re: [tapx-dev] Tracking tests over time
Andy Armstrong writes: > Michael Peters wrote: > > > Why can't you just use test names? > > > > ok($is_ok, 'HTML nested bullet list, open tags (bug 5497)'); > > You certainly could - but I tend to think of them as descriptions > rather than names and therefore not entirely ideal as the primary key. > I don't really want to be forced not to change a description because > I'll break the test system if I do. In the above surely "(bug 5497)" is sufficient as the primary key; the rest of the text could change. So this system could be made to work for anybody who has a bug database and consistently labels tests with them -- which obviously isn't everybody, but it might not be an excessive requirement for those who want the tracking feature. Actually you don't even need for the external database to exist, so long as you include the unique bug IDs somewhere in the description. You just need to pass to the tracker the pattern that matches your format of IDs from the descriptions; for the above example that'd be: qr/\(bug \d+\)/ Smylers
Re: use_ok and %INC
Ovid writes: > http://use.perl.org/~Ovid/journal/33441 > > In this case, I think either use_ok() should fail (which might break > some test suites since its behavior has changed), or that it should > issue a warning if $module is found in %INC. How about instead having use_ok simply removing the %INC entry if it's found, so that it can be used safely in exactly this situation? Would doing that silently obscure any genuine bugs? Smylers
Re: The price of synching STDOUT and STDERR
chromatic writes:
> There ought to be a way to capture diagnostic information in the TAP
> stream because it's useful for diagnosing problems.
Does that help with the case where it's an 'ordinary' Perl-generated
warning ("Use of uninitialized value" and the like), which runtests is
also swallowing at the moment?
Smylers
Re: The price of synching STDOUT and STDERR
Michael G Schwern writes: > ... TAP::Parser guarantees that STDOUT and STDERR will be in sync, > something Test::Harness does not guarantee. > > First thing is breaks, and probably most important: No warnings. That caught me by surprise this week. runtests was showing a failure I didn't understand, so for no particular reason better than 'randomly changing stuff to see if it makes a difference' I tried it with prove, and got a useful warning. Fortunately Ovid was passing my desk at the time, and explained that runtests needs the -v flag to show warnings. But I hadn't been expecting runtests to differ from prove in such a critical way, nor that warnings would be eaten by default. And once TAP::Parser becomes more widely used, it'll be hard for Ovid to happening to be passing the desk of every user who is puzzled by this ... > The simplest solution, and one which unblocks Test::Harness 3.0 > because TAP::Parser breaks compatibility in its current state, is to > remove the syncing feature. I have no view on whether syncing should be removed (or relegated to an option, or whatever), but I am in favour of warnings showing up by default. Smylers
Re: nested groups in TAP
Fergal Daly writes: > On 12/03/07, chromatic <[EMAIL PROTECTED]> wrote: > > > On Monday 12 March 2007 10:53, A. Pagaltzis wrote: > > > > > Making "either the consumer talks to the producer and the > > > producer has to implement some complex logic, or the producer > > > sticks to the plainest TAP possible" a requirement would just > > > lead to most ad-hoc implementations of producers being forever > > > stuck with producing the plainest TAP possible. > > > > +1, has the historical support in the example of, oh, just about every > > standard Internet protocol ever devised. > > What? > > GET / HTTP/1.1 > > there it is right there, the consumer saying "I'd like to speak this > version, can you handle it?" No it isn't. It's the instigator of the converstation saything "I am going to speak version 1.1; deal with it". Which seems the right thing to do with Tap as well. Each Tap stream merely needs to be identified by the number a version with which it is compatible (ideally the lowest such version, but it'll still work even if not). Then something reading that stream knows that if it supports the version number in question it will be able to understand the stream, and that otherwise it should give up. Smylers
Re: TAPx::Parser -> TAP::Parser?
Ovid writes: > Schwern ... does not object to renaming TAPx::Parser to TAP::Parser. Please do it! For anybody first encountering it after the rename it is one less arbitrary fact to remember, one less question they have about it. And obviously the sooner the better, from the point of view of minimizing the number of people who have to cope with the change. Smylers
Re: Test::Harness 3.0
Eric Wilhelm writes: > # from Smylers > # on Sunday 21 January 2007 11:50 pm: > > > Eric Wilhelm writes: > > > > > If that isn't enough, I suppose you could do "if the env var is an > > > executable, run it and capture the output"? > > > > Nice -- so that if you manage to trick somebody into setting that > > environment variable you can get them to run any code you want the > > next time they install a Cpan module that doesn't explicitly set > > this variable? > > Sure. That, and $EDITOR. I think there's a difference. Apart from EDITOR being much more well-known and standard, invoking an editor is something which the user is usually expecting, and can see when it's happened. Even without the variable being set some default editor runs. This would be an environment variable which optionally ran some additional process; if that process is silent that from user's point of view it doesn't look any different from the variable not being set. And if they were just installing a Cpan module they very likely don't even know that such a hook exists (if they even know about testing). > I don't think defining an environment variable to point to an > executable is a huge issue. Agreed, it isn't huge. But it'd be better not to introduce even a small risk if we don't have to. And Adam now says we don't need this for his desired features. Smylers
Re: Test::Harness 3.0
Eric Wilhelm writes: > PERL_TEST_HARNESS_DUMP_TAP="$(test_dir_for_this_dist)" > > If that isn't enough, I suppose you could do "if the env var is an > executable, run it and capture the output"? Nice -- so that if you manage to trick somebody into setting that environment variable you can get them to run any code you want the next time they install a Cpan module that doesn't explicitly set this variable? Smylers
Re: Test::Builder feature request
Nadim Khemir writes: > On Thursday 11 January 2007 18:04, Ovid wrote: > > > > Just one, Shouldn't the mailer "object" be mocked and the mail > > > sending checked? > > > absolutely, but how do you know to mock it or really send the email > > unless you know that you're being run by tests? > > Aren't you mixin contexts here? The code to be checked is sending mail > (right?). The test framework mocks the mail object so only the test > code needs to do something special and I believe the test code knows > the test code is running. You're assuming that the mail-sending code is an object, and separate from other things that should be run in the test. In bad code (which Ovid stated this is) those aren't reasonable assumptions to make! > > ... when one is working with ugly code, sometimes it can be very > > difficult to refactor complicated bits out so that they can be > > easily overridden by tests > > Yeah, you have a bunch of ugly code and the best way is to make it > even more ugly by making it aware of the testing. Sometimes. At least in the short term. > I've never seen any requirement that says: "when testing, don't do > this and that". Me neither. But I've seen lots of requirements that don't make any mention of testing at all. And lots of code that doesn't even have any requirements about anything. But suppose some code does a bunch of processing and then finally interacts with the outside world in some way (sending an e-mail, completing a financial transaction, controllig a robot) during testing it may be readily apparent that it would be disruptive to have that action occur. > I'm actually wondering if code which has knowledge of it being tested > is testable at all! well ' it's not. because you can never test the > "send mail" feature. Sure. But you can at least test everything else, all the processing up to the point which sends the mail. And you could in the test environment put the data that would be in the mail somewhere else, which can be tested, so that just leaves the actual mail transport being untested -- and that's probably being done with a Cpan module or something which has been tested elsewhere. > I'm sure you're not inventing this but one actually made the code > worse whith that kind of hack. IMHO, just enabling this kind of code > is going against everything you want to achive, testable applications. No, it's enabling you to test other code that is near an action which absolutely cannot be run in a test environment, thereby making more of the application testable. Smylers
Re: CPAN code searches
Michael G Schwern writes: > A few people have asked how I do my CPAN scans. I keep a minicpan > handy and have a little script called "grep_cpan" ... > > http://schwern.org/src/grep_cpan 404. But this works: http://www.schwern.org/~schwern/src/grep_cpan Smylers
Re: TAP 2.0
Fergal Daly writes: > didn't we have this debate before? Yes. What's more at the start of this thread Ovid specifically said: There are all sorts of little details there, but basically, got/expected (or whatever names are settled on) are to be free-form text. The main question is whether or not those forms are HERE docs or follow a pseudo-YAML convention). That is, he accepts that the names aren't final but what he wants opinions on here is the format[*0], not the name. Please let's not repeat things said last time in the name debate; the archive is available for anybody who wishes to see what the various arguments are. [*0] Talking of which, I'm another vote on the no-here-docs side. Which is looking to be the unanimous preference of folks who've answered on this list -- obviously whichever of Schwern or Ovid came up with the here-doc syntax has got very peculiar tastes! Smylers
Re: A Suitable Iterator for TAPx::Parser
Ovid writes: > From: Shlomi Fish <[EMAIL PROTECTED]> > > > H you may wish to differentiate between #2 and #3 by saying > > that a filename is passed as a plain string, while a string is > > passed by taking a reference to it. That's what Template Toolkit and > > other modules are doing. > > Good call. That's a common enough idiom that I think it will work > fine. Yes, but make sure you do it right. I've been caught out by passing something like a Path::Class::File object, which stringifies just fine as a file path, to modules like this if only they'd just treat it as a string -- but instead overjealous checking spots that it's a reference and declines to stringify it. If you get a reference to a blessed object and that object has overloaded stringification then please just treat it as a string, not a reference. Smylers
Re: Breaking compatability with Test::Harness and friends?
Fergal Daly writes: > On 17/09/06, chromatic <[EMAIL PROTECTED]> wrote: > > > I'm uncomfortable effectively recommending that people *not* use a > > module until someone deliberately reports "Hey, it works better than > > you think on this platform!" to the author and the author releases a > > completely new version to un-TODO those tests. > > having thought about it a bit more, I don't even accept it as a > legitimate use of TODO. If you don't support a platform (or you don't > support platforms that do X) then you test for X and then SKIP. But that involves the module author having detailed knowledge about platforms to which she doesn't have access. It seems entirely plausible that an author wishes to add features A and B to a module, and has a go at it. She codes A and it works fine. Her initial attempt at B happens to include something platform-specific in it, but she hasn't yet tracked down what. Or perhaps she merely _suspects_ there are some platform-specific bits in there: the code runs fine on her computer but she's some vague recollection that something she used won't work on at least some other OSes; she decides she needs to read up on best practices in this area. However there are users clamouring for feature A urgently, so she makes a release before she's completely done feature B; in other words B is still to do, so she marks the B tests as such. It may happen that some of B's tests pass on some platforms, because her initial non-rigorous code does get the correct answer in some circumstances; it just isn't yet in a position where the author would claim it gets the right answer all the time on all platforms. So it doesn't follow that those currently passing tests are a problem. > So while I don't think I'd vote for stopping the install because of an > expected pass, I don't think it's ever a good sign. They aren't a good sign. But they aren't (necessarily) a bad sign either. > The idea of something working "better" than than the author expected > is a bit dubious. Sure -- it possibly happened by chance. Which is why the result usually isn't interesting. Smylers
Re: TAP ain't "Test All Perl"
Adrian Howard writes: > Like anybody reads my use.perl journal :-) Apparently I do -- I saw your journal entry in Bloglines shortly after reading the above message. I seeded my Bloglines subscriptions with those on Planet Perl, so maybe that's why I'm following your feed? If you are on Planet Perl then probably quite a few folks are reading your output. Smylers
Re: Idea for updating the plan
A. Pagaltzis writes: > * David Golden <[EMAIL PROTECTED]> [2006-08-09 05:25]: > > > map ,V :source $VIMRC:filetype detect:exe ":echo 'vimrc > > reloaded'" > > Seems like way too much commotion to me. I just say > > :so % That only works if your .vimrc is still in the current window. It's no use if, for example, you've just used a different instance of Vim to edit .vimrc and now wish to reload it in some other Vim. > Always works, whether I have my .vimrc on that machine or not. Eh? How can you reload a .vimrc that's on a different machine. > That `%` expands to the filename for the active buffer. Very handy to > know about. Indeed: :!cvs dif % :lcd %:h Smylers
Re: $parser->unexpectedly_succeeded;
David Landgren writes:
> Ovid wrote:
>
> > if ( $result->unexpectedly_succeeded ) { ... }
>
> todo_succeeded
That sounds good to me.
Smylers
Re: use Tests; # ?
Fergal Daly writes:
> On 20/07/06, chromatic <[EMAIL PROTECTED]> wrote:
>
> > On Wednesday 19 July 2006 18:10, Fergal Daly wrote:
> >
> > > > On Thu, Jul 20, 2006 at 12:39:11AM +0100, Fergal Daly wrote:
> > > >
> > > > > Simple question. Given this code:
> > > > >
> > > > > sub foo {
> > > > > my $thing;
> > > > > is($thing->x(), "x"); # line 4
> > > > > is($thing->y(), "y");
> > > > > }
> > >
> > > Whether I add $T::B::L or not, I don't get any useful output. If
> > > you disagree please show me your fixed version.
> >
> > sub foo :TestHelper
> > {
> > # this was just a typo; it's not part of my fix
> > my $thing = shift;
> > is($thing->x(), "x", 'checking x attribute'); # line 4
> > is($thing->y(), "y", 'checking y attribute');
> > }
>
> Wrong!!
I'm not so sure.
> Funnily enough your fixed version only provides 1 of these, it will
> tell you that the failure was "checking attribute x" and it will also
> point to line 4 (the line where that test runs). If you had
> incremented $T::B::L at the start then it would have pointed to line 9
> where foo() is called and we'd actually have enough info.
>
> So, you got it wrong when I presume you were being careful. If that's
> not a clear demonstration that something could be a whole lot better
> then I don't know what is.
Note also that the :TestHelper attribute has been added to the sub, to
indicate that it's a sub that helps with testing.
See Aristotle's message earlier in the thread for the suggestion of
using that as a less verbose alternative than having to set the level
variable manually.
Smylers
Re: fetching module version from the command line
Fergal Daly writes: > On 12/07/06, Smylers <[EMAIL PROTECTED]> wrote: > > > I have this one-liner as ~/bin/pmv: > > > > #! /bin/sh > > perl -m$1 -le 'print '$1'->VERSION || die "No VERSION in '$1'\n"' > > These all fail for modules that do interesting things. For example > Test::NoWarnings performs a Test::Builder test in an END block to make > sure there were no warnings. So? It still seems to work, for the purposes of determining what version of the module is loaded: $ pmv Test::NoWarnings 0.082 Smylers
Re: fetching module version from the command line
David Wheeler writes:
> On Jul 12, 2006, at 03:41, Gabor Szabo wrote:
>
> >perl -MModule -e'print $Module::VERSION'
>
> I have this alias set up:
>
> function pv () { perl -M$1 -le "print $1->VERSION"; }
Along similar lines, I have this one-liner as ~/bin/pmv:
#! /bin/sh
perl -m$1 -le 'print '$1'->VERSION || die "No VERSION in '$1'\n"'
> I think that calling ->VERSION is more correct.
So do I. In fact I used to use $VERSION in my script, but changed to
->VERSION after some situation in which it worked and $VERSION didn't.
Sorry, I can't right now remember what that was.
Smylers
Re: TAP diagnostic syntax proposal
demerphq writes: > On 7/12/06, Smylers <[EMAIL PROTECTED]> wrote: > > > David Landgren writes: > > > > > Expected and actual has a long tradition in scientific endeavour, > > And are still sucky as they are different lengths meaning the two > outputs are offset on the screen making it harder to see the failure. True. But we could emit a couple of extra spaces after "actual". Previously I've used "expect" and "actual", which are the same length and close enough to the standard terms that they pass for them. > > They strike me as the teams most intuitively recognizable and least > > open to misinterpretation. > > I think its more important to instantly see the difference between two > simple outputs than it is to use the most absolutely appropriate > terms. True; lining up is important. > Also how can people misinterpret: > > Want: X > Have: Y I don't think they can; that pair certainly passes the "not open to misinterpretation" test. But I claim it isn't as good on the "intuitively recognizable" test. When you first suggested those terms earlier in this thread I did find that I had to slow down when reading them to work out which is which. I had no such slowdown on reading David Landgren's mail. I think it's just that "want" and "have" aren't in widespread use, so it took me a little longer to parse them. But they aren't a terrible choice, and I certainly won't complain if that's what we end up with. Smylers
Re: TAP diagnostic syntax proposal
David Landgren writes: > Expected and actual has a long tradition in scientific endeavour, They strike me as the teams most intuitively recognizable and least open to misinterpretation. Smylers
Re: TAP Namespace Nonproliferation Treaty
Ian Langworth writes: > Ovid: TAP::Parser::Pedantic > > Schwern: TAP::Parser::Heuristic That's the best idea: nobody claiming that his particular parser is _the_ TAP::Parser (or even _the_ TAPx::Parser), but giving them parallel names with adjectives that hint at ways in which they differ. Even if the hints are obscure it's better than just having arbitrary different names. At least, nobody claim that namespace _yet_. A few years down the line we might've collectively learnt enough that we bless one particular parser as canonical, or perhaps we'll make TAP::Parser be a front end multiplexer for picking which parser to load. > I've always feared /^[A-Z]+x::/ namespaces because I never understood > them. They make sense in some circumstances, where there is a well defined project or system which itself is spread across several modules, and other folk wish to contribute modules which are not part of that system but are extensions to it. But ... I don't think that applies here; it isn't that there's a "core" Tap system and a clear distinction between that and extensions to it, and I think it would just get confusing trying to remember which are TAP:: and which are TAPx:: modules. Smylers
Re: CPANTS is not a game.
Michael G Schwern writes: > There's a problem. CPANTS is not a game. If you make it a game, the > system does not work. Hi there. I made a similarish point on this list about a year ago, to which you replied: http://groups.google.co.uk/[EMAIL PROTECTED] Your reply included: Finally, the scoreboard does have a purpose. Part of the original idea of CPANTS was to provide an automated checklist for a good distribution. ... Then, if this were a web page, the author could just click on that to get an explaination of why this is a Good Thing and what they can do to fix it. ... How do you get authors to actually look at the CPANTS information and make corrections? Well, we like competition. Make it a game! So it was you -- or somebody impersonating you on this list -- who managed to persuade me that actually Cpants being a game was a good thing! Smylers
Re: Binary distributions
On January 29th Tyler MacDonald wrote: > Sébastien Aperghis-Tramoni <[EMAIL PROTECTED]> wrote: > > > Did anybody here have played with CPANPLUS::Dist::Deb? > > http://search.cpan.org/dist/CPANPLUS-Dist-Deb/ > > > > Believing its documentation, it should build a valid Debian package > > and take care of its dependencies (dunno if that means just listing > > or actually adding them in the package). > > This excited me, but even with all the debian helper utils installed > and CPANPLUS on default settings running as root, it crashes not being > able to find the right dist_cpan object out of CPANPLUS. :-( For what it's worth, this now works for me, on Ubuntu 5.10. I had to install the March release of CPANPLUS (so it isn't surprising that it didn't work in January!). Note the version of CPANPLUS on debian.pkgs.cpan.org isn't recent enough -- so you have to go outside of Apt and install CPANPLUS manually. So if anybody is wanting .deb packages of Cpan distributions, this does seem to be a reasonable way of doing it, and without having to know anything about the internals or format of either sort of packages. Basically you just need to do: $ cpan2dist --format CPANPLUS::Dist::Deb Crypt::GeneratePassword then wait for cpan-libcrypt-generatepassword-perl_0.03-1_all.deb to spring into existence. Smylers
Re: Non-Perl TAP implementations
Geoffrey Young writes: > Adam Kennedy wrote: > > > That is irrelevant. You put something into CPAN, get massive numbers > > of people using it, and leave it alone and have it remain stable for > > 4 years, it becomes an API whether you wanted it to be or not :) > > really? I'm with Adam on this one. > so I use an age old (but undocumented) feature of Config, then Config > changes and it's _not_ my fault?[1] > > [1] http://dev.perl.org/perl5/list-summaries/2005/20051003.html But there Schwern invoked Tom Christian: You are wicked and wrong to have broken inside and peeked at the implementation and then relied upon it. -- tchrist in <[EMAIL PROTECTED]> Note that's specifically to do with peeking inside. If you don't peek inside some code but merely observe its output, and its output always behaves in a particular way, then relying on that output is not the same as relying on implementation details. Smylers
Re: Module requirements
David Cantrell writes: > rsnapshot (for example) has its own code for traversing a directory > tree, its own cut-down Memoize, and probably a few others that I've > not found yet. > > That said, I don't want to see those things go into the core, because > I'm in the "the core is too big already" camp. Um, surely File::Find and Memoize are already in the core? Smylers
Re: getting round Test::More test formatting trickiness
Dr Bean writes:
> I've gotten comfortable with Test::More conventions, but it's
> difficult in my editor to really quickly create lots of tests.
Which editor is that?
> is($o->index('You'), 1, 'objects index 1');
> isnt($o->index(1), 1, 'objects index 2');
> isnt($o->index(2), 2, 'objects index 2');
> is($o->index($t), 3, 'objects index 3');
>
> Incrementing numbers and substituting words and letters takes
> time
If your editor happens to be Vim then be aware that Ctrl+A adds 1 to the
number currently under the cursor, or the next number on the current
line if the cursor isn't on a number.
That means that if you press Enter to move on to a line (so the cursor
is at the start of it) Ctrl+A will increment the first number (or if
you're already on the line, press 0 first to move to the start of it).
And if there's a second number you wish to increment on the same line
then simply press l to move to the right of the first number (which the
cursor will have moved to on pressing Ctrl+A the first time) and then
press Ctrl+A again.
If you're doing this a lot then you can use q to record a keyboard macro
for processing one line and moving on to the next, then it's just a case
of holding down the @ key to do as many lines as you want.
Smylers
Re: Surprising use_ok false positive
Chris Dolan writes:
> The problem is that I don't know how to distinguish between a load
> failure or a compile failure.
In this particular case you can get the failure by removing Bar from
%INC before checking that it loads:
use_ok('Foo');
delete $INC{'Bar.pm'};
use_ok('Bar');
In general that seems a reasonable thing for use_ok to do itself: if you
want to test that the specified module can actually load right now then
you don't the results to be contaminated with what might've gone before.
Smylers
Re: Best practice for version control of locally installed CPAN modules
Michael Peters writes: > Krang keeps a local copy of all of the CPAN modules it uses in it's > source repo. Also, each module is installed locally to krang so that > it can be installed without affecting an existing installation. ... We've used software that does things like that; I think the OTRS ticketing system is one of them. > the benefits are pretty big: ... Never having to worry about this > application screwing with some other application. ... But there are also downsides. If a bug or security problem is found with one of those modules, it won't be addressed by us upgrading the Cpan module in the standard location, because the app has its own copy. We also got bitten by deploying such an app (without realizing that it was done in this way) using mod_perl, on the same Apache server (but different virtual host) as some of our other things. The app's old versions of some Cpan modules were loaded into Apache instead of the up-to-date ones installed in the usual system directories, which managed to break some of our code; discovering exactly what was wrong was painful -- we didn't expect an app to pollute our Apache with older versions of Cpan modules. Bundling all dependencies with an app makes installation easier for end users who don't even know what Perl is, but harder for those who are comfortable with Cpan and the like and are used to things being done in standard ways. My preferred way of getting such an app would be as a deb package of just the app, but with dependencies on any required modules (and with debs of those modules if they aren't already available), but I realize that not everybody is running an OS which uses debs so this isn't a sufficient solution. Smylers
Re: Binary distributions
Offer Kaye writes: > I see what you mean... what threw me off was that [List::Util and > Scalar::Util are] not listed under: > http://search.cpan.org/dist/perl-5.8.8/ Well spotted! List/Util.pm (including pod) is here: http://search.cpan.org/src/NWCLARK/perl-5.8.8/ext/List/Util/lib/List/Util.pm Math::Complex (for instance) is included in the above list; Math/Complex.pm is here: http://search.cpan.org/src/NWCLARK/perl-5.8.8/lib/Math/Complex.pm So it seems the extra level of subdirectories are causing List::Util (and a whole bunch of other modules) not to show up in the main perl dist page. Is Cpan Search's heuristic for what gets included documented anywhere? In other words, is it that perl5.8.8 is failing in some way to meet some published criteria for distributions to have modules listed; or is it that no such spec exists and therefore perl5.8.8's dist is reasonable, it's just that Cpan Search isn't sufficiently thorough in looking for places where modules might be hiding? I note that Kobes's Search doesn't have List::Util listed on the Perl distro page either: http://cpan.uwinnipeg.ca/dist/perl The code behind that is available, so we can presumably look to see what the heuristic is ... Smylers
Re: Binary distributions
Offer Kaye writes: > On 2/5/06, Offer Kaye wrote: > > > [http://ppm.activestate.com/BuildStatus/5.8-windows/windows-5.8/Scalar-List-Util-1.15.txt > > > Something funky here... Last night I looked at "Scalar-List-Util"... > but the correct name as Tyler said is "Scalar-List-Utils", with an "s" > at the end. If you have a look at the Backpan directory of Graham Barr (the maintainer of Scalar-List-Utils) you'll see that mostly he's called the distribution Scalar-List-Utils-version.tar.gz, but in May last year version 1.15 was released as Scalar-List-Util-1.15.tar.gz, without the "s". This shouldn't matter: Perl dependencies are on modules, not on distributions. And the names of the modules didn't change when the package was named differently. If this is confusing ActiveState then that sounds like a bug in their system. Smylers
Re: [Module::Build] [RFC] author tests
Tyler MacDonald writes: > Chris Dolan <[EMAIL PROTECTED]> wrote: > > [lots of author test examples, including:] > > > * versionsync.t - Checks that the $VERSION is the same in all bin/* > > and *.pm files. This test is pointless after release, since it's > > already been tested before release > > * pod-coverage.t - Checks POD completeness. This test is pointless > > after release, since it's already been tested before release > > ... since there's absolutely no value in these types of tests for > anybody else except the module author, there's no real point in having > a convention, just stick 'em anywhere that the make/buildfiles will > ignore them. That premise is untrue. Some module authors sync the version numbers of all files (like Chris apparently does); some prefer only to bump the version number in a file if that particular file has changed. If I'm sending a patch to a module author I sometimes try to make it be a complete patch for the distro -- including the code, the docs, updating the change-log, and so on. And hopefully I'd run the tests before submitting the patch. So the inclusion of versionsync.t would clue me in that Chris wanted all the version numbers changing, and I could do that in the patch, thereby reducing the work Chris has to do to accept my patch and increasing the chance of me getting the feature I wanted included. Or perhaps as a user of a module I think the docs could do with a little improvement. Before submitting a patch it'd be useful to run pod-coverage.t to check that I haven't missed anything or made matters worse. Remember that this is software with the source available -- so there isn't a strict author v user distinction: any user can suddenly elect to modify the code and become an author (even if only for modifying a private version). Smylers
Re: Module::Install (Was: YAML and Makefile.PL)
Adam Kennedy writes: [Tels writes:] > > if there is a problem with Module::Install, you have to update all > > your dists with the new version > > But if there is a problem with EU::MM or Module::Build, you have to > update every installation in the entire world with the new version. Not necessarily; for example if the problem is just with, say, Cygwin then only Cygwin users need to update their installations -- something which they have control over, and only need to do once for all module installations to work again. > Module::Install is more about containing the problem. If there's a bug > related to Module::Install, at least you only have to fix the problem > in one place. And the number of bugs that require a rerelease is > falling, most of the recent versions have fixed things like > cygwin-specific bugs ... So a module author who doesn't use Cygwin might not care enough to bother doing a new dist of my module just for a Cygwin fix in Module::Install. But a Cygwin user needs all of the modules she uses to have been updated like this, otherwise some modules are no longer installable and it's hard for her to remedy the situation -- certainly much more effort than simply upgrading Module::Build locally. And even if all authors did upgrade all their distributions with Module::Install changes, that would still be less than ideal for the amount of noise that it creates in things like Cpan recent changes listings. If a bug is only in one place, it only needs fixing in one place ... Smylers
Re: First (developers) Release of Test::Shlomif::Harness
Shlomi Fish writes:
> On Wednesday 02 November 2005 08:23, Michael G Schwern wrote:
>
> > Given that the existing interface is pretty simple, I really don't
> > see why one has to completely fork the code base just to put on a
> > new interface. Surely any new code with a fancy new interface can
> > continue to emulate the old one.
>
> Well, here is my beef with it:
>
> 1. It's not an object ...
> 2. In Test::Harness :
> my($tot, $failedtests) = _run_all_tests(@tests);
> _show_results($tot, $failedtests);
> my $ok = _all_ok($tot);
> Vs. mine:
> my($failedtests) =
> $self->_run_all_tests();
> $self->_show_results();
> my $ok = $self->_all_ok();
> 3. Various functions return flat hashes instead of hash references.
> 4. Many things are accessed by a field ($ref->{'field'}) instead of by
> accessor ($ref->field()).
> 5. Various global variables instead of instance variables.
Those show why your interface is better, which isn't being disputed.
But it isn't directly addressing why some kind of emulation layer can't
be provided -- so that somebody can call a function using the old, bad
interface, but internally that function just creates a 'default' object
and then uses your nice new shiny interface.
Of course it won't be possible as a user to do as much like that
compared with using your interface properly -- but that's OK, so long as
existing uses are supported. The idea is that new code should be
written to use your interface, but existing code can be shoehorned into
working with it somehow.
> 6. I had done a lot of refactoring and revamp and broke a lot of the
> interface.
Sure; that's quite reasonable. But again, it doesn't preclude
compatibility now being added.
Best wishes
Smylers
--
May God bless us with enough foolishness to believe that we can make a
difference in this world, so that we can do what others claim cannot be done.
Re: Test::More & Test::Builder::Tester
Michael G Schwern writes: > I'm absorbing Test::Builder::Tester into the Test-Simple distribution. Thank you for doing this -- a practical way of sorting out the situation. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Plugging CPANTS
Thomas Klausner writes: > CPANTS is an acronym for CPAN Testing Service. That would be better simplified to: CPANTS is the CPAN Testing Service. -- cos it's pretty obvious where the name comes from, and its being an acronym is hardly the most important thing about it. But if you really want to talk about it being an acronym, it should be: "CPANTS" is an acronym for "CPAN Testing Service". -- cos there you're talking about the words rather than about the entities. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: New kwalitee test, has_changes
Thomas Klausner writes: > On Thu, Sep 15, 2005 at 11:52:00AM +1000, Adam Kennedy wrote: > > > Rather than do any additional exploding, I'd like to propose the > > additional kwalitee test "has_changes". I've noticed that a > > percentage (5-10%) of dists don't have a changes file, so it can be > > hard to know whether it's worth upgrading, or more importantly which > > version to add dependencies for. > > I like it So do I. But I think the name is misleading. When I read the subject and first two sentences of Adam's proposal I thought has changes would be a test for actually having changes, rather than for having a file which lists changes. Calling the test has_changelog would be much better, since that makes it obvious was is being tested. > For a start, I will only check for a file called /changes/i. After the > run, we'll see how many percent use it and try to find a regex that > matches more dists. As others have pointed out, ChangeLog is also popular and given special treatment by Cpan Search; please allow CHANGES or ChangeLog. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Sub::Uplevel
Ovid writes: > Guess what the following modules all have in common (aside from the > fact that I wrote them)? > > AI::NeuralNet::Simple > AI::Prolog > Games::Maze::FirstPerson > > All of them have failed at one time or another because the target > computer didn't have Sub::Uplevel installed. I'm going to have to > track down what's really causing that, Sounds like a cunning plan by the Sub::Uplevel author to get you to add that module as a prereq for all yours, thereby increasing his kwalitee. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Devel::Cover Problem: testing || for a default value.
Michael G Schwern writes: > On Tue, Jul 12, 2005 at 07:45:35AM +0000, Smylers wrote: > > > A good way of putting assumptions into code is with (Michael's > > excellent) Carp::Assert: > > > > assert $p || $q, 'Either $p or $q must be supplied' if DEBUG; > > > While I get where you're going, I don't see how this could be made to > work. Fair enough; I'm prepared to believe you on that. > What, exactly, is Devel::Cover supposed to do with those assert > statements? I've no idea how Devel::Cover cover manages to do what it does. I suppose I must've been thinking something along the lines of if it can react to a specially formatted comment (urgh!) then it can react to an assert statement. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: AnnoCPAN and a wiki POD idea
Ivan Tubert-Brohman writes: > I'm happy to listen to your suggestions; AnnoCPAN is a work in > progress and I'm still adding features and fixing bugs. Thanks Ivan for your work on this. > I'm also considering the possibility of sending automated emails, > similar to what rt.cpan.org does. However, I was still unsure about > whether to do it opt-in or opt-out (or no-option ;-). Opt-out (or no-option) would irritate far too many people (not necessarily rationally, but it would). But people can only opt in if they know it exists to opt in to. Perhaps you could send one mail once, the first time each author has a note added (or as soon as the mail feature is deployed, for authors that already have notes) letting her know about the feature and inviting her to opt in to receive further notifications. Doubtless there will be some people who complain about receiving even this single mail, but it should cause less grief. > There is a (yet unused) mailing list, where we could discuss AnnoCPAN > in more detail if it's considered off-topic for this perl-qa. I probably don't care about AnnoCpan enough to bother to subscribe to yet another mailing list, but I'd be happy to read about it here -- or possibly on module-authors, which might be even more on-topic than this list. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Devel::Cover Problem: testing || for a default value.
Michael G Schwern writes:
> "my $foo = $p || $q" is not boolean. I'm not even sure you can call
> it "pseudo-boolean" without understanding the surrounding code. How
> do you know that $q can never be false?
So we want some way of annotating the code which will let Devel::Cover
know that you're assuming that $p and $q won't both be false.
Devel::Cover wouldn't be the only beneficiary of having such an
assumption explicitly stated: other humans having to read the code might
appreciate it too.
I can see why having 'inline' directives specifically for Devel::Cover
could be seen as ugly, but if instead they are general-purpose
assumptions then it's obviously better to have them next to the code
that's relying on the assumption than in some other file.
A good way of putting assumptions into code is with (Michael's
excellent) Carp::Assert:
assert $p || $q, 'Either $p or $q must be supplied' if DEBUG;
That improves your code, makes it easier for other people who have to
deal with it, and should be enough to keep Devel::Cover happy.
> The other examples in the ticket play out the same way:
>
> bless {}, ref $class || $class;
>
> $class being false would be quite the error.
Ditto, so you could make the assumption explicit with an assertion. Or
perhaps something like Params::Validate would be better.
Smylers
--
May God bless us with enough foolishness to believe that we can make a
difference in this world, so that we can do what others claim cannot be done.
Re: AnnoCPAN and a wiki POD idea
Adam Kennedy writes: > Adrian Howard wrote: > > > > http://www.annocpan.org/~ADAMK/ > Nope... > > PPI (1 notes; latest 2005-07-04 ) Yup. > Click on PPI... > > List of 9 versions of PPI, no information on the notes. Given that one of the advertised features of AnnoCpan is how it automatically carries notes over from one version of the docs to another, you know you only have to click on the most recent one to get the note. > Bzzz... > > bug? :) Seems to work to me. Notice that on the authors page the note is marked as being in the PPI distribution, which doesn't necessarily mean it's in the PPI module. Once you're on the PPI page it can be seen that it's actually in PPI::Token::Symbol, but you have to scroll down to get that far (or even to realize that the information is provided at all, since the modules that fit on the first screenful don't give much clue that the information would be provided if they did have any notes.) But the scrolling is only necessary on distros containing large numbers of modules, and PPI is quite unrepresentative in that respect! Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Fwd: [demerphq@gmail.com: Re: fixing is_deeply]
demerphq writes: > Well that says there are two different behaviours that people expect. > They are exclusive. Yes. We all want to do the least surprising thing, but it seems different people are surprised by different things; whichever behaviour is implemented some people are going to suffer unexpected results. If the number of people expecting each behaviour is similar then there isn't much we can do about that. If it happens that one expectation is held by a small number of people then it might make sense for them to be the unlucky ones. I'm obviously only a single data point, and I don't claim my expectations match anybody else's; I'm quite prepared to accept that I'm in the set of "unlucky ones". > i think youve missed the point here. > > > > $a3=[$x,$x]; > > > $a4=[$y,$z]; > > > > ... the data structures in each case are the same. > > No, the $x hash is present twice. Thus altering the $x hash and the $y > hash equivelently results in what we all agree are different > structures. Im saying that if that is possible then the originals > structures were different in the first place. Ah, I see what you mean. Yes, I was missing the point -- thank you for your patience and explaining it so well. Finally grasping that however hasn't changed my expectations. > The argument being made goes something like this: > > Because both arrays index 0 and 1 reference empty hashes they are > equivelent. Reading the data from either of those data-structures, as they are at the moment, they can be used interchangeably. But, as you've pointed out, once you've written to them they may no longer be interchangeable. > If we were going to be consistant with our logic on this then we > should say that > > [1,1] and [1,2] > > are the _same_ because both arrays index 0 and 1 contain numbers. Why > do we care if they are they same numbers or not here, but we dont care > when they are hashes? Clearly reading the data from those arrays yields different results; they are interchangeable for neither reading nor writing. Surely you can distinguish between structures which read the same as each other and those that don't (even if you don't consider that a useful distinction to make)? > Like I said. If I have two structures that are deep copies of each > other I should be able to alter both in an equivelent way and still > have them equivelent. That's the crux of the matter. It's plausible to consider structures equivalent if they read the same; equally it's plausible to do so if they behave the same when altered. Both are valid things to do, and consistent within themselves. And both appear to be things that people consider deep comparisons. But obviously each of them will not do in a situation where you wanted t'other one. > > > is_deeply should be reliable and do what its name says. > > > > I think everybody's in agreement about that! There's just the > > matter of agreeing what its name does say ... > > No. there is no agreement about it. You seriously think that anybody here is purposefully advocating a position where the function should be unreliable or not do what it says? It transpires that people have different expectations about what a deep comparison entails; but I don't think anybody is considering the 2 possible behaviours, thinking that the term "deep comparison" applies to one of them and then suggesting that is_deeply should use the other one! > > > The fact is it wouldnt be too difficult to support both [behaviours] > > > off of the same code base. > > > > Then we're still in the position of arguing which should be the default > > behaviour ... > > Yes. And I posit it should be the one that is most consistant. That was my point -- that even if we agree to have a flag, it doesn't help the debate at all, cos all of the original arguments still apply. > And the existing logic is not consistant. I reckon they're both consistent, just different. Hmmm. I see we aren't even "consistent" on how to spell that particular word! I can't be bothered to look it up to see which of us is right (or whether both are valid) ... Smylers
Re: Fwd: [EMAIL PROTECTED]: Re: fixing is_deeply]
demerphq writes:
> On 7/1/05, Michael G Schwern <[EMAIL PROTECTED]> wrote:
>
> > ... I'm of the opinion that is_deeply() is currently doing the right
> > thing ... Largely it comes down to the Principle of Least Surprise.
>
> I cant agree with this analysis. If you go down this route surprise
> abounds.
>
> is_deeply($x,$y); #ok
>
> $x->[0]{a}=1;
> $y->[0]{a}=1;
>
> is_deeply($x,$y); #Surprise!
For those of us not following along too closely (and without the right
stuff installed to test the above line right now), it isn't clear what
the answer to the above is -- since it's the thing being disputed and
there are apparently arguments on each side, either result is of course
possible -- that you find surprising.
$x and $y look to contain the same data when looked at deeply to be, so
I'd expect them to compare the same; I'd be surprised if they didn't,
which I think means I disagree with you.
> Thats a MUCH bigger surprise IMO. And a fatal one for anybody really
> relying on is_deeply.
I'd say exactly the same about it comparing them differently.
> > In fact, I would even argue that this violates some of the black
> > boxness of the test. In much the same way that we don't check to
> > see if a given piece of data is blessed, tied or overloaded, we
> > wouldn't care if we have a repeating reference or two different
> > references to equivalent data.
>
> But they COMPLETELY change the datastructure. Utterly and totally.
Not necessarily. It depends on what you're using the data for.
> The question you have to ask yourself is why should a reference be
> treated different from any other value? It is a VALUE.
Except it isn't. Or at least, not all the time: it depends how you wish
to look at it. If you just consider a reference to be a value
(effectively a pointer, a memory address) then you aren't examining a
data structure _deeply_; you're just doing a _shallow_ comparision of it
as a reference.
> IE: why should [tweaked as per later mail]
>
> $x=1;$y=1;$z=2;
>
> $a1=[$x,$x];
> $a2=[$y,$z];
>
> $a1 and $a2 are different but
Clearly, because you've assigned different values into $x and $z. If
you'd assigned the same values into each of them then they should
compare as the same:
$x=1;$y=1;$z=1;
> $x={};$y={};$z={};
>
> $a3=[$x,$x];
> $a4=[$y,$z];
>
> $a3 and $a4 are not?
In this case you've assigned the same value -- an empty hash ref -- to
each of $x and $z, so the data structures in each case are the same.
> Isn't the situation identical?
No. Just look at the source code:
$x=1;$y=1;$z=2;
$x={};$y={};$z={};
A system in which the 1st of those lines puts something different in the
variables but in which the 2nd puts the same in each variable seems
entirely plausible to me.
> ... the refs turn into their addresses and you will consider them
> different? This just doesnt make sense.
Yes it does, cos once you're looking at the value of refs that's become
a shallow copy -- you're no longer bothered by what's inside them.
Going back to your first example:
> $x=1;$y=1;$z=2;
> $a1=[$x,$x];
> $a2=[$y,$z];
$a1 and $a2 are different, and you seem happy with that. Suppose it
were instead:
$a1=[1, 1];
$a2=[1, 1];
Would you be happy for $a1 and $a2 now to compare the same? If not then
obviously you never need to use is_deeply cos you could just use ==
instead (and treat the array-refs as values). If you are happy for them
to compare the same, despite being different anon array-refs in memory,
then why aren't you happy for the same rule to apply nested within those
array-refs?
To me 'deeply' implies recursing as deep as the data structure goes, not
that there's a special rule for the top-level that's treated differently
from the others.
> is_deeply should be reliable and do what its name says.
I think everybody's in agreement about that! There's just the matter of
agreeing what its name does say ...
> The fact is it wouldnt be too difficult to support both [behaviours]
> off of the same code base.
Then we're still in the position of arguing which should be the default
behaviour ...
Smylers
--
May God bless us with enough foolishness to believe that we can make a
difference in this world, so that we can do what others claim cannot be done.
Re: is_deeply() and code refs
David Landgren writes: > Michael Schwern wrote at the beginning of this thread: > > > What it *shouldn't* do is what Test.pm does, namely execute the > > code ref and compare the values returned. It would just compare > > the refernces. > > Why should it not do that? Is this because of subs with side effects? > Isn't that more an issue of "Doctor, it hurts when I hit my knee with a > hammer"? Also there are subs that do different things which might happen to return the same value for particular input parameters. Talking of which, presumably the only way to invoke the code-ref is without any parameters, which might be completely inappropriate. Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Kwalitee and has_test_*
chromatic writes: > On Sat, 2005-04-16 at 20:59 -0500, Andy Lester wrote: > > > And the more the better! > > Well sure. Two-space indent is clearly better than one-space indent, > and four-space is at least twice as good as that. Negative quality for anybody who includes a literal tab character anywhere in the distro's source! Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Kwalitee and has_test_*
Adam Kennedy writes: > Michael Graham wrote: > > > Another good reason to ship all of your development tests with code > > is that it makes it easer for users to submit patches with tests. > > Or to fork your code and retain all your development tools and > > methods. > > Perl::MinimumVersion, which doesn't exist yet, could check that the > version a module says it needs is higher than what Perl::MinimumVersion > can work out based on syntax alone. > > And it uses PPI... all 55 classes of it... which uses Class::Autouse, > which uses Class::Inspector, and prefork.pm, and Scalar::Util and > List::Util, oh and List::MoreUtils and a few other bits and pieces. > > I'm not going to push that giant nest of dependencies on people just so > they can install Chart::Math::Axis... You don't have to -- have a test which depends on Perl::MinimumVersion, but which skips itself entirely if Perl::MinimumVersion isn't installed. Then anybody without all those dependencies isn't inconvenienced in the slightest -- but anybody who wants to patch the module can see the test's existence and choose to go to the effort of installing it if she so desires. (Note I'm not specifically agreeing with the point that shipping dev tests makes sense -- I think it's fine for some authors to choose to do so and some to choose not to; I'm merely disagreeing with the suggestion that shipping a dev test _necessarily_ imposes a burden on mere users of the module.) Smylers -- May God bless us with enough foolishness to believe that we can make a difference in this world, so that we can do what others claim cannot be done.
Re: Kwalitee and has_test_*
Adam Kennedy writes:
> Christopher H. Laco wrote:
>
> > Tony Bowden wrote:
> >
> > > What's the difference to you between me shipping a a .t file that
> > > uses Pod::Coverage, or by having an internal system that uses
> > > Devel::Cover in a mode that makes sure I have 100% coverage on
> > > everything, including POD, or even if I hire a team of Benedictine
> > > Monks to peruse my code and look for problems?
> > >
> > > The only thing that should matter to you is whether the Pod
> > > coverage is adequate, not how that happens.
> >
> > How as a module consumer would I find out that the Pod coverage is
> > adequate again? Why the [unshipped] .t file in this case.
> >
> > The only other way to tell is to a) write my own pod_coverage.t test
> > for someone elses module at install time, or b) hand review all of
> > the pod vs. code. Or CPANTS.
>
> The main point is not so much that you define a measure of quality,
> but that you dictate to everyone the one true way in which they must
> determine it.
I'm completely with Tony and Adam on this particular point: that
TIMTOWTDI applies to checking pod coverage, and it doesn't make sense to
dismiss a label as being of lower quality because it doesn't perform one
particular check in one particular way.
But ...
Remember that we aren't measuring quality, but kwalitee. Kwalitee is
supposed to provide a reasonable indication of quality, so far as that's
possible. So what matters in determining whether a kwalitee heuristic
is appropriate is whether there is a correlation between modules that
pass the heuristic and those that humans would consider to be of high
quality.
(Theoretically) it doesn't actually matter whether the heuristic _a
priori_ makes sense. If it happens to turn out that the particular
string "grrr', @{$_->{$" occurs in many modules that are of high quality
and few that are of low quality, then it happens that looking for the
existence of that string in a distribution's source will provide a
useful indication when assessing the module. It doesn't have to make
sense in order for that to be the case. (Think of the rules that neural
net or Baysian spam detectors come up with for guessing the quality of
e-mail messages.)
I say "theoretically" cos in the case of Cpants kwalitee the rules are
publicly available -- so even if a neural net (or whatever) did come up
with the above substring heuristic, once it's know then authors can game
the system by artificially crowbarring into their modules' sources, at
which point the heuristic loses value.
So while I agree the pod coverage test criterion makes no sense, and
that it's perfectly valid not to distribute such a test, what I think is
more important is whether that criterion works.
In other words, which are the modules of poor quality but with high
kwalitee (and vice versa)? And what can be done to distinguish those
modules from modules of high (low) quality? It may be that removing the
pod coverage test criterion is an answer to that question (or it may
not).
> Why not give a kwalitee point for modules that bundle a test that
> checks for kwalitee?
If it produces a good correlation, then yes, have such a criterion.
Smylers
--
May God bless us with enough foolishness to believe that we can make a
difference in this world, so that we can do what others claim cannot be done.
Re: Why a scoreboard?
David A. Golden writes:
> Andy Lester wrote:
>
> > Why is there a scoreboard? Why do we care about rankings? Why is
> > it necessary to compare one measure to another? What purpose is
> > being served?
>
> Why is there XP on perlmonks? Or Karma on Slashdot?
Indeed, and those also have odd effects: rather than being pure measures
of users' abilities/reputations/whatever, their very existence changes
how some users behave, where they do things specifically to increase XP
rather than because they have intrinsic value.
In the UK the government a few years ago introduced league tables for
hosptials, taking things such as length of waiting lists into account.
This has had ridiculous consequences where a hospital will choose to
postpone a complex operation in favour of several simpler ones, because
the latter will remove more people from the waiting list and improve
their ranking in the league tables!
I know clothes shop workers who were given weekly sales targets, with a
bonus for meeting them. If they were close to the target towards the
end of the week they might buy a few garments themselves to trigger the
bonus, then bring those clothes back for a refund early the following
week. Or if they'd already met the target for one week, they might
start hoarding some sales tags and keeping them to enter into the
computer system the following week, to increase the chance of getting
that bonus too.
The fact that measurements are taking place changes how people behave.
Sometimes this only results in gaming the measurement system
("kwalitee") and actually having a negative affect on the thing that the
measurers really want improving ("quality").
So I largely agree with Andy's point. Except that I don't really mind:
anybody who's getting worked up about the kwalitee score of their
modules is taking kwalitee too seriously, and they deserve what they
get.
> I think the advantage of a scoreboard system is that metrics like this
> are a motivator. Rather than defining a qualitative standard of "good
> module style", CPANTS defines a quantitative standard and measures
> against it, Many programmers may well be motivated to improve their
> metrics, either for personal improvement or through competitive spirit.
Yup, but all the above apply even without a public chart.
> The advantage of a scoreboard is that it provides a peer benchmark,
Depends what "peer" means. For example, here's a journal entry
complaining that is_prereq is worthless because modules with high-level
functionality specifically aimed at being used in scripts rather than by
other modules will never achieve it:
http://use.perl.org/~ethan/journal/23949
I reckon that doesn't matter, and when considering which of several
modules to use, all the candidates are going to be of one type or the
other, so their relative kwalitees are comparable and useful. But the
leaderboard encourages apples-and-oranges comparisions between different
sorts of modules.
The above author is also frustrated that he's getting a non-perfect
score for is_prereq yet there isn't anything he can directly do about
this. That's true, but it shouldn't matter: kwalitee is trying to
approximate the quality of a module, and if a kwalitee-measured factor
has a correlation with quality then the causation of that factor is
obviously irrelevant.
Yet it apparently does matter to some people. The fact that there's a
public chart on which they appear with a lower ranking than others
through something that they can't change probably doesn't help here.
> We can/should debate the metrics, but not the philosophy of
> measurement.
It isn't the measurement that Andy was objecting to, but the
presentation of it as a per-user chart. As far as possible it should be
ensured that the measuring itself doesn't have negative consequences ...
Smylers
