Re: post-YAPC::Europe CPANTS news
Le mardi 12 septembre 2006 à 11:15, Chris Dolan écrivait: On Sep 12, 2006, at 9:24 AM, Salve J Nilsen wrote: Any metric that catches bad things, particularly bad technical things, is going to be just fine. Metrics that try to push good behavior are fraught with trouble, because they start pushing people in odd directions. Do you have an example on this? (Any pointer would be wonderful.) I have two: pod.t and pod_coverage.t. These are pointless to run on an end-user's machine. At best they are redundant to immutable tests already run on the author's machine and just waste processor cycles. At worst they fail and cause false negative test reports. The prevalence of those two tests in CPAN modules is almost entirely due to the influence of CPANTS. At least has_test_pod can be rewritten as no_pod_errors and achieve the same goal (check that the documentation is syntactically correct pod-wise). Since running Test::Pod on all .pm and .pod file doesn't require actually running the code itself, it sound perfectly acceptable for both parties (those who want to check that the pod is correct and those that think that multiple copies of t/pod.t shouldn't clutter CPAN). ... I just had myself motivated enough to write such a metric, when I discovered that Module::CPANTS::Analyse already has a no_pod_errors metric! That makes me wonder about the utility of has_test_pod, since no_pod_errors is even more interesting: we want to give points to people that *have* a correctly written pod, rather than to those who merely try, don't we? -- Philippe BooK Bruhat What everyone wants, nobody gets, What nobody gets, everybody wants. (Moral from Groo The Wanderer #47 (Epic))
Re: post-YAPC::Europe CPANTS news
Adam Kennedy wrote: I'm find with adding an additional environment variable though for the packaging state. But please lets not decide on anything right now, AUTOMATED_TESTING is already a sub-optimal name, I'd rather make sure that the EU::MM, M:B and M:I modules all agreed on a single consistent name before we go and add support for it. Whatever it is, prefix it with PERL_ to avoid conflicts.
Re: post-YAPC::Europe CPANTS news
A. Pagaltzis wrote: Adam Kennedy wrote: On the other hand, the downside with this is that modules could well have URIs that take actions in them, *pulls out HTTP RFC* *starts beating random bad programmers over the head with it* `GET` SHOULD BE SAFE AND IDEMPOTENT! `GET` SHOULD BE SAFE AND IDEMPOTENT! `GET` SHOULD BE SAFE AND IDEMPOTENT! So I have a URL in a module, which *must* be POSTed for it to do whatever it needs to do because it's not intended to be impotent. I arrange matters so that GETting it is forbidden, because GETting it is meaningless. Your GET test then correctly fails, and would be incorrectly recorded as a failure. That will really give me confidence in the quality of CPANTS. -- David Cantrell
Re: post-YAPC::Europe CPANTS news
In article [EMAIL PROTECTED], Chris Dolan [EMAIL PROTECTED] wrote: On Sep 12, 2006, at 9:24 AM, Salve J Nilsen wrote: Any metric that catches bad things, particularly bad technical things, is going to be just fine. Metrics that try to push good behavior are fraught with trouble, because they start pushing people in odd directions. Do you have an example on this? (Any pointer would be wonderful.) I have two: pod.t and pod_coverage.t. These are pointless to run on an end-user's machine. At best they are redundant to immutable tests already run on the author's machine and just waste processor cycles. I've actually discovered POD problems when users run these tests. They aren't immutable because people use different versions of tools and different versions of the various POD modules. With simple fixes, I can make some things readable by people even with old Perl distributions. Having said that, I still find value in them. If the installer watches the tests go by, they see that the documentation is being tested. I hope that gives them a little more confidence in the module. And, since this is open source, I distribute all the source I use to develop the module. That's the idea, isn't it? If the user changes something, they still have all the tests, including one to remind them to document their new function using the proper format. :)
Re: post-YAPC::Europe CPANTS news
David Golden wrote: Adam Kennedy wrote: But with that in mind, I still don't see much point in running them at install-time, so lately I've modified my pod.t test so that it's skip message is now skipped: Author tests not required for installation or the like, and the tests now only run when AUTOMATED_TESTING is on. So now the tests will still get run during CPAN Testers and the like, but regular installation will not be impacted. Personally, I prefer AUTHOR_TESTING and run it as part of my automated release process. I turn on AUTOMATED_TESTING for all the Vanilla builds to skip past things that want user prompts -- I think of AUTOMATED_TESTING as limiting tests to things that are safe for non-attended operation. It would be a surprise to me to find more tests turned on for AUTOMATED_TESTING. Sorry to burst your bubble, but this is already happening. There are modules using AUTOMATED_TESTING to detect automated testing systems and adding extra dependencies, so that additional tests can be run that would be overly onerous for someone doing an ordinary installation to run. I can't point to any off the top of my head, but I think at least one thing I've written adds a dependency on SQLite under AUTOMATED_TESTING so that it can run a whole slew of extra tests, when under ordinary installation circumstances I wouldn't want to force someone to install SQLite just for testing my module. I'm find with adding an additional environment variable though for the packaging state. But please lets not decide on anything right now, AUTOMATED_TESTING is already a sub-optimal name, I'd rather make sure that the EU::MM, M:B and M:I modules all agreed on a single consistent name before we go and add support for it. Adam K
Re: post-YAPC::Europe CPANTS news
Adam Kennedy wrote: Of course some authors don't care about having a community around their software, and some don't consider their CPAN package as important or big enough to warrant a community (despite it probably being licensed with an open source-friendly license). These people are entirely free to continue do nothing. :) Yes, but we've seen what happens once the metrics are created. The natural competitive nature of people comes out and they start doing things just because there's a metric for it. This, to me, is a good thing - ripe for exploitation! :-D There's nothing as cool as letting strangers do good things (like contribute to an open source community) just by allowing them to do what's natural. :) Any metric that catches bad things, particularly bad technical things, is going to be just fine. Metrics that try to push good behavior are fraught with trouble, because they start pushing people in odd directions. Do you have an example on this? (Any pointer would be wonderful.) I think it's important that we take some care about the metrics that get created that encourage people to take good behaviors (as opposed ones that just encourage not-bad behavior). Agreed. Giving good names to new META.yml fields and documenting them in a complete, terse way that isn't open to (mis-) interpretation should be done _before_ any implementation. (Especially when it comes to something that can affect community growth in some significant way. :) Finally, I don't personally see an obvious (causative or otherwise) link between a non-author community support channel, and module Kwalitee (or quality for that matter). CPAN modules are all (?) Free Software/Open Source projects, and with all such projects, the feature, code and documentation quality is in many ways a product of the amount of attention it is given. More attention = better products (Given enough eyeballs, all bugs are shallow, you know). It's obviously not as easy as that - one still needs to attract people that care, have time and enough competency to contribute in a positive way, and then actually allow and motivate these people to contribute in the way they like. Still if we are going to find more people that want to contribute/become part of the community, then we need to do all kinds of marketing - including telling users that if they use Perl, they'll easily find both the useful CPAN software and the communities around that software. How to attract people to the Perl community: 1) Show them Perl and it's community exists 2) Show them what kind of useful options they get from Perl and the community 3) Let them try out the things they want to - easily 4) Show them where to find help when they need it --- WE'RE IMPROVING THIS 5) Show them we'd like their feedback (and bug reports (and patches)) 6) Allow them to give back to/help the community when they feel ready 7) Allow them to become project developers when they feel ready (I'm sure this list is missing some crucial points, but I hope it still makes some sense. :) - Salve -- Salve J. Nilsen salvejn at met dot no / Systems Developer Norwegian Meteorological Institute http://met.no/ Information Technology Department / Section for Development
Re: post-YAPC::Europe CPANTS news
On Sep 12, 2006, at 9:24 AM, Salve J Nilsen wrote: Any metric that catches bad things, particularly bad technical things, is going to be just fine. Metrics that try to push good behavior are fraught with trouble, because they start pushing people in odd directions. Do you have an example on this? (Any pointer would be wonderful.) I have two: pod.t and pod_coverage.t. These are pointless to run on an end-user's machine. At best they are redundant to immutable tests already run on the author's machine and just waste processor cycles. At worst they fail and cause false negative test reports. The prevalence of those two tests in CPAN modules is almost entirely due to the influence of CPANTS. Despite the criticisms above, the CPANTS POD tests have ultimately succeeded: they have convinced authors to do a better job of documenting all methods, or marking private methods as such. I think nobody can argue that the POD tests, in particular, have had a net positive effect on the quality of CPAN. === Now begins a huge digression on encouraging good behavior vs. discouraging bad behavior leading to a recommendation for CPANTS. One flaw in the language of Adam's assertion is that he doesn't properly distinguish the goals and metrics of CPANTS. Discouraging a specific bad behavior is just a way encouraging other unspecified behavior, which could be good or bad. IF FEASIBLE, it's always better to encourage good behavior. The danger is not metrics that encourage good behavior, but instead metrics that encourage a specific good behavior when there are a multitude of equally-valid good behaviors. In that case, discouraging the bad behaviors is the best you can do. I believe that's what Adam was trying to say. I'm going to continue with the specific example of POD Kwalitee. The CPANTS goal is (obviously) to encourage higher quality documentation. However, that's a hard thing for a computer to measure. So, instead we try to discourage specific bad behaviors: POD syntax errors and undocumented subroutines. Let me run through an exercise. In the first step, consider how one would arrive at the need for CPANTS POD tests: Goal: encourage high-quality CPAN packages Assertion: high-quality packages have high-quality documentation Assertion: high-quality documentation is parseable by doc tools Subgoal: discourage invalid POD Measure: Is the POD valid for each module in the package? Assertion: high-quality documentation describes every public subroutine Subgoal: discourage undocumented subs Measure: Does each module in the package have documentation for every public sub? The next step in the exercise becomes how to implement those measures. In the current CPANTS simple proxies are used for those measures. Namely, we assume that if there is a t/*pod.t file present then the pod is valid, and if there is a t/*pod_coverage.t present then all subroutines are documented. Note that my subgoals are stated as discouraging bad behavior. It's always easier to test for failures than successes (case in point: governments usually create laws, not commandments). The CPANTS POD tests, however, check for good behavior (Thou shalt add pod.t to thy package) instead of checking for bad behavior (Thou shalt not include invalid POD in thy module). Wouldn't it be better to just measure the POD validity directly instead of using a proxy for that measurement? As an outsider, I'll guess that CPANTS resorts to the proxies for these reasons, in order of importance: 1) reliability 2) ease of implementation 3) speed of evaluation Certainly, CPANTS wants to avoid false negatives at all costs. It's impact on the community is purely voluntary, so it wants to avoid antagonizing authors. If CPANTS mistakenly says that your module has incomplete POD coverage when you *know* that you have documented every method, you're going to be annoyed. Some authors may decline to participate in CPANTS if they get annoyed enough. So, false negatives are perilous to the success of the entire project. I believe the main reason that CPANTS measures t/*pod*.t existence instead of directly running Test::Pod/Test::Pod::Coverage is that the latter is harder for the author to judge consistently before he/she uploads to CPAN. But, with the improved availability of offline CPANTS analysis (via Module::CPANTS::Analyse), it should be feasible for authors to get rid of more complex false negatives before uploading to CPAN. So, as a technological expedient, CPANTS is encouraging a sub-optimal good behavior (adding t/*pod*.t to CPAN releases) in the process of trying to discourage a bad behavior. To fix this, we need to remove the need for the expedient. That means letting CPANTS perform more complicate analyses and letting authors test those analyses offline exactly as they would be
Re: post-YAPC::Europe CPANTS news
Hi!
On Tue, Sep 12, 2006 at 11:15:50AM -0500, Chris Dolan wrote:
Wow, excellent post!
Some notes:
The next step in the exercise becomes how to implement those
measures. In the current CPANTS simple proxies are used for those
measures. Namely, we assume that if there is a t/*pod.t file present
then the pod is valid, and if there is a t/*pod_coverage.t present
then all subroutines are documented.
In fact, CPANTS actually runs Pod::Simple::Checker on the POD, so the
t/*pod.t check is more or less redunant. If there's a problem with the
POD, a dist won't get points for 'no_pod_errors'. (In the next release
of Module::CPANTS::Analyse the errors reported by Pod::Simple::Checker
will be available on cpants.perl.org)
The problem with pod_coverage is that Pod::Coverage executes the code to
get a list of all functions, and then uses some Pod parsing to check if
they are all covered. But running code is something CPANTS just cannot
do (at least not for all of CPAN (Win32::*, interfaces to various C
libraries, etc).
Thus, I finally get to an action item: CPANTS should encourage
authors to run Module::CPANTS::Analyse offline before uploading to
CPAN. I assert that if we can convince authors to perform more
thorough tests of their packages at author-time, then the quality of
CPAN will improve. And the more closely the metrics match our real
quality goals, the bigger the quality delta we will achieve.
Well, I could add a metric like 'has_test_kwalitee' to check for
t/99_kwalitee.t which uses Test::Kwalitee :-)
Basically, nearly most of what you said describes quite exactly what I
think.
When I took over CPANTS, I was targetting low-hanging fruits (like some
simple-to-check files). Now, we are implementing more complex metrics
that are obviously more meaningfull than simple
does-this-file-exists-stlye metrics.
In theory, we can now remove some of the very simple metrics (that
might not make that much sense), i.e. has_test_pod (no_pod_errors is
much better) and has_test_pod_coverage
The main reason why they're still here is because I do get reports from
people that say that adding these two test files (which they maybe never
heard of before (not everybody hangs around on YAPCs, IRC, use.perl and
perlmonks...)) made them add more/better docs. Which is good.
--
#!/usr/bin/perl http://domm.zsi.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}
Re: post-YAPC::Europe CPANTS news
Gabor Szabo wrote: On 9/13/06, Thomas Klausner [EMAIL PROTECTED] wrote: In theory, we can now remove some of the very simple metrics (that might not make that much sense), i.e. has_test_pod (no_pod_errors is much better) and has_test_pod_coverage The main reason why they're still here is because I do get reports from people that say that adding these two test files (which they maybe never heard of before (not everybody hangs around on YAPCs, IRC, use.perl and perlmonks...)) made them add more/better docs. Which is good. I would not remove them just reduce their weight in the overall rating. They will remind people who don't know about them but will let others achive high kwaletee rating even if they don't have them. I concur. I'd certainly like to see, at the very least, the pod coverage one become an optional metric. And I think Chris clarified my position pretty well. The problem with encouraging a good is that it is almost always a specific good behavior and creates a perfect environment for cargo-culting, exactly the sort of fallout we've seen with the pod and pod_coverage tests. That said, I've have come around a little on the subject of author tests. The one advantage they do have, is that their inclusion means that while I may check POD structure, a pod.t ensures that in the event of someone else taking over the module, these things will continue to be checked. But with that in mind, I still don't see much point in running them at install-time, so lately I've modified my pod.t test so that it's skip message is now skipped: Author tests not required for installation or the like, and the tests now only run when AUTOMATED_TESTING is on. So now the tests will still get run during CPAN Testers and the like, but regular installation will not be impacted. Adam K
Re: post-YAPC::Europe CPANTS news
On Sep 12, 2006, at 6:54 PM, Adam Kennedy wrote: [...] That said, I've have come around a little on the subject of author tests. The one advantage they do have, is that their inclusion means that while I may check POD structure, a pod.t ensures that in the event of someone else taking over the module, these things will continue to be checked. But with that in mind, I still don't see much point in running them at install-time, so lately I've modified my pod.t test so that it's skip message is now skipped: Author tests not required for installation or the like, and the tests now only run when AUTOMATED_TESTING is on. So now the tests will still get run during CPAN Testers and the like, but regular installation will not be impacted. That's fine, but in my opinion, a slightly better solution for author tests is to include them in revision control (i.e. SVN) but to exclude them from the public distro via MANIFEST.SKIP. That's what we do for the Perl::Critic self-compliance tests, for example, which are time-consuming and not relevant to the functionality of Perl::Critic. However, your approach to author-test skipping works well, and is more appropriate for modules which may not have a publicly-exposed revision control system. *shrug* I could probably be convinced that your approach is better. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: post-YAPC::Europe CPANTS news
On Sep 12, 2006, at 9:44 PM, Adam Kennedy wrote: That's fine, but in my opinion, a slightly better solution for author tests is to include them in revision control (i.e. SVN) but to exclude them from the public distro via MANIFEST.SKIP. That's what we do for the Perl::Critic self-compliance tests, for example, which are time-consuming and not relevant to the functionality of Perl::Critic. However, your approach to author- test skipping works well, and is more appropriate for modules which may not have a publicly-exposed revision control system. *shrug* I could probably be convinced that your approach is better. For something highly visible like Perl::Critic that's fine, but what tipped it over the line for me was that for ordinary modules repositories are transient, and can't always be relied on. The only canonical and reliable source we have is what gets uploaded. [...] That's a very good point. I'm convinced. To restate what you've said, CPAN is a better long-term software repository than any other system for open source Perl programmers. Chris -- Chris Dolan, Software Developer, Clotho Advanced Media Inc. 608-294-7900, fax 294-7025, 1435 E Main St, Madison WI 53703 vCard: http://www.chrisdolan.net/ChrisDolan.vcf Clotho Advanced Media, Inc. - Creators of MediaLandscape Software (http://www.media-landscape.com/) and partners in the revolutionary Croquet project (http://www.opencroquet.org/)
Re: post-YAPC::Europe CPANTS news
Gabor Szabo wrote: On 9/7/06, Salve J Nilsen [EMAIL PROTECTED] wrote: Thomas Klausner wrote: Oh, and if you want to join the fun and help a bit, here's a (probably incomplete) list of tasks: - Metrics: [snip] Would the metrics for community support channels that were suggested a while ago be welcome? (The discussion about them sort of died out :-\) [snip] The question then might be if that channel is used. E.g. are there (recent) posts on the forum? How many posts are there? Have the questions been answered? Has the module author blessed the channel (or for that mater decided to point people to another support channel)? Exactly. Having a metric like primary_community_resource: URL (or similar) would at least hint of which forum(s) the author intends to use. This is obviously useful information, since it lets the user inspect the forum(s) without first having to search for them. Of course some authors don't care about having a community around their software, and some don't consider their CPAN package as important or big enough to warrant a community (despite it probably being licensed with an open source-friendly license). These people are entirely free to continue do nothing. :) - Salve -- Salve J. Nilsen salvejn at met dot no / Systems Developer Norwegian Meteorological Institute http://met.no/ Information Technology Department / Section for Development
Re: post-YAPC::Europe CPANTS news
Adam Kennedy wrote: Salve J Nilsen wrote: Thomas Klausner wrote: Oh, and if you want to join the fun and help a bit, here's a (probably incomplete) list of tasks: - Metrics: [snip] Would the metrics for community support channels that were suggested a while ago be welcome? (The discussion about them sort of died out :-\) I think the main issue with this was that it was really only a valid metric for huge modules, and for 90% of the smaller things there wasn't much point. For example, Config::Tiny or Catalyst::Plugin::Some::Random::Small::Plugin. Why? Having such a metric is quite useful even for the smaller moules, IMO. Firstly, it says something about the author's ambitions (I'll be supporting this, I will continue developing features, I accept patches, I'd like to help you use my software). And there's nothing wrong if several tiny modules point to a common mailing list... E.g. that certain Acme::* module authors subscribe to a hypothetical [EMAIL PROTECTED] mailing list. Or that the Catalyst::Plugin::Some::Random::Small::Plugin author tells that she'll monitor irc://irc.perl.org/catalyst for questions... And frankly, I don't think there's a good way to distinguish between should have a community and shouldn't need a community. That's obviously entirely up to the author. What we, the CPAN community can do is urge the authors to consider having and using such a resource, since doing this in general /helps the community/, both in the general sense (showing the world that the CPAN community is easily accessible for outsiders and new users) and in the specific sense (make Perl software easier to use, since support apparently is easily available). On the other hand, what WOULD be interesting, is a check to make sure that the URIs of anything mentioned are still valid. Heh. Yeah, that would be a nice project all by itself. :) So if the META.yml has a URI with a community page or what have you, that the page exists. The same sort of uris_exist could also check URIs in the main documentation. Good idea. :) - Salve -- Salve J. Nilsen salvejn at met dot no / Systems Developer Norwegian Meteorological Institute http://met.no/ Information Technology Department / Section for Development
example metric (was Re: post-YAPC::Europe CPANTS news)
Thomas Klausner wrote: The one advantage of dedicated examples for me is that I can take that example file (mostly downloaded from search.cpan.org), run it, modifiy it, run it etc. Cutting and pasting from the docs works as well, no network required. And its going to be the example for the right version (ie. the one you have installed, not the latest on CPAN). This hardly works with code embedded in the docs, as this code tends to be overly verbose (eg no 'use strict' etc). That's spurious. The quality of the code has nothing to do with whether its in a file or a POD document. In fact, with Test::Inline I can test my example code in the POD.
Re: post-YAPC::Europe CPANTS news
Of course some authors don't care about having a community around their software, and some don't consider their CPAN package as important or big enough to warrant a community (despite it probably being licensed with an open source-friendly license). These people are entirely free to continue do nothing. :) Yes, but we've seen what happens once the metrics are created. The natural competitive nature of people comes out and they start doing things just because there's a metric for it. Any metric that catches bad things, particularly bad technical things, is going to be just fine. Metrics that try to push good behavior are fraught with trouble, because they start pushing people in odd directions. I think it's important that we take some care about the metrics that get created that encourage people to take good behaviors (as opposed ones that just encourage not-bad behavior). Finally, I don't personally see an obvious (causative or otherwise) link between a non-author community support channel, and module Kwalitee (or quality for that matter). Adam K
Re: post-YAPC::Europe CPANTS news
Michael Peters wrote: Adam Kennedy wrote: It might be an interesting idea to also add a dependencies_exist metric, that makes sure that all the dependencies that are declared actually exist in the CPAN. Dunno, could be of dubiously little value, but I just managed to somehow upload something with bad deps that I had installed on my local machine, but that weren't all on CPAN yet... I'm not sure this is such a good idea. There are several examples of modules that rely on other Perl modules that aren't on CPAN. For instance, svk relies on SVN::Mirror which is a part of svn. SWISH::HiLiter needs SWISH::API which is a part of swish-e. There are lots of perl modules tied to various projects that don't exist independently on CPAN. But frankly, if you are uploading modules to CPAN that can't possibly ever be installed with the CPAN client because you have to go install modules from somewhere else first, I'd consider that worth docking a kwalitee point over. 98%* of modules are going to pass this, 1% will fail and encourage more people to upload more things, and 1% will fail due to genuine errors we want to assist authors in catching. On another subject that came up today one one of my modules (specifically the new Test::Object dependency of PPI) it seems like it could be a bad idea to have explicit dependencies on the latest version of a dual-life module. One of the linux distro guys pinged me about Test::Object needing the very latest (CPAN-only) version of Test::Builder, because it means they can't package it properly for the distros without upgrading the main Perl package. Some packaging systems can't handle having the same file in more than one distro it seems. While this might seem like punishing people for depending on things that work, it's another red box to alert you to a possible problem, and the red box will dissapear on it's own when the next version of Perl comes out. I think it's worth considering. Adam K * Not actual percentages
Re: post-YAPC::Europe CPANTS news
Thomas Klausner wrote: Hi! On Thu, Sep 07, 2006 at 10:23:39AM +1000, Adam Kennedy wrote: - has_example I thought we were generally negative on this one, because it would encourage people to spuriously add trivial example directories to their distributions... Yes, but I've recently introduced the concept of 'optional' metrics. Optional metrics are not used to calculate an authors rank in the CPANTS game. Current optional metrics are 'is_prereq' and 'has_examples'. This also makes it possible to get 110% kwalitee at the moment :-) I saw that. The question is how many people are likely to be driven to add it just to get the green mark anyway (even if it doesn't alter the result of the game). Perhaps as an initial fix, we could make the visual distinction seem smaller? For example, instead of white box with squiggle, change it to a grey box with the same level of brightness as the green box? So the difference is reduced to a fairly subtle shade difference? As a secondary point, if we keep it you might also want to include /sample(s)/ and /demo(s)/. Thanks, added - declares_dependencies It might be an interesting idea to also add a dependencies_exist metric, that makes sure that all the dependencies that are declared actually exist in the CPAN. Dunno, could be of dubiously little value, but I just managed to somehow upload something with bad deps that I had installed on my local machine, but that weren't all on CPAN yet... It might also catch possible CPANTS bugs. Good point. I like it then :) - no_open_bugs If we are going to do this, we have to make sure that we DON'T include wishlist bugs in this metric, or it blows the validity all to hell. I was thinking of something like 'no open bugs with severity wishlist (or even unimportant) and more than two weeks old'. 'Taken' bug reports are obviously also not counted. Something along those lines seems good, the more sophisticated the better. The big question is, can you get data exported out of RT with enough resolution to do what you want? Adam K
Re: post-YAPC::Europe CPANTS news
Salve J Nilsen wrote: Thomas Klausner wrote: Oh, and if you want to join the fun and help a bit, here's a (probably incomplete) list of tasks: - Metrics: [snip] Would the metrics for community support channels that were suggested a while ago be welcome? (The discussion about them sort of died out :-\) I think the main issue with this was that it was really only a valid metric for huge modules, and for 90% of the smaller things there wasn't much point. For example, Config::Tiny or Catalyst::Plugin::Some::Random::Small::Plugin. And frankly, I don't think there's a good way to distinguish between should have a community and shouldn't need a community. On the other hand, what WOULD be interesting, is a check to make sure that the URIs of anything mentioned are still valid. So if the META.yml has a URI with a community page or what have you, that the page exists. The same sort of uris_exist could also check URIs in the main documentation. On the other hand, the downside with this is that modules could well have URIs that take actions in them, or aren't meant to be valid... maybe... Adam K
Re: post-YAPC::Europe CPANTS news
On 9/7/06, Salve J Nilsen [EMAIL PROTECTED] wrote: Thomas Klausner wrote: Oh, and if you want to join the fun and help a bit, here's a (probably incomplete) list of tasks: - Metrics: [snip] Would the metrics for community support channels that were suggested a while ago be welcome? (The discussion about them sort of died out :-\) For every uploaded module automatically forum is created on http://www.cpanforum.com/ I know not everyone likes this idea and some others have issues with the current form of CPAN::Forum (yes, I am working on some of these issues) but this basically means that every module *has* a community support channel. The question then might be if that channel is used. E.g. are there (recent) posts on the forum? How many posts are there? Have the questions been answered? Has the module author blessed the channel (or for that mater decided to point people to another support channel)? Oh, I would love a metric that is called has_posted_on_cpan_forum_that_author_is_monitoring_the forum even if it had a shorter name. Gabor
Re: post-YAPC::Europe CPANTS news
On Fri, Sep 08, 2006 at 09:15:47PM +1000, Adam Kennedy wrote: One of the linux distro guys pinged me about Test::Object needing the very latest (CPAN-only) version of Test::Builder, because it means they can't package it properly for the distros without upgrading the main Perl package. Some packaging systems can't handle having the same file in more than one distro it seems. Said maintainers can work round the deficiencies of their packaging systems by breaking out all the dual life modules from the core Perl package, and making them into other packages on which the core Perl package depends. In my opinion. Clearly this might be a bit tricky if the packager of Test::Object isn't the same person as the packager of core Perl, because then the former has to persuade the latter of the need for such a change. Alternatively said maintainers can work round the deficiencies of their packaging system by having another directory tree in @INC ahead of the core library paths, into which they install newer versions of dual life modules. This way no files need to be overwritten on disk, or dual-owned, yet Perl will still pick up the correct (newer) version. Nicholas Clark
Re: post-YAPC::Europe CPANTS news
On 9/8/06, Adam Kennedy [EMAIL PROTECTED] wrote: On another subject that came up today one one of my modules (specifically the new Test::Object dependency of PPI) it seems like it could be a bad idea to have explicit dependencies on the latest version of a dual-life module. One of the linux distro guys pinged me about Test::Object needing the very latest (CPAN-only) version of Test::Builder, because it means they can't package it properly for the distros without upgrading the main Perl package. Some packaging systems can't handle having the same file in more than one distro it seems. I think that this is a general problem that probably requires some pushback towards the packaging system maintainers. The concept that a package can contain numerous parts, some of which may be superseded by later released stand alone parts shouldnt be a surprise to anyone. It happens all the time in every aspect of life. This came up recently for me with regard to EU::MakeMaker and EU::Install. The package EU::Install is contained by both EUMM and EUI. In principle the EUMM package shouldnt install its EU::I unless the existing EU::I is older than the one it contains. OTOH, the EUI package should always be used if one wants to upgrade it alone. I personally dont think that this is a bizarre use case. EUMM needs to have EUI to do its thing, and EUMM needs to have EUI around to do its thing. But they can be updated independently. Such a chicken and egg kind of relationship probably isnt that unusual. The only solution can't be that you have to bundle them together or bundle them all independently. On win32 this is an old problem that is mostly dealt with transparently to the user (except when its done badly in which case things can go horribly wrong). Its common for a package to bundle various items, particularly .dll's, but not install them when it finds that later versions are available from some other source. BTW, im not saying that this is an easy problem to resolve, or that im interested in rushing out and solving it myself. But it seems to me to be one that needs solving. :-) I mean, we are talking about tools here, can you imagine going into a hardward store and finding out that you cant replace a tool from a toolset without replacing the entire toolset? Or conversly that nobody offers a complete toolbox just because then they would be forbidden from supplying the individual tools alone? Anyway, sorry, minor rant. Ill shut up now. :-) Cheers, Yves -- perl -Mre=debug -e /just|another|perl|hacker/
Re: post-YAPC::Europe CPANTS news
Hi!
On Wed, Sep 06, 2006 at 04:28:56PM -0400, Michael G Schwern wrote:
Thanks for the updates, Thomas. And now on with the complaining!
:-)
- has_example
An optional metric that checks if the author included a dir called
'eg|ex|example(s?)' which in turn includes at least on *.pl
IMO examples in an example directory are a detriment, not a bonus.
The one advantage of dedicated examples for me is that I can take that
example file (mostly downloaded from search.cpan.org), run it, modifiy
it, run it etc.
This hardly works with code embedded in the docs, as this code tends to
be overly verbose (eg no 'use strict' etc).
In fact I was only made aware of 'examples' during the discussion here a
while ago. I used examples several times since, as a first step to get
used to a distribution.
So I think they are valuable.
Examples go in the installed docs or they are lost.
The docs could state that there are some ready-to-run examples in that
dir in the distribution.
Additionally, if you're going to keep this thing around, restricting the
examples to .pl files penalizes HTML (Mason or other templating system),
POD and module examples. [1] It would be best to just say that the
directory contains something.
You're right (and several other people that suggested that). I'll change
it in the next run.
--
#!/usr/bin/perl http://domm.zsi.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}
Re: post-YAPC::Europe CPANTS news
Hi!
On Thu, Sep 07, 2006 at 10:23:39AM +1000, Adam Kennedy wrote:
- has_example
I thought we were generally negative on this one, because it would
encourage people to spuriously add trivial example directories to their
distributions...
Yes, but I've recently introduced the concept of 'optional' metrics.
Optional metrics are not used to calculate an authors rank in the CPANTS
game. Current optional metrics are 'is_prereq' and 'has_examples'.
This also makes it possible to get 110% kwalitee at the moment :-)
As a secondary point, if we keep it you might also want to include
/sample(s)/ and /demo(s)/.
Thanks, added
- declares_dependencies
It might be an interesting idea to also add a dependencies_exist
metric, that makes sure that all the dependencies that are declared
actually exist in the CPAN. Dunno, could be of dubiously little value,
but I just managed to somehow upload something with bad deps that I had
installed on my local machine, but that weren't all on CPAN yet...
It might also catch possible CPANTS bugs.
- no_open_bugs
If we are going to do this, we have to make sure that we DON'T include
wishlist bugs in this metric, or it blows the validity all to hell.
I was thinking of something like 'no open bugs with severity wishlist
(or even unimportant) and more than two weeks old'.
'Taken' bug reports are obviously also not counted.
- Signature checking ?
I looked into this a bit more recently, with all the Module::Signature
changes. I think this is a bad idea.
..
I agree
- has_rating ?
Bad idea I think as well, because it will encourage spurious ratings.
This is the same reason that ratings aren't factored into the
search.cpan result order. CPAN Ratings currently has most of it's value
because it doesn't matter much, and so it encourages high quality reviews.
I also do not really like this metirc, I just wrote it down because it
was suggested some time ago.
--
#!/usr/bin/perl http://domm.zsi.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}
Re: post-YAPC::Europe CPANTS news
On 9/7/06, Thomas Klausner [EMAIL PROTECTED] wrote: Examples go in the installed docs or they are lost. The docs could state that there are some ready-to-run examples in that dir in the distribution. In one module where we had planty of examples I addeded a script that would creata a Module::Name::Examples.pm that is a collection of the example files in pod format. This modules gets installed so the examples are right at hand. I am not sure if this was a good idea but I can think of exapanding it into by adding some code that will split up the pm file to create the example files. Gabor
Re: post-YAPC::Europe CPANTS news
Jonathan Rockway wrote: I could be wrong here, but I think the check is to make sure that tar doesn't set +x on Makefile.PL or Build.PL, thus forcing the user to run the proper version of perl instead of automagically running the perl that shebang points to. (Example: Makefile.PL says #!/usr/bin/perl, but you really want to run /home/jon/blead/bin/perl. Forcing you to type this out is a good thing.) I personally don't see the value of this, I always run perl Makefile.PL anyway. I have two perls on this 'ere machine, one of which I use for general scripting and hacking on the machine itself. That's the first one in the path, so is what gets invoked when I say perl Makefile.PL. The other is in /opt/ and is built the same as the perl on our QA and production machines (where it also lives in /opt). It's what I use for writing code that will eventually go live to the public. Obviously, I need different modules for those two cases, and just as obviously, when installing a module I need to tell Makefile.PL which perl to install it for. (actually I have three perls, but one is Debian's own version which I never touch) -- David Cantrell
Re: post-YAPC::Europe CPANTS news
On Thursday 07 September 2006 00:51, Gabor Szabo wrote: In one module where we had planty of examples I addeded a script that would creata a Module::Name::Examples.pm that is a collection of the example files in pod format. This modules gets installed so the examples are right at hand. I am not sure if this was a good idea but I can think of exapanding it into by adding some code that will split up the pm file to create the example files. Pod::ToDemo, for example? -- c
Re: post-YAPC::Europe CPANTS news
On 9/7/06, chromatic [EMAIL PROTECTED] wrote: On Thursday 07 September 2006 00:51, Gabor Szabo wrote: In one module where we had planty of examples I addeded a script that would creata a Module::Name::Examples.pm that is a collection of the example files in pod format. This modules gets installed so the examples are right at hand. I am not sure if this was a good idea but I can think of exapanding it into by adding some code that will split up the pm file to create the example files. Pod::ToDemo, for example? Sort of but not exactly. As I can see in the SDL::Tutorial where Pod::ToDemo is in use, there can be only one example perl .pm file and the real example is not in the POD which means I have to duplicate the example in the text of the pod. Gabor -- c -- Gabor Szabo http://www.szabgab.com/ Perl Training in Israel http://www.pti.co.il 08-975-2897 054-4624648
Re: post-YAPC::Europe CPANTS news
On Thursday 07 September 2006 03:28, Gabor Szabo wrote: On 9/7/06, chromatic [EMAIL PROTECTED] wrote: Pod::ToDemo, for example? Sort of but not exactly. As I can see in the SDL::Tutorial where Pod::ToDemo is in use, there can be only one example perl .pm file and the real example is not in the POD which means I have to duplicate the example in the text of the pod. The first is a genuine limitation, but the second is fixable with a little seek on *DATA and flipflop action. -- c
Re: post-YAPC::Europe CPANTS news
On 9/7/06, Gabor Szabo [EMAIL PROTECTED] wrote: On 9/7/06, chromatic [EMAIL PROTECTED] wrote: On Thursday 07 September 2006 00:51, Gabor Szabo wrote: In one module where we had planty of examples I addeded a script that would creata a Module::Name::Examples.pm that is a collection of the example files in pod format. This modules gets installed so the examples are right at hand. I am not sure if this was a good idea but I can think of exapanding it into by adding some code that will split up the pm file to create the example files. Pod::ToDemo, for example? Sort of but not exactly. As I can see in the SDL::Tutorial where Pod::ToDemo is in use, there can be only one example perl .pm file and the real example is not in the POD which means I have to duplicate the example in the text of the pod. Here is the module where we generate a .pm file from the files in the eg/ directory: http://search.cpan.org/~ctrondlp/Win32-GuiTest-1_50.5/Examples.pm Gabor
Re: post-YAPC::Europe CPANTS news
Thomas Klausner wrote: Oh, and if you want to join the fun and help a bit, here's a (probably incomplete) list of tasks: - Metrics: [snip] Would the metrics for community support channels that were suggested a while ago be welcome? (The discussion about them sort of died out :-\) - Salve
Re: post-YAPC::Europe CPANTS news
Adam Kennedy wrote: It might be an interesting idea to also add a dependencies_exist metric, that makes sure that all the dependencies that are declared actually exist in the CPAN. Dunno, could be of dubiously little value, but I just managed to somehow upload something with bad deps that I had installed on my local machine, but that weren't all on CPAN yet... I'm not sure this is such a good idea. There are several examples of modules that rely on other Perl modules that aren't on CPAN. For instance, svk relies on SVN::Mirror which is a part of svn. SWISH::HiLiter needs SWISH::API which is a part of swish-e. There are lots of perl modules tied to various projects that don't exist independently on CPAN. -- Michael Peters Developer Plus Three, LP
Re: post-YAPC::Europe CPANTS news
* Salve J Nilsen [EMAIL PROTECTED] [2006-09-07 14:15]: Would the metrics for community support channels that were suggested a while ago be welcome? (The discussion about them sort of died out :-\) Only if it’s possible for an author to state that this metric is meaningless. F.ex., I can’t imagine what benefit my Bencode module would derive from anything beyond the RT queue it already has. Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/
Re: post-YAPC::Europe CPANTS news
On Wednesday 06 September 2006 02:53, Thomas Klausner wrote: - buildtool_not_executable Check if the buildtool (Makefile.PL, Build.PL) are not executable (and thus need to be called with 'perl Build.PL' thereby specifying which exact version of Perl you want) I'm not sure of the value of this one; how does an author make the buildtool executable on Windows, for example? I have the impression (not using Windows) that users must always call the tool with 'perl Build.PL' on such platforms. Am I mistaken? -- c
Re: post-YAPC::Europe CPANTS news
On 9/6/06, chromatic [EMAIL PROTECTED] wrote: On Wednesday 06 September 2006 02:53, Thomas Klausner wrote: - buildtool_not_executable Check if the buildtool (Makefile.PL, Build.PL) are not executable (and thus need to be called with 'perl Build.PL' thereby specifying which exact version of Perl you want) I'm not sure of the value of this one; how does an author make the buildtool executable on Windows, for example? I have the impression (not using Windows) that users must always call the tool with 'perl Build.PL' on such platforms. Am I mistaken? on windows the command interpreter uses file associations instead of C#!. files with extension C.pl are usually associated with Cperl.exe, and invoked without specifying Cperl on the command line. this is user-modifiable, so it's possible to configure files with the C.pl extension to be associated with another executable, or none at all. ~jerry
Re: post-YAPC::Europe CPANTS news
I could be wrong here, but I think the check is to make sure that tar doesn't set +x on Makefile.PL or Build.PL, thus forcing the user to run the proper version of perl instead of automagically running the perl that shebang points to. (Example: Makefile.PL says #!/usr/bin/perl, but you really want to run /home/jon/blead/bin/perl. Forcing you to type this out is a good thing.) I personally don't see the value of this, I always run perl Makefile.PL anyway. Do +x *.PL files cause problems with CPAN installs? (Or am I completely wrong in my interpretation of this test? :) Regards, Jonathan Rockway chromatic wrote: On Wednesday 06 September 2006 02:53, Thomas Klausner wrote: - buildtool_not_executable Check if the buildtool (Makefile.PL, Build.PL) are not executable (and thus need to be called with 'perl Build.PL' thereby specifying which exact version of Perl you want) I'm not sure of the value of this one; how does an author make the buildtool executable on Windows, for example? I have the impression (not using Windows) that users must always call the tool with 'perl Build.PL' on such platforms. Am I mistaken? -- c
Re: post-YAPC::Europe CPANTS news
On Wednesday 06 September 2006 10:27, Jonathan Rockway wrote: I could be wrong here, but I think the check is to make sure that tar doesn't set +x on Makefile.PL or Build.PL, thus forcing the user to run the proper version of perl instead of automagically running the perl that shebang points to. (Example: Makefile.PL says #!/usr/bin/perl, but you really want to run /home/jon/blead/bin/perl. Forcing you to type this out is a good thing.) Oh, right. Yes, that's obvious now. I misinterpreted it. -- c
Re: post-YAPC::Europe CPANTS news
Thanks for the updates, Thomas. And now on with the complaining! Thomas Klausner wrote: - has_example An optional metric that checks if the author included a dir called 'eg|ex|example(s?)' which in turn includes at least on *.pl IMO examples in an example directory are a detriment, not a bonus. Any uninstalled documentation in the source directory will likely never been seen by the user (think about a CPAN shell installation) or even if they do see it it will be thrown out with the source tree once the module is installed and thus lost. Examples go in the installed docs or they are lost. Additionally, if you're going to keep this thing around, restricting the examples to .pl files penalizes HTML (Mason or other templating system), POD and module examples. [1] It would be best to just say that the directory contains something. [1] Also people like me who don't like to put a .pl extension onto their Perl executables. You don't run ls.c or cpp.sh. The user does not care and should not know what language their executable is in.
Re: post-YAPC::Europe CPANTS news
Jonathan Rockway wrote: I could be wrong here, but I think the check is to make sure that tar doesn't set +x on Makefile.PL or Build.PL, thus forcing the user to run the proper version of perl instead of automagically running the perl that shebang points to. (Example: Makefile.PL says #!/usr/bin/perl, but you really want to run /home/jon/blead/bin/perl. Forcing you to type this out is a good thing.) I personally don't see the value of this, I always run perl Makefile.PL anyway. Do +x *.PL files cause problems with CPAN installs? (Or am I completely wrong in my interpretation of this test? :) There's a previous thread on this that explains the details. In short, the perl you call Makefile.PL with, is the one that automagically gets used later when calling make test, make install etc. So for cases where there is more than one Perl installed (and there's more than you'd think) it ensures that people are installing modules to the correct Perl, and prevents accidents. Adam K
Re: post-YAPC::Europe CPANTS news
Thomas Klausner wrote: Hi! During YAPC::Europe in Birmingham I did a (rather short and un-hackish) hackathon. Slides are available here: http://domm.plix.at/talks/2006_birmingham_cpants/ During YAPC, I added a few new metrics: - manifest_matches_dist Check if the stuff listed in MANIFEST matches what's in the dist - buildtool_not_executable Check if the buildtool (Makefile.PL, Build.PL) are not executable (and thus need to be called with 'perl Build.PL' thereby specifying which exact version of Perl you want) Glad to see this included, it caught about 20 modules of mine that had executable Makefile.PLs... although working out how the hell I managed to do that might take a little while, I've added another step to my release automation to stop it. - has_example An optional metric that checks if the author included a dir called 'eg|ex|example(s?)' which in turn includes at least on *.pl I thought we were generally negative on this one, because it would encourage people to spuriously add trivial example directories to their distributions... As a secondary point, if we keep it you might also want to include /sample(s)/ and /demo(s)/. This resulted in quite some movement in the CPANTS game... I've also changed the display of the authors page http://cpants.perl.org/author/DOMM optional metrics (is_prereq and has_example) are now visually seperated from the core metrics. I like the new style, not being red is a good thing. The site cpants.perl.org is still not very stable due to a Catalyst/DBIx::Class-Bug (or more likely a bug in my usage of them) Oh, and if you want to join the fun and help a bit, here's a (probably incomplete) list of tasks: - Metrics: - has_license - no_boilerplate - has_valid_filenames - declares_dependencies It might be an interesting idea to also add a dependencies_exist metric, that makes sure that all the dependencies that are declared actually exist in the CPAN. Dunno, could be of dubiously little value, but I just managed to somehow upload something with bad deps that I had installed on my local machine, but that weren't all on CPAN yet... - no_open_bugs If we are going to do this, we have to make sure that we DON'T include wishlist bugs in this metric, or it blows the validity all to hell. - Signature checking ? I looked into this a bit more recently, with all the Module::Signature changes. I think this is a bad idea. Signature checking HAS to be done by the CPAN client, and NOT ever by the module itself, because it limits the ability of the client to modify or tweak files (for example, we might potentially want to have Makefile.PL alter the META.yml to localise the deps for the current platform). By the time make test runs, it's WAY too late to be checking signatures. has_signature is fine, but another test along the lines of the Test::Pod and Test::Pod::Coverage ones would be a bad idea. - has_rating ? Bad idea I think as well, because it will encourage spurious ratings. This is the same reason that ratings aren't factored into the search.cpan result order. CPAN Ratings currently has most of it's value because it doesn't matter much, and so it encourages high quality reviews. Anything that would make the quality of the reviews worse (or malicious) isn't a negative thing. - relation of subs to lines of code ? Dubious. Makes a naive and blanket judgement call. Adam K
Re: post-YAPC::Europe CPANTS news
Anything that would make the quality of the reviews worse (or malicious) isn't a negative thing. Or rather, it IS a negative thing. :) Adam K
