Script to find Module Dependency Test Results...
I have a prototype Perl script that will determine the dependencies of a given
CPAN distribution, and then check CPAN Testers for any failure reports of that
distro or dependent distros for a given platform.
I would like to work with other people to turn this into something of use to
the community, at the very least a module but ideally something that could be
integrated with one of the CPAN-related web sites. (I'm also starting
graduate school in the fall and looking for people to take this idea and run
with it, as I won't have as much time.)
For an example of what this does, if I ask it to search for dependencies for
the disto 'Pixie', it tells me the following:
Pixie-2.06:
DBIx-AnyDBD-2.01: {}
Data-UUID-0.11: {}
Test-Class-0.03:
Test-Differences-0.47:
Text-Diff-0.35:
Algorithm-Diff-1.15: {}
Test-Exception-0.15:
Sub-Uplevel-0.09: {}
Test-Tester-0.09: {}
Test-Tester-0.09: {}
for Windows machines, it also tells me this:
Algorithm-Diff-1.15: []
DBIx-AnyDBD-2.01:
- action: FAIL
distversion: DBIx-AnyDBD-2.01
id: 79987
platform: MSWin32-x86-multi-thread
report_url: |-
http://nntp.x.perl.org/group/perl.cpan.testers/79987
version: 2.01
Data-UUID-0.11:
- action: FAIL
distversion: Data-UUID-0.11
id: 145033
platform: MSWin32-x86-multi-thread
report_url: |-
http://nntp.x.perl.org/group/perl.cpan.testers/145033
version: 0.11
- action: FAIL
distversion: Data-UUID-0.11
id: 146960
platform: MSWin32-x86-multi-thread
report_url: |-
http://nntp.x.perl.org/group/perl.cpan.testers/146960
version: 0.11
Pixie-2.06: ~
Sub-Uplevel-0.09: []
Test-Class-0.03:
- action: FAIL
distversion: Test-Class-0.03
id: 144608
platform: MSWin32-x86-multi-thread
report_url: |-
http://nntp.x.perl.org/group/perl.cpan.testers/144608
version: 0.03
Test-Differences-0.47: []
Test-Exception-0.15: []
Test-Tester-0.09: []
Text-Diff-0.35: []
In other words, there are no test reports for Windows users of Pixie, but
that's likely because they can't get several distros that Pixie requires to
work on Windows (Test-Class, DBIx-AnyDBD-2.01, and Data-UUID-0.11).
This information would be of use to various quality-assurance types, as well
as the module author. It's probably of use to module users too.
Re: [Module::Build] requires_one_of
On 7/18/2004 5:14 AM Smylers wrote: [...] Rather than the dependent app (or module) having a list alternatives that are known to work, it could instead depend on some 'abstract' package. Other distros are then able to say that they 'provide' that abstract package. So if another module is writtent that has equivalent functionality and the same interface then it just needs to label itself as 'provide'-ing that 'abstract' package, and the dependent app will just work with it. That requires the other distro authors to modify their packages. If they don't, then the feature can't be used. Take modules which interface with the serial port, for example. There's Win32::SerialPort (which hasn't been updated in years), and there's Device::SerialPort. One is for Windows, the other for Unix and imitates the Windows interface. So a module which needs the serial port requires either Win32::SerialPort or Device::SerialPort. My only way around it for some of the GPS::* modules I wrote was to say it 'recommends' both modules, which it really doesn't. I could put a dynamic config in the Build.PL but that causes the Makefile.PL and META.yml to reflect the platform that the dist was built on. Not good. It makes more sense to say 'requires_one_of'. Changes would only have to be made to the modules that handle builds and installation. Nobody else needs change their distros unless they want to use that feature. That nicely puts control of whether a distro provides certain functionality in control of each distro's author; the knowledge is in the system as a whole, and no one person has to keep an exhaustive list up to date. What one person needs to keep an exhaustive list? An exhaustive list of what? Expecting hundreds of CPAN authors to change their distros, and to have groups of them agree on abstract interfaces to provide, is not practical. Also, David has an app that depends on something Pg-or-mysql-like; suppose I do too, as do several other people. When another Pg-or-mysql-providing module appears it doesn't make sense for every single one of us app authors to have to note this, tweak our install settings, and upload a new version to Cpan; that's lots of duplicated and redundant effort. The choice as to whether to support another feature is up to a module author. If you don't want to put the extra work in, don't. Other authors would. The obvious flaw in my proposal for this particular instance is that Pg and mysql don't provide identical interfaces, so I'm guessing that David hasn't written code that just happens to work with either of them but has conditions in it specifically to deal with their differences. To make a 'provides' feature work, those differences would have to be abstracted out elsewhere. ...
Re: Future of the Module List
So, if I want to write a review of Net::SMTP, I'd do the following. 1. Use Module::Build, or ExtUtils::MakeMaker to create Review::Net::SMTP::CHRISJ, or whatever. 2. Make sure I have my README.txt, CHANGES, and MANIFEST file. 3. Write my review in POD format, and throw in some META.yml indexing code as well. 4. Make sure the module/review is executable so it passes CPAN testing. 5. Upload the review to CPAN. On Tue, 20 Jul 2004, Sam Vilain wrote: I nominate the Review::* Namespace for author-submitted module indexes and in-depth reviews, in POD format. I think this has a number of advantages. Let's use the infrastructure we already have, no? The .pm versions of these modules could potentially contain lists (in YAML format of course) for use by the indexing systems, which would, after double checking, be used as a master data source for the auto-generated indexes and modules lists. This probably needs to be prototyped a little, and a model review module (say that 5 times quickly) posted before reviews start coming in in earnest. Sam. Christopher Josephes [EMAIL PROTECTED]
Re: CPAN Rating
[EMAIL PROTECTED] (A. Pagaltzis) writes: * Nicholas Clark [EMAIL PROTECTED] [2004-06-16 21:24]: Well, I guess to run a patched version of search.cpan.org on your local system you need to start by running an unpatched version of search.cpan.org. I'm not sure whether the source to it is available, and if so, where to get it. But resolving that seems to be the first step. That means asking Ask, if I am correctly informed, right? No, I didn't make search.cpan.org. - ask -- ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
Re: CPAN Rating Wrap up
[EMAIL PROTECTED] (Khemir Nadim) writes: - We have troubles finding out who is holding onto the code, were it is and how to change it. Which site are you talking about then? Surely it can't be http://cpanratings.perl.org. If you click on a random link in the menu bar (there are 3) you have a 33% chance of getting to a page with a link to installation instructions for installing a local copy of the site *and* a TODO list. I'll check in on this list again on this topic when the TODO list is a bit shorter. (*cough*patches*cough*) - ask -- ask bjoern hansen, http://www.askbjoernhansen.com/ !try; do();
Re: Future of the Module List
On Jul 20, 2004, at 11:57 AM, Chris Josephes wrote: So, if I want to write a review of Net::SMTP, I'd do the following. 1. Use Module::Build, or ExtUtils::MakeMaker to create Review::Net::SMTP::CHRISJ, or whatever. 2. Make sure I have my README.txt, CHANGES, and MANIFEST file. 3. Write my review in POD format, and throw in some META.yml indexing code as well. 4. Make sure the module/review is executable so it passes CPAN testing. 5. Upload the review to CPAN. I was sort of hoping this idea would just die on its own, but now it looks like people are actually getting ready to do it. In my opinion this is a bad idea. I don't want a bunch of reviews all over CPAN disguising themselves as modules. I also don't want CPAN sites to have to figure out what's a review and what's not, so they can filter out the reviews. What's wrong with making one of these newfangled things called web sites to host reviews? Oh, I know - that already exists. So how about working with those people to fix whatever you think is broken about them before polluting CPAN with all this non-code? -Ken
Re: source for search.cpan.org
# The following was supposedly scribed by # Randy W. Sims # on Thursday 22 July 2004 04:50 pm: Ask posted with information about where to find the source for CPAN Ratings, But, most of the interface improvements (IMO) need to be done to search.cpan.org. I found http://combust.develooper.com/install.html, but the installation instructions lead me to a password-protected subversion repository at http://svn.develooper.com/ Yes, cpanratings.perl.org could use some bugfixes (and maybe rating-ratings (e.g. your review was stupid, please justify it or remove it.)) But, first I think we need to get the ratings to be more visible on search.cpan.org. --Eric -- There are three terrible ages of childhood -- 1 to 10, 10 to 20, and 20 to 30. --Cleveland Amory
Re: Future of the Module List
On Thu, 22 Jul 2004 13:50:37 -0500, Ken Williams [EMAIL PROTECTED] wrote: I was sort of hoping this idea would just die on its own, but now it looks like people are actually getting ready to do it. In my opinion this is a bad idea. I don't want a bunch of reviews all over CPAN disguising themselves as modules. I also don't want CPAN sites to have to figure out what's a review and what's not, so they can filter out the reviews. I too was hoping this would die on its own as an Obviously Bad Idea. Having a bunch of POD-only modules is about as appropriate as a review medium as using email signatures. Please, don't do this. Make the CPAN ratings web site better, or start your own web site, and use a storage medium other than the CPAN module repository. -John
Re: Future of the Module List
On Thu, 22 Jul 2004, Ken Williams wrote: I was sort of hoping this idea would just die on its own, but now it looks like people are actually getting ready to do it. In my opinion this is a bad idea. I don't want a bunch of reviews all over CPAN disguising themselves as modules. I also don't want CPAN sites to have to figure out what's a review and what's not, so they can filter out the reviews. Agreed. I kind of hoped that by pointing out the steps that would be involved in posting a review, people would kind of get the clue that the proposal needs a little work. What's wrong with making one of these newfangled things called web sites to host reviews? Oh, I know - that already exists. So how about working with those people to fix whatever you think is broken about them before polluting CPAN with all this non-code? It would be really easy to set up a blog or whatever to handle this, but inevitably someone always asks the question What About CPAN? I'm probably not on enough Perl mailing lists to make an expert opinion, but the impression I get is that CPAN is a system without a clearly defined future. Questions always come up on who maintains it, where is the code, how do we add this feature, why is the module list out of date, etc, etc. CPAN works great for distributing perl code and modules, but as a software package, CPAN is a mystery to a lot of us. -Ken Christopher Josephes [EMAIL PROTECTED]
Re: Ratings and Reviews ne Modules
Eric Wilhelm [EMAIL PROTECTED] quoth: * *ok, now we see the ratings, click-through to get reviews... * http://cpanratings.perl.org/d/CGI.pm * *Or do we? I get a 404. Not much use. Well, that's an error and one you should take up with the people who run the rantings pages at [EMAIL PROTECTED] Graham only pulls the rantings from their site and it's not local. *Finally, here we are at a page with reviews. Frankly, I've never seen one of *these before, and I can't say that this one really does me a huge amount of *good. I'm much better served by reading the documentation (which should *speak for the quality of the module on it's own.) However, the 1-5 star *rating DOES provide a nice at-a-glance view of others' opinions. * *So, we have ratings, and we have reviews. What's wrong? That's a big question and I have a long answer for that but mostly I've only seen stuff like Randal extorting authors with 1-star ratings, authors sniping at each other via rantings and in module docs and, when they aren't petty and vile they're 5-stars with 'great module'. There's a smattering of useful reviews but, as on Amazon, YMMV. What's wrong isn't a software issue. e.
Re: Ratings and Reviews ne Modules
On Thu, 22 Jul 2004, Eric Wilhelm wrote: Okay, but we have requirements for both search.cpan.org and cpanratings.perl.org, right? Yes. cpanratings could display more in depth statistics of the various modules and also allow for being to view a module as a whole and not just one particular verson of a module. search.cpan.org is the front-end to most module-searching, and I think it should stay that way, but should have more visible (read: not so deep) links to ratings/reviews and show the star-bar in more places. For instance it would be nice to be able to sort search results by ratings. Or it'd be nice if that were factored in somehow when doing search. Maybe this would iron out the greatest module since sliced bread that people who don't know the secret handshake have never heard about problem. If these sites have separate maintainers, we should be working on two requirements lists. Some requirements would seem to be related. For instance, cpanrating may need to provide a convenient way to get data that would the be utilized on search.cpan.org to make the impact less painful. Maybe some sort of local ratings cache would need to be maintained? -- /chris
Re: Ratings and Reviews ne Modules
On Thu, Jul 22, 2004 at 03:26:56PM -0500, Elaine -HFB- Ashton wrote: *Finally, here we are at a page with reviews. Frankly, I've never seen one of *these before, and I can't say that this one really does me a huge amount of *good. I'm much better served by reading the documentation (which should *speak for the quality of the module on it's own.) However, the 1-5 star *rating DOES provide a nice at-a-glance view of others' opinions. * *So, we have ratings, and we have reviews. What's wrong? That's a big question and I have a long answer for that but mostly I've only seen stuff like Randal extorting authors with 1-star ratings, authors sniping at each other via rantings and in module docs and, when they aren't petty and vile they're 5-stars with 'great module'. There's a smattering of useful reviews but, as on Amazon, YMMV. What's wrong isn't a software issue. How would you suggest we deal with this? Maybe a bit of moderator intervention? I concur with this, btw. The earlier sacrifice stone thread had me picturing my modules (ok, me really) strapped to the stone and a venomous critic with a very sharp knife getting ready to perform amateur surgery. One of the big reasons I haven't published much lately is it isn't worth it for me to put on an asbestos suit to try to contribute. That said, perhaps it would be useful to have a set of standard sort of comments that could be applied to the module, such as some tests don't seem to pass or requires compilation on the target platform, module shares similar functionality as XX, etc. along with other ratings. This would hopefully be useful but not condescending information. Austin
Re: Ratings and Reviews ne Modules
On Thu, 22 Jul 2004, Randy W. Sims wrote: Agreed. Reviews as modules is not the best solution. CPAN does one thing and does it well. Adding reviews as modules is likely to cause more problems that it solves. Do we really need reviews ? I am afraid not many people will take the time to do a deep analyzis of a module. On the other hand, I guess, they would be ready to give their short opinion or ideas on how to use the module, or which other module to use instead. For part of this cpanratings fits well, maybe it needs improvements but that's one direction. There are two other options for some improvement: What about creating an annotation system similar to what PHP has on its documentation (see www.php.net ) ? It might be part of the rating system or might be separate. What about a web based discussion board that is module specific ? Easy to search, categorized by modules, easy to post - no need to subscribe to a zillion mailing lists. Gabor
Re: Ratings and Reviews ne Modules
On Fri, 23 Jul 2004, Gabor Szabo wrote: Do we really need reviews ? Short of some better sort of solution for helping guide people to the better choices of modules. I am afraid not many people will take the time to do a deep analyzis of a module. It doesn't take many people to provide a statistically large enough sample to be useful in helping perl newbies avoid the modules that somebody cooked up five years ago and hasn't updated since. What about a web based discussion board that is module specific ? Easy to search, categorized by modules, easy to post - no need to subscribe to a zillion mailing lists. That'd be useful. A lot of modules might form little communities to save them from author neglect if they could find each other. So many modules have no mailing list and the author never responds. Now I don't blame the author for not responding, but without some central place for people to congregate and trade patches thing die on the vine that wouldn't have to. -- /chris
Re: Ratings and Reviews ne Modules
* Austin Schutz [EMAIL PROTECTED] [2004-07-23 01:03]: That said, perhaps it would be useful to have a set of standard sort of comments that could be applied to the module, such as some tests don't seem to pass or requires compilation on the target platform, module shares similar functionality as XX, etc. along with other ratings. This would hopefully be useful but not condescending information. These facts are metadata and shouldn't be mixed with the reviews at all. Some of the ones you propose are in fact already available. Regards, -- Aristotle If you can't laugh at yourself, you don't take life seriously enough.
Re: Ratings and Reviews ne Modules
On 7/22/2004 5:50 PM, Randy W. Sims wrote: We just need to organize and do it. 1st crack at organizing ideas/suggestions made in this thread and in Ask's TODO list. Comments/Omissions? Also available at http://www.thepierianspring.org/perl/cpan-ratings.notes -- I) CPAN Ratings A) Abuse Authors abusing the system for political statements, to sabatoge authors of similars modules, etc. 1) The usuall solution is a Karma type system. Number of reviews contributed by a reviewer. Thumbs up/down for individual reviews by a reviewer (Helpfulness ratings). Thresholds on Karma that automatically invoke a moderator. 2) See also item I.C.1. B) All Reviews Pages (per module): 1) Header with average rating and other summary information. C) All Reviews Pages (per reviewer): 1) Header with reviewer information. (Obfuscated email address for one thing; try to keep people accountable). D) Searching 1) If the number of reviews per module gets large, sorting on rating/date/version may be useful. 2) Search results should include direct link to all reviews. 3) Search results should include average rating. (average per version? overall average?) 4) Allow direct search for reviews. I.E. don't send user off to CPAN Search, but do try to use it behind the scenes. E) Browsing for modules (?) 1) By module/author. F) Author's Administrative interface 1) Edit review? - Original author only. 2) Delete this review - Original author only. G) Top rated modules list - Would this encourage abuse? H) RSS Feeds 1) RSS feed of sitewide recent reviews 2) Include rating numbers in the RSS feeds 3) Subscribe to reviews of certain distributions (preferably by author) 4) Reviews of modules by a certain author (for CPANID.rss feeds) [RWS: Same as above?] I) Ratings 1) If a reviewer reviews two or more versions of a module, how are the averages calculated? 2) Expand set of rated attributes? 3) Let 'Overall' rating be calculated based on other specific attributes or let there be some additional type of 'overall' that is calulated, and let it be used in summaries--to _encourage_ a more balanced review (and discourage abuses). J) Other 1) Reviews in other languages (with filters etc). 2) Parse Embperl-2.0b9 correctly. 3) Include the other rating numbers on [some page]. II) Misc: A) Module Pages Usage tips experiences not directly related to reviews. Doesn't have to be organized around modules; it can be organized around topics (ala emacswiki). Linked to from CPAN Ratings? CPAN Search? B) Best of breed reviews. A single comparative review written on several similar modules. How would this show up in a search? Does this belong with CPAN Ratings? III) Search CPAN: A) make /d/CGI.pm work Bug seemingly in Search CPAN for CPAN.pm (others?) possibly due to '.pm' being part of name. This appears on the module dist page for links to CPAN Testers CPAN Ratings as 404 errors. B) Sorting Search results by relevance/rating. C) Searching 1) Improve searching with Keywords (META.yml)
Re: Ratings and Reviews ne Modules
# The following was supposedly scribed by # Randy W. Sims # on Thursday 22 July 2004 07:56 pm: Comments/Omissions I) CPAN Ratings B) All Reviews Pages (per module): 1) Header with average rating and other summary information. duplicate this in III.B.2 III) Search CPAN: A) make /d/CGI.pm work move ^-- this to I.J.4, since it is a cpanratings.perl.org problem B) Sorting 1) Search results by relevance/rating. 3) include 'star-bar' on manpage (as link to reviews) e.g. http://search.cpan.org/~lds/CGI.pm-3.05/CGI.pm - Module Version: 3.05 + Module Version: 3.05 Source --Eric -- When the going gets tough, most people quit. --Unknown
Re: Reviewing reviewers
# The following was supposedly scribed by # Randy W. Sims # on Thursday 22 July 2004 07:56 pm: A) Abuse Authors abusing the system for political statements, to sabatoge authors of similars modules, etc. 1) The usuall solution is a Karma type system. Number of reviews contributed by a reviewer. Thumbs up/down for individual reviews by a reviewer (Helpfulness ratings). Thresholds on Karma that automatically invoke a moderator. Okay, so if I go on a bashing-fest and then you come through and thumbs-down all of my reviews, I'll go through and thumbs-down your reviews too and then bash on your modules if I haven't made it there already. Does that trigger a karma threshold of some sort? Seems that it would be hard to detect. How about peer-review of peer-review: If I say that your review was bad, I think the next step is for you to defend your review (unless it has previously gotten a thumbs-up, in which case I must support my thumbs-down with a critique of your review.) There may be a somewhat recursive process of attack and rebuttal here, but the point is that a mean review is likely not going to be defended, and even if it had received a spurious thumbs-up, a critical dismissal of said mean review is likely to be supported rather than dismissed (thus giving weight to the dismissal and counting further towards the thumbs-down.) Recursion to level 3-or-so (pi) of the attack-rebuttal tree may invoke a moderator (or just a chanting, blood-lusting crowd/mob.) Additional weight can be given to reviewers who have posted many reviews and received many thumbs-up, etc. But, the idea behind the tree is that it localizes the debate to the review in question (rather than risk weighting solely on what may have been karma generated by a flaming disagreement about a completely different module's merits.) Absolute dead-beats can still be identified by their failure to provide a rebuttal or continually reaching level pi() with nonsensical or null arguments. --Eric -- You can't win. You can't break even. You can't quit. --Ginsberg's Restatement of the Three Laws of Thermodynamics
Re: Reviewing reviewers
On 7/22/2004 11:41 PM, Eric Wilhelm wrote: # The following was supposedly scribed by # Randy W. Sims # on Thursday 22 July 2004 07:56 pm: A) Abuse Authors abusing the system for political statements, to sabatoge authors of similars modules, etc. 1) The usuall solution is a Karma type system. Number of reviews contributed by a reviewer. Thumbs up/down for individual reviews by a reviewer (Helpfulness ratings). Thresholds on Karma that automatically invoke a moderator. Okay, so if I go on a bashing-fest and then you come through and thumbs-down all of my reviews, I'll go through and thumbs-down your reviews too and then bash on your modules if I haven't made it there already. Does that trigger a karma threshold of some sort? Seems that it would be hard to detect. How about peer-review of peer-review: If I say that your review was bad, I think the next step is for you to defend your review (unless it has previously gotten a thumbs-up, in which case I must support my thumbs-down with a critique of your review.) There may be a somewhat recursive process of attack and rebuttal here, but the point is that a mean review is likely not going to be defended, and even if it had received a spurious thumbs-up, a critical dismissal of said mean review is likely to be supported rather than dismissed (thus giving weight to the dismissal and counting further towards the thumbs-down.) Recursion to level 3-or-so (pi) of the attack-rebuttal tree may invoke a moderator (or just a chanting, blood-lusting crowd/mob.) Additional weight can be given to reviewers who have posted many reviews and received many thumbs-up, etc. But, the idea behind the tree is that it localizes the debate to the review in question (rather than risk weighting solely on what may have been karma generated by a flaming disagreement about a completely different module's merits.) Absolute dead-beats can still be identified by their failure to provide a rebuttal or continually reaching level pi() with nonsensical or null arguments. I don't think anything this complicated or involved is needed. Neither am I convinced that the Karma solution is the best or a complete solution. However, This is pretty much the system that Amazon.com uses and it works pretty well IMO. Also, for this to get into back and forth arguments/bashing, the reviewers would have to attempt to track all of their reviews. The system would not provide an easy way to do that-there is no legitimate need for such a feature since the system is about reviews not reviewers. That's not to say it's not possible, but it's not easy either. I think that will discourage a large percentage of abuse, and high percentages is the best you can shoot for. Regards, Randy.
