Re: Devel::Cover better practice to identify uncovered files

2017-04-18 Thread James E Keenan
On 04/18/2017 07:43 PM, Jason Pyeron wrote: Currently we are using this script to find files (line 48) under the scripts directory and add them to the coverage report. After running the programs under test, the script is run: $ ./tests/script/cover-missing.pl load ingest 17 known covered

Re: Devel::Cover missing lines?

2017-04-18 Thread James E Keenan
On 04/18/2017 12:32 PM, Jason Pyeron wrote: I am "require"ing a file but Devel::Cover is not logging the statements, only the sub, use, and eval portions. Exactly *which* statements is it not logging which you believe should be logged? I've been using Devel::Cover for 13 years. I would

Re: Updated CPAN River available

2017-04-16 Thread James E Keenan
On 04/16/2017 09:46 PM, Ricardo Signes wrote: * James E Keenan <jkee...@pobox.com> [2017-04-12T22:15:22] For me the most interesting aspect of this fifth round is that 29 distributions which appeared in order-of-battle-20170409.txt (the previous round focusing on no-dot) no longer

Re: Updated CPAN River available

2017-04-12 Thread James E Keenan
On 04/09/2017 08:54 PM, James E Keenan wrote: On 04/09/2017 11:33 AM, James E Keenan wrote: An updated printout of the CPAN River is available here: https://gist.github.com/jkeenan/a7670b2ffe1ffe3e8d8cbdbc2b3dd721 Today I began a fifth round of testing. As in the previous round

Re: Updated CPAN River available

2017-04-09 Thread James E Keenan
On 04/09/2017 09:03 PM, Kent Fredric wrote: On 10 April 2017 at 12:54, James E Keenan <jkee...@pobox.com> wrote: 7. Used a revised version of 'order-battle.pl' to record the order in which various modules *first* appeared in the 'fails' file and the total number of times each module was

Re: Updated CPAN River available

2017-04-09 Thread James E Keenan
On 04/09/2017 11:33 AM, James E Keenan wrote: An updated printout of the CPAN River is available here: https://gist.github.com/jkeenan/a7670b2ffe1ffe3e8d8cbdbc2b3dd721 Using this data we can conduct a fourth round of testing. Today I began a fourth round of testing. I am doing so

Updated CPAN River available

2017-04-09 Thread James E Keenan
An updated printout of the CPAN River is available here: https://gist.github.com/jkeenan/a7670b2ffe1ffe3e8d8cbdbc2b3dd721 This is an update of the data presented by David Golden in 2016 here: https://gist.github.com/xdg/af7a32c5b21d45a6b255 I have chosen to present this in .csv format to make

Re: BBC testing from upriver downwards

2017-04-04 Thread James E Keenan
On 04/04/2017 05:37 PM, Kent Fredric wrote: On 5 April 2017 at 08:28, James E Keenan <jkee...@pobox.com> wrote: the tip of the order of battle looks substantially different I must say though, your results confuse me. In my testing, the things that are holding back the most are ...

Re: BBC testing from upriver downwards

2017-04-04 Thread James E Keenan
On 03/30/2017 04:36 PM, James E Keenan wrote: [snip] 7. Wrote second perl program (attached) called 'order-battle.pl' to record the order in which various modules *first* appeared in the 'fails' file and the total number of times each module was cited. Results are attached as 'order-of-battle

Re: BBC testing from upriver downwards

2017-03-30 Thread James E Keenan
On 03/30/2017 05:16 PM, Kent Fredric wrote: On 31 March 2017 at 09:36, James E Keenan <jkee...@pobox.com> wrote: For example, today David Golden prepared a new version of Sub::Uplevel -- the #1 distro in the order of battle. Once he releases that to CPAN, a tremendous number of down

Re: BBC testing from upriver downwards

2017-03-30 Thread James E Keenan
On 03/20/2017 09:32 PM, James E Keenan wrote: On 03/17/2017 09:42 PM, Karen Etheridge wrote: ​> ​ 1. Compose a list of CPAN distros starting with those farthest up river, i.e., distros that only depend on the perl 5 core. Within that set of distros I'd like to order them from most reve

Re: BBC testing from upriver downwards

2017-03-20 Thread James E Keenan
On 03/17/2017 09:42 PM, Karen Etheridge wrote: ​> ​ 1. Compose a list of CPAN distros starting with those farthest up river, i.e., distros that only depend on the perl 5 core. Within that set of distros I'd like to order them from most reverse dependencies to fewest. Then go down river from

BBC testing from upriver downwards

2017-03-17 Thread James E Keenan
perl-5.25.11 will be released on Monday March 20. Since this will be the first monthly dev release to reflect the banishment of '.' from the default @INC, it is the first monthly release in which we can assess the effect of that change on CPAN. Up until now we've mainly relied on Andreas and

Re: Test::Smoke failing to test most recent commit to Perl 5 blead

2017-01-30 Thread James E Keenan
On 01/29/2017 11:37 AM, demerphq wrote: I have some further comments in the other thread. Nothing that needs to stop your patch from being applied, but I noticed some stuff I found quite surprising, although I only did a quick scan so I very well may be talking out my ass in that thread. Yves

Re: Test::Smoke failing to test most recent commit to Perl 5 blead

2017-01-29 Thread James E Keenan
On 01/27/2017 10:58 PM, demerphq wrote: On 28 Jan 2017 6:32 a.m., "James E Keenan" <jk...@verizon.net <mailto:jk...@verizon.net>> wrote: On 01/14/2017 10:10 AM, James E Keenan wrote: Here is the first part of the smokecurrent.log for this run --

Re: Test::Smoke failing to test most recent commit to Perl 5 blead

2017-01-27 Thread James E Keenan
On 01/14/2017 10:10 AM, James E Keenan wrote: On 01/13/2017 11:25 PM, James E Keenan wrote: The most recent smoke testing of Perl 5 blead I have conducted in my FreeBSD-10.3 VM is that found at: Here is an additional instance of the problem, which I have reported as: https://rt.cpan.org

perl5.test-smoke.org feature request: explanation of git-id

2017-01-17 Thread James E Keenan
When you go to http://perl5.test-smoke.org/search, one of the columns presented is "Git-id". Its contents look like this: v5.25.8-269-g01f9673 v5.25.8-196-g3626fc2 v5.25.8-208-g05053e2 v5.25.8-260-g57b3a6b6ec What these strings mean is not entirely self-evident. I can infer that 'v5.25.8'

perl5.test-smoke.org feature request: order by column

2017-01-17 Thread James E Keenan
When you go to http://perl5.test-smoke.org/search, you see a table whose columns are these: ArchHostOS Git-id Smoke date Status AFAICT, you have no ability to select the way the rows in the table are ordered. It appears that the sorting is done on the columns in

Re: Test::Smoke failing to test most recent commit to Perl 5 blead

2017-01-14 Thread James E Keenan
On 01/13/2017 11:25 PM, James E Keenan wrote: The most recent smoke testing of Perl 5 blead I have conducted in my FreeBSD-10.3 VM is that found at: http://perl5.test-smoke.org/report/53193 The commit tested there was 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e: # commit

Test::Smoke failing to test most recent commit to Perl 5 blead

2017-01-13 Thread James E Keenan
The most recent smoke testing of Perl 5 blead I have conducted in my FreeBSD-10.3 VM is that found at: http://perl5.test-smoke.org/report/53193 The commit tested there was 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e: # commit 3626fc2020b81652c4b3c7dd5ef0822b194d2d5e Author: Karl

Re: CPANifying our test framework - or parts of it

2016-09-10 Thread James E Keenan
On 09/09/2016 09:34 PM, Sam Kington wrote: Hi, At $WORK we have an extensive test suite that we’ve built up over the years, with loads of convenience methods for calling e.g. Dancer endpoints and checking returned data structures against what we expect. One of our dev team recently left for

Re: Why does File::Path not respond to 'cover'?

2015-06-28 Thread James E Keenan
, James E Keenan jk...@verizon.net wrote: Richard Elberger and I recently received co-maintenance bits on the File-Path distribution. As is my wont, I ran it through Devel::Cover -- but was surprised to see that 'cover' reported zero coverage. Does anyone know why File-Path is unresponsive

Why does File::Path not respond to 'cover'?

2015-06-26 Thread James E Keenan
Richard Elberger and I recently received co-maintenance bits on the File-Path distribution. As is my wont, I ran it through Devel::Cover -- but was surprised to see that 'cover' reported zero coverage. # $ perl Makefile.PL make Generating a Unix-style Makefile Writing Makefile for

Re: new Stream Test::Simple question, XS requirement

2015-04-26 Thread James E Keenan
On 04/26/2015 12:15 AM, Chad Granum wrote: I am not sure it is entirely my decision. If it is entirely my decision than this is it: Must have a working scalar::util::weaken to use the new Test::Simple. The Test-Simple dist itself will not require a c compiler to install, as in no xs is included

New York City Perl Hackathon

2015-04-07 Thread James E Keenan
As some readers of this list already know, two weeks after the upcoming Perl QA Hackathon in Berlin we are having a more locally based hackathon in New York City: the Second New York City Perl Hackathon. Our wiki starts here: https://github.com/nyperlmongers/nyperlhackathon2015/wiki Having

hackathon web page displays strangely

2015-03-21 Thread James E Keenan
http://act.qa-hackathon.org/qa2015/main In Firefox, at least, all the pages on this site are displaying in a strange way. Each page has a left sidebar with internal links and a right sidebar with sponsors. But the main content for each page only starts to appear *below* the end of the

Re: hackathon web page displays strangely

2015-03-21 Thread James E Keenan
On 03/21/2015 09:45 AM, Shlomi Fish wrote: Hi James, On Sat, 21 Mar 2015 08:51:42 -0400 James E Keenan jk...@verizon.net wrote: http://act.qa-hackathon.org/qa2015/main In Firefox, at least, all the pages on this site are displaying in a strange way. Each page has a left sidebar

Re: Final TPF Devel::Cover grant report

2014-10-08 Thread James E Keenan
On 10/07/2014 05:42 PM, Paul Johnson wrote: In accordance with the terms of my grant from TPF this is the final report for my work on improving Devel::Cover. What a long, strange trip it's been! Since the last report I have released versions 1.16 and 1.17. The stable release of perl 5.20.1

Re: Discussion/request on Test::More , have it add t/lib to @INC if -d

2014-01-31 Thread James E Keenan
On 1/30/14 10:25 PM, Leon Timmermans wrote: On Fri, Jan 31, 2014 at 1:59 AM, Torbjørn Lindahl torbjorn.lind...@gmail.com wrote: It seems t/lib is a common place to put modules used to support testing, how about having Test::More push that path to @INC if -d 't/lib' ? It would save me one ,

Re: New Test::Builder version broke Test::ParallelSubtest

2014-01-13 Thread James E Keenan
On 1/13/14 6:55 PM, Jan Seidel wrote: I initially skipped this as I saw another open bug that hasn't been addressed since 2 years now. But I agree that for tracking purposes it is nonetheless a good idea to open a bug: https://rt.cpan.org/Public/Bug/Display.html?id=92119 Heh! Even much

Re: TPF Devel::Cover grant report July 2013

2013-09-03 Thread James E Keenan
On 9/1/13 10:53 PM, Ricardo Signes wrote: * Paul Johnsonp...@pjcj.net [2013-08-31T19:01:09] In accordance with the terms of my grant from TPF this is the monthly report for my work on improving Devel::Cover covering July 2013. +1, Thanks, Paul! The rest of the work this month was spent on

Testing coverage-related work at 2013 Perl QA Hackathon

2013-04-14 Thread James E Keenan
The following was posted at http://2013.qa-hackathon.org/qa2013/wiki?node=Achievements, but I figured that it should be posted on-list as well. ## At the 2013 Perl QA Hackathon in Lancaster, England, testing coverage-related issues were handled by several participants, including the

Devel::Cover path analysis not present

2013-03-20 Thread James E Keenan
In Devel/Cover.pm's documentation, I read: # =head1 OPTIONS -blib - use blib and ignore files matching \bt/ (default true iff blib directory exists). -coverage criterion - Turn on coverage for the specified criterion. Criteria include statement, branch, condition, path,

Booked by Organisers: which nights

2013-03-18 Thread James E Keenan
Re: http://2013.qa-hackathon.org/qa2013/wiki?node=Accommodation, specifically, the section: Booked by Organisers What nights are covered by this booking? I ask because I may want to arrive in Lancaster on Wed, Apr 10, and if I do I'd like to stay at the same hotel we're using for the event.

Re: TPF Devel::Cover grant report Weeks 30-32

2013-02-11 Thread James E Keenan
On 2/11/13 4:56 PM, Paul Johnson wrote: On Wed, Jan 02, 2013 at 10:39:21PM -0500, James E Keenan wrote: So, thanks again for making this coverage available, and let me recommend that you include all packages included with the Perl 5 core. That's a great suggestion. I have done just

Re: TPF Devel::Cover grant report Weeks 30-32

2013-01-02 Thread James E Keenan
On 1/2/13 9:36 AM, Paul Johnson wrote: In accordance with the terms of my grant from TPF this is the report for weeks 30-32 of my work on improving Devel::Cover. This report covers 08.12 to 28.12. Last week I mentioned the pull requests I had received. I this week I merged a pull request from

Re: Feedback on the undocumented --rules option for 'prove'

2012-08-19 Thread James E Keenan
On 8/17/12 3:53 PM, Mark Stosberg wrote: In 2008 Alex Vandiver contributed a patch to prove that allowed you to specify that you wanted some tests to run in parallel and others in serial. This is a great feature for those of us with large test suites that would like to test advantage of

Re: Feedback on the undocumented --rules option for 'prove'

2012-08-19 Thread James E Keenan
On 8/19/12 1:07 PM, James E Keenan wrote: And it's not alone in being undocumented. Go to http://search.cpan.org/~ovid/Test-Harness-3.25/lib/App/Prove.pm#Attributes and note that 'rules' is one of over 40 attributes which are described as existing but are otherwise undocumented. Slightly OT

Test::Builder2: running coverage fails to display key modules

2012-08-04 Thread James E Keenan
When I run Devel::Cover over the Test::Builder2 test suite (v1.005000_001-193-g00d4609), I get no coverage reports for blib/lib/Test/Builder2.pm or blib/lib/Test/More.pm. See: http://thenceforward.net/test-more/coverage/coverage.html Any thoughts on why that is happening? Thank you very

Re: Test::Builder2: running coverage fails to display key modules

2012-08-04 Thread James E Keenan
On 8/4/12 10:54 AM, Paul Johnson wrote: How did you generate your coverage report? Main part of my shell script: cd $SOURCEDIR \ cover -delete $COVERAGEDIR \ PERL5OPT=-MDevel::Cover=-db,$COVERAGEDIR/ \ perl Makefile.PL make make test \ cover $COVERAGEDIR \ -coverage statement \

Re: Test::Builder2: running coverage fails to display key modules

2012-08-04 Thread James E Keenan
On 8/4/12 11:16 AM, James E Keenan wrote: On 8/4/12 10:54 AM, Paul Johnson wrote: How did you generate your coverage report? Main part of my shell script: cd $SOURCEDIR \ cover -delete $COVERAGEDIR \ PERL5OPT=-MDevel::Cover=-db,$COVERAGEDIR/ \ perl Makefile.PL make make test \ cover

Re: Test::Builder2: running coverage fails to display key modules

2012-08-04 Thread James E Keenan
On 8/4/12 11:34 AM, James E Keenan wrote: On 8/4/12 11:16 AM, James E Keenan wrote: On 8/4/12 10:54 AM, Paul Johnson wrote: How did you generate your coverage report? Main part of my shell script: cd $SOURCEDIR \ cover -delete $COVERAGEDIR \ PERL5OPT=-MDevel::Cover=-db,$COVERAGEDIR

Re: Devel::Cover: uninitialized value warning in 'cover'

2012-06-04 Thread James E Keenan
On 6/4/12 2:46 PM, Matthew Horsfall (alh) wrote: On Sat, Jun 2, 2012 at 11:49 AM, James E Keenanjk...@verizon.net wrote: Use of uninitialized value $Devel::Cover::Ignore_filenames in regexp compilation at /usr/local/lib/perl5/site_perl/5.16.0/i686-linux/Devel/Cover/DB.pm line 882. On a

Devel::Cover: uninitialized value warning in 'cover'

2012-06-02 Thread James E Keenan
As I have described previously, I use Devel::Cover to measure the extent to which parts of the Parrot virtual machine's test suite exercises certain Perl 5 programs and modules in Parrot's configuration and build systems. Today for the first time I noticed this uninitialized value warning,

Re: TPF Devel::Cover grant report Week 1

2012-05-29 Thread James E Keenan
On 5/28/12 7:48 PM, Paul Johnson wrote: What is this? You may recall that recently I submitted a grant proposal to The Perl Foundation to work on Devel::Cover. [snip] So, the work I have completed in the time covered by this report is: Closed RT tickets: 75944 Warning about Moose

Re: Devel::Cover problem: getting analysis of .xs file but not of .pmfile

2012-04-26 Thread James E Keenan
On 4/25/12 9:08 PM, James E Keenan wrote: Friends, Can anyone advise as to why I am getting coverage analysis of Util.xs but not of lib/Hash/Util.pm? Locking %ENV frustrates Devel::Cover. Starting at approximately line 172 of t/Util.t, we have a test which, after I've reformatted a few

Devel::Cover problem: getting analysis of .xs file but not of .pm file

2012-04-25 Thread James E Keenan
Friends, As an extension of my work on https://rt.perl.org/rt3/Ticket/Display.html?id=112126, I am trying to measure the test coverage for Perl 5 core distribution Hash-Util provided by the Perl 5 test suite. For ease of development, I have copied ext/Hash-Util/lib/Hash/Util.pm and

Re: Revert use_ok() change to allow lexical effects?

2012-04-11 Thread James E Keenan
On 4/11/12 12:33 PM, Michael G Schwern wrote: Let me reiterate, I have no plans to *deprecate* `use_ok`. Even if I wanted to there are simply too many users to make deprecation worth while. It works fine if what you want is a runtime require + import + assert, and sometimes you want that.

Perl QA Hackathon: What was accomplished?

2012-04-06 Thread James E Keenan
Could someone post a summary of what was done/not done at this year's QA hackathon in Paris (http://2012.qa-hackathon.org/qa2012)? Thank you very much. Jim Keenan

Re: Devel::Cover: whence the reference to Class::MethodMaker?

2012-03-15 Thread James E Keenan
On 3/14/12 10:30 PM, Michael G Schwern wrote: Is it possible you had a pre-existing cover_db directory? No. Since, to the best of my knowledge, Devel::Cover had never been installed on this machine, I had never tried to run 'cover' on it. On 3/15/12 9:07 AM, Paul Johnson wrote:

Devel::Cover: whence the reference to Class::MethodMaker?

2012-03-14 Thread James E Keenan
On a machine where I can only install libraries underneath my home directory, I today tried to install and use Devel::Cover. After considerable effort, I was able to install Devel-Cover-0.79. But when I went to use it in conjunction with 'prove' I got error output like this: # $

Re: Task: Write a TAP v12 formatter for Test::Builder1.5

2011-11-16 Thread James E Keenan
On 11/16/11 2:15 AM, Michael G Schwern wrote: That's the task. Write a TAP version 12 formatter. Should we proceed on the assumption that this specifies v12? http://testanything.org/wiki/index.php/TAP_specification ... and that this specifies v13?

What does t/test.pl do?

2011-11-05 Thread James E Keenan
In a github issue (https://github.com/schwern/test-more/issues/73), Michael Schwern argued that Test::Builder2 ought to be tested with a completely unrelated test system. We have one, t/test.pl from the Perl core. I thought to myself, How difficult could that be? I vaguely remembered

Re: Small pile of Test::Builder 1.5 refactoring tasks

2011-11-02 Thread James E Keenan
On 11/2/11 4:35 PM, Michael G Schwern wrote: Here's all the current gardening tasks. https://github.com/schwern/test-more/issues?labels=Gardening%2CTest-Builder2sort=createddirection=descstate=openpage=1 I will start to look at these ... but would these be suitable for Google Code-In

Re: Can TAP::Harness::Archive be made to play well with TAP::Parser::Aggregate

2011-10-27 Thread James E Keenan
This is now available on CPAN: http://search.cpan.org/dist/TAP-Harness-Multiple/ Tested on Linux and Darwin but not yet on Windows. Thank you very much. Jim Keenan

Re: Do we need subtests in TAP?

2011-10-26 Thread James E Keenan
On 10/25/11 11:56 PM, Michael G Schwern wrote: I keep looking at subtests and keeping thinking that if there wasn't a test count to manage, would we need subtests? Do we need all that complexity? If it's just about the test count, can it be managed a better way? I haven't followed this

Re: Can TAP::Harness::Archive be made to play well with TAP::Parser::Aggregate

2011-10-22 Thread James E Keenan
On 10/16/11 10:33 PM, James E Keenan wrote: For certain modifications to the Parrot project's smoke testing mechanism, I need to create an archive from an aggregation of test harnesses and then send that archive to our smoke server. Has anyone tried to combine these two modules with good

Re: Can TAP::Harness::Archive be made to play well with TAP::Parser::Aggregate

2011-10-21 Thread James E Keenan
On 10/16/11 10:33 PM, James E Keenan wrote: However, we don't simply want to print this information to the console. When our harness is passed '--archive --send-to-smolder', we want the *aggregated* test output to be archived and transmitted to our smoke server. Unfortunately, I have

Can TAP::Harness::Archive be made to play well with TAP::Parser::Aggregate

2011-10-17 Thread James E Keenan
For certain modifications to the Parrot project's smoke testing mechanism, I need to create an archive from an aggregation of test harnesses and then send that archive to our smoke server. Currently, we create an archive from just a single run of a given testing target. So we have a 'make

Re: Devel::Cover 'time' html column

2010-05-26 Thread James E Keenan
On May 25, 2010, at 3:32 PM, Paul Johnson wrote: Certainly you should be using NYTprof for anything serious in this area, but it didn't exist when I first wrote this code. Now it does, I'm considering removing the time coverage. Does anyone have any thoughts one way or the other about

Re: Devel::Cover 'time' html column

2010-05-25 Thread James E Keenan
Dan MacNeil wrote: What information is in the 'time' column ? Like you, I could not find any information about this in any of: perldoc Devel::Cover perldoc cover perldoc Devel::Cover::Tutorial ISTR a post by Paul Johnson about this a few years back. You might want to search the

Re: qa.perl.org

2010-01-01 Thread James E Keenan
Leo Lapworth wrote: Hi, Please note this is cross posted. Several of the QA people have kindly said they will help with updating http://qa.perl.org/. 3) Phalanx Though in one sense it pains me to say it, Phalanx does not need to be a major tab. It can be demoted to a link somewhere.

Devel::Cover: metadata about files being reported on

2008-09-04 Thread James E Keenan
At http://thenceforward.net/parrot/coverage/configure-build/coverage.html I have for over a year displayed the results of coverage analysis on Parrot's configuration and build tools. I have come to realize that while these reports are very useful for me as the maintainer of the Perl 5 aspect

Re: Devel::Cover anomaly: Double reporting of statement coverage

2008-08-28 Thread James E Keenan
Paul Johnson wrote: There does seem to be something rather strange going on here. The results for the unshift a few lines earlier are also duplicated. Yes, I noticed that after the OP. But more stragely, the statement counts on lines 109 and 124 seem somewhat excessive. My best guess

Re: Devel::Cover anomaly: Double reporting of statement coverage

2008-08-28 Thread James E Keenan
Paul Johnson wrote: If you delete all the coverage information and start from scratch does that solve the problem? Problem was reproduced: http://thenceforward.net/parrot/coverage/configure-build/config-auto-pmc-pm.html

Devel::Cover anomaly: Double reporting of statement coverage

2008-08-27 Thread James E Keenan
In the attachment, I show several lines from this file coverage report on a Perl 5 module in the Parrot distribution: http://thenceforward.net/parrot/coverage/configure-build/config-auto-pmc-pm.html http://tinyurl.com/5fovnc Two statements, lines 261-262, each of which is a simple 'push'.

Re: Rebuilding the Test::Builder community

2008-08-10 Thread James E Keenan
Michael G Schwern wrote: Next step is to get people who want to contribute to the project signed up as project members. To do this, please send me your Google account email address. And a reminder to those who have signed up: When you go to do your initial 'svn co', the password you

Re: cpants.perl.org giving 404 not founds

2008-06-21 Thread James E Keenan
Paul Fenwick wrote: G'day QA folks, Just wanted to report that: http://cpants.perl.org/ is presently giving me 404 (not found) errors. Appears fixed as of the time of this message.

Re: Non-standard use of Devel::Cover

2008-06-08 Thread James E Keenan
Rick Fisk wrote: Has anyone been successful with producing meaningful coverage reports when the test files are in a non-standard location? Any help would be appreciated. If you make judicious use of the command-line options to cover, as well as to Devel::Cover's own options, you can

Re: best practices for temporary files during testing

2008-06-08 Thread James E Keenan
Gabor Szabo wrote: Should I just use File::Tempdir ? I haven't yet explored File::Tempdir. But File::Temp has the advantages of being core, having both functional and OO interfaces, and widely used. We use it extensively in testing Parrot -- and just tonight I converted one case where

YA Hackathon Topic: Help Improve Parrot's Smoke Testing

2008-03-31 Thread James E Keenan
If by some chance you run out of things to hack on, you can help out the Parrot project by taking a look at our smoke testing system -- specifically, its reliance on, and hackish overriding of, two CPAN modules.

Re: Hackathon logistics

2008-03-26 Thread James E Keenan
Michael G Schwern wrote: My concern is that we'll be spending time futzing with git rather than hacking on QA stuff. We all know SVN/SVK and can stand to use it for a little while longer. Agreed. One hackathon I attended wasted one-third of the participants' available time because

My Perl QA Hackathon Wishlist

2008-03-24 Thread James E Keenan
1. A canned training session, Learn Test::Harness 3.0, that would provide the user with a guided tour of T::H 3+ and have him/her actually type and run examples of the new functionality. Canned in the sense that at a local Perlmongers meeting everyone could download a tarball and work

Perl Testing and QA at YAPC::NA::2008 in Chicago, June 16-18

2008-03-09 Thread James E Keenan
A couple of points about YAPC::NA: 1. It's almost two weeks earlier this year than it has been in the past three years. So if your mental clock has been set to 3 days of YAPC + 2 days of hackathon + 3-day 4th-of-July weekend (as mine has), please reset that clock. 2. Consequentially, the

Re: Providing command-line arguments to tests run via 'prove'

2008-02-26 Thread James E Keenan
James E Keenan wrote: I would like to be able to provide the tests run via 'prove' with options something like this: some_variant_of_prove t/*.t --option1 --option2 arg1 arg2 ... where those 4 command-line options/arguments would be available to *each* of the individual test files

Re: CPANTS Site Seems... Confused

2008-02-22 Thread James E Keenan
Thomas Klausner wrote: which wasn't proof-read by any english speaking person yet, so if @you want to: patches welcome ;-) s/not longer dreadlocked/no longer dreadlocked/

Re: buildbot - an experiment

2008-01-04 Thread James E Keenan
David Cantrell wrote: On Tue, Jan 01, 2008 at 08:23:52PM -0500, James E Keenan wrote: David Cantrell wrote: If anyone can give me an idiots' guide to how to grab the most recent source tree, build it, and test it, then I can test it on the same boxes as I do CPAN testing, plus maybe a couple

Re: buildbot - an experiment

2008-01-02 Thread James E Keenan
David Cantrell wrote: On Sat, Dec 29, 2007 at 05:51:50PM -0500, James E Keenan wrote: If anyone can give me an idiots' guide to how to grab the most recent source tree, build it, and test it, then I can test it on the same boxes as I do CPAN testing, plus maybe a couple of others. svn co

Re: buildbot - an experiment

2007-12-29 Thread James E Keenan
Matisse Enzer wrote: I've spent some of this holiday season learning how to set up BuildBot (http://buildbot.net/) which is a Continuous Integration system that is especially aimed at open-source style projects: You set up a central build master, and one or more build slaves - and it is very

Re: Auto: Your message 'FAIL IO-AIO-2.51 i386-freebsd-thread-multi 6.2-release' has NOT been received

2007-12-20 Thread James E Keenan
Ovid wrote: --- Dave Rolsky [EMAIL PROTECTED] wrote: [snip] I think there's some truth to this view. For support I submit this bug ticket - http://rt.cpan.org/Ticket/Display.html?id=27208 Sorry Dave, but I understand Imacat's point of view. I think part of the issue is that English is not

Re: Milton Keynes PM coding collaboration

2007-12-12 Thread James E Keenan
Edwardson, Tony wrote: Anyone written any CPAN modules for which the testing coverage needs to be improved ? [snip] Something from http://qa.perl.org/phalanx/ http://qa.perl.org/phalanx/ would be good Aspiring hoplites, I salute you! You can read about the adventures of grizzled

Providing command-line arguments to tests run via 'prove'

2007-11-24 Thread James E Keenan
Let's suppose that I have a suite of test files which I customarily run with 'prove': prove t/*.t ... where the tests in t/ are: alpha.t beta.t gamma.t Let's further suppose that each of these three tests simulates the operation of a Perl program which is normally called with

Re: Devel::Cover Oddity: Test returned status 0 ... after all subtestscompletedsuccessfully

2007-10-29 Thread James E Keenan
James E Keenan wrote: James E Keenan wrote: James E Keenan wrote: t/configure/029-option_or_datadubious Test returned status 0 (wstat 11, 0xb) after all the subtests completed successfully I'm still in much the same situation as I was last month. I'll

Re: Devel::Cover Oddity: Test returned status 0 ... after all subtestscompletedsuccessfully

2007-09-23 Thread James E Keenan
James E Keenan wrote: James E Keenan wrote: t/configure/029-option_or_datadubious Test returned status 0 (wstat 11, 0xb) after all the subtests completed successfully This problem is maddening because it is intermittent. After not appearing for more

Re: Planning talk on Devel::Cover

2007-09-21 Thread James E Keenan
Michael Carman wrote: On 9/12/2007 7:41 PM, James E Keenan wrote: I'd also like to hear any other coverage-related ideas you think would be worthwhile bringing up at this workshop. Here's my 2¢ For applications with code in multiple modules use the options for including and excluding files

Re: Planning talk on Devel::Cover

2007-09-21 Thread James E Keenan
On Sep 21, 2007, at 10:35 AM, Paul Johnson wrote: Actually, the options Michael gave *are* for Devel::Cover itself. There are similar (but slightly different - and I hate that) options for cover. Well, then, I'm glad I began this thread. While I'm a D::C enthusiast, I guess my own

Re: Planning talk on Devel::Cover

2007-09-13 Thread James E Keenan
On Sep 13, 2007, at 2:14 AM, Joshua ben Jore wrote: Do you know of any addons to the cover program to use information like this method call site always resolves to the following destinations or if execution went through this part, what else probably happened? Sorry, I don't know of any

Planning talk on Devel::Cover

2007-09-12 Thread James E Keenan
I have had a proposal accepted to do a presentation at the Pittsburgh Perl Workshop (Oct 13-14) on Better Code via Coverage Analysis during Testing (http://pghpw.org/ppw2007/talk/725). During this presentation I hope to: 1. Channel pjcj to the best of my ability. 2. Share some things I've

Re: Test::Harness 2.99 again

2007-09-03 Thread James E Keenan
Eric Wilhelm wrote: Last week was me spending way too much time programming with two hands behind my back to get 5.5.3 compatibility, bundling Test::More, creating IO::Capture, etc. In this package's POD, it would probably be good to distinguish this package from the IO::Capture

Re: Win32:GuiTest

2007-09-01 Thread James E Keenan
Smylers wrote: James E Keenan writes: The subject of this thread is Win32::GuiTest -- I know nothing about it, but given it's a module for use with testing it seems on-topic for this list. You are correct. I focused exclusively on the body of the message and not on the subject. Sorry.

Re: Win32:GuiTest

2007-08-31 Thread James E Keenan
Gergely Brautigam wrote: Hi There! Does anybody know why is that so that sometimes when trying to set a focuse to a window the window does not come into forground but it stays on the menubar and just blinks...? Is that a windows problem or a Win32 module problem? Hi Gergely. Glad to have

Devel::Cover Oddity: Test returned status 0 ... after all subtests completed successfully

2007-08-25 Thread James E Keenan
Working on Parrot, I frequently do coverage analysis of the tests I and others have written for Parrot's configuration and build tools, components of the project which are written in Perl 5. I display the results on my server:

Re: Devel::Cover Oddity: Test returned status 0 ... after all subtestscompleted successfully

2007-08-25 Thread James E Keenan
James E Keenan wrote: t/configure/029-option_or_datadubious Test returned status 0 (wstat 11, 0xb) after all the subtests completed successfully I see that the source of this error message is in Test::Harness. But I don't yet understand why it gets

Re: Summarizing the pod tests thread

2007-07-31 Thread James E Keenan
David Golden wrote: * Module::Starter -- Andy wants these tests run by default and he doesn't seem to be swayed; So people need to fork this and start advocating for an alternative. Ditto other module generators. (And mea culpa for my own.) You don't need to fork Module::Starter. You

Re: .t files as specs

2007-06-19 Thread James E Keenan
Andy Lester wrote: On Jun 19, 2007, at 10:52 AM, Mike Malony wrote: So I'm working my project, and I've got one other more junior coder with me. Has anyone tried writing test files as part of their spec's? An overview document would also be needed, and some time walking through the

Re: Unit vs. use case/functional testing

2007-03-02 Thread James E Keenan
Adrian Howard wrote: [snip] Adrian: How about posting this part on http://perl-qa.yi.org/index.php/Main_Page? For more general testing discussions I'd recommend joining all of: * [EMAIL PROTECTED] * [EMAIL PROTECTED] * [EMAIL PROTECTED] You also might want to look at all or some of: *

Re: Devel::Cover newbie - what does 'yourprog' have to be?

2007-02-02 Thread James E Keenan
Mike Malony wrote: I'm into testing, got some nice .t files, and prove tells me things I'd rather not hear. So, my next step on the straight and narrow path of testing, is to gauge my testing coverage. IN the doc, the synopsis suggests perl -MDevel::Cover yourprog args cover But what can

A Modest Request

2007-01-23 Thread James E Keenan
What with all the activity on this list in the last week (TAPx::Parser about to morph into Test::Harness), it's all been more than I can keep up with. I would like to suggest that one or more or the hackerati who are working on all these revisions to our core testing functionality write an

Re: Test::Builder feature request

2007-01-13 Thread James E Keenan
chromatic wrote: the Star Trek: Generations fallacy. You steal a spaceship, which flies through space, to fly through space to a planet, flying through space, where a temporal anomaly, which also flies through space, deflected by a supernova, which you flew through space in your spaceship

Re: What do I make of this CPANTS failure report?

2006-11-26 Thread James E Keenan
A. Pagaltzis wrote: Hi all, I just got a failure report [1] on one of my modules that I don’t know how to read correctly. There’s only one line pertaining to an error of any sort, and it only mentions the type of error, but not where or how it happened. What I’d like to know first and foremost

  1   2   >