Why not a perl-current smoke test database ?
(I re-send this, I didn't find it on list) After see a smoke report from Nicolas Clark using ccache, I took a look at ccache web site. And find http://build.samba.org. So I've begin something like this, thing I would see on http://qa.perl.org: http://www.alianwebserver.com/cgi-bin/smoke_db Comments are welcome. -- Alain BARBET
Why not a smoke db ?
After see a smoke report from Nicolas Clark using ccache, I took a look at ccache web site. And find http://build.samba.org. So I've begin something like this, thing I would see on http://qa.perl.org: http://www.alianwebserver.com/cgi-bin/smoke_db Comments are welcome. -- Alain BARBET
Re: Why not a perl-current smoke test database ?
> * searchable for the past (and for keywords in failures or fulltext, like > "bigint" Yep I will add this shortly. > * spares me the "smoke foo" messages, that contain all Ok and fool me into > thinking there was some smoke Sorry my so poor english doesn't understand that. Coud you explain more about that ? Regards, -- Alain BARBET
Re: Why not a smoke db ?
> Oooh, they've got a Cray in theirs. See (1) > Part in English, part in French? :-) Oh yes, Compilateur => Compiler > I suppose that I should be thankful that it's not part Finnish, part Tamil. (1) Damm as you see, my english is not the best langage interface with me and I can't understand these two sequences. (sorry this give me lot of frustation to not understand what you mean :-( ) > And as I seem to be running 1.06 (with tweaks) this would explain why you're > not seeing it. Waiting update, can you provide your uname -a pls ? > Nor are all Solaris boxes sparc. > (or x86 for that matter - it was internally ported to alpha to check 64 bit > cleanliness before true 64 bit sparc chips were available) Yes. I will put it manually waiting update of Test-Smoke. (I think know all smoker config and ask for others.)
Re: Why not a smoke db ?
> Do you get it from daily-build only? Yes even if I take some report on p5p because: 1/ daily-build is the best place to find report for daily-build no ? ;-) 2/ p5p have to more traffic 3/ this is easlier to add a parser that auto. fetch report on this newsgroup.
Smoke db (was perl-current doesn't compil on MinGw-2.0.0-3)
http://www.alianwebserver.com/cgi-bin/smoke_db updated. Reports from 17500 => 17967. My db server on alianwebserver allow me to only fetch 60k per cgi request, so I've limit to 17500 smoke reports. (don't try "All" with all browse list, it's same problem, it's stop before display). If we can find another server else I will install it elsewhere in 1 month. PS: failed tests from Merijn reports are not parsed for now :-( Regards, -- Alain BABRET
Test::Smoke::Database
The uploaded file Test-Smoke-Database-1.2.tar.gz has entered CPAN as file: $CPAN/authors/id/A/AL/ALIAN/Test-Smoke-Database-1.2.tar.gz NAME Test::Smoke::Database - Add / parse /display perl reports smoke database SYNOPSIS $ admin_smokedb --rename --suck --import --update_archi $ lynx http://localhost/cgi-bin/smokedb.cgi DESCRIPTION This module help to build an application that parses smoke-reports for perl-current and puts the results in a database. This allows for a simple overview of the build status on as wide a variety of supported platforms (operating system/architecture) as possible. This distribution come with 2 perl scripts: admin_smokedb Fetch / Import smoke report in a mysql database. See admin_smokedb smokedb.cgi A www interface to browse this smoke database SEE ALSO admin_smokedb, Test::Smoke::Database::FAQ, Test::Smoke
Re: Test::Smoke::Database
The uploaded file Test-Smoke-Database-1.4.tar.gz has entered CPAN as file: $CPAN/authors/id/A/AL/ALIAN/Test-Smoke-Database-1.4.tar.gz size: 23532 bytes md5: 9bdcffb8b057004df3cde7a2af6e30cd Changes: 1.4 2003/01/05 21:45:55 alian - Fix for parsing hm. brand reports with 5.6 - Fix test with 5.6 1.3 2003/01/05 01:15:55 alian - Add a special parser for HM Brand's reports - Remove --rename option - Rewrite code for better daily use with no --clear option - Add tests for report parsing - Update POD
Re: Test::Smoke::Database
Example: http://www.alianwebserver.com/perl/smoke/smoke_db.cgi?last_smoke_fil=18390 POD doc: http://www.alianwebserver.com/informatique/languages/perl/Test/Smoke/Database.html I put this module on CPAN because my ISP have set very low limit for MySQL request. I can't do an application like this with this limitation and hope that someone can handle this on his own server with less problem.
Some funny graph about smoke bleadperl
Number of configure run / number of configure that pass all test by smoke: http://www.alianwebserver.com/tmp/conftested.png Number of os tested by smoke: http://www.alianwebserver.com/tmp/ostested.png 17865 has been made in september 2002. On the good way ... PS: Here a futur feature for smokedb. Idea welcome. -- Alain BARBET
Re: Some funny graph about smoke bleadperl
Some others numbers: http://www.alianwebserver.com/perl/smoke/stats.html
Re: Some funny graph about smoke bleadperl
Some others numbers: http://www.alianwebserver.com/perl/smoke/stats.html Same but just for 5.9.0 (all smoke after 17700) Big "smoker" by os: HM Brand for HP UX, just before Jarkko with dec_osf. Big "configurer" by os: Nicholas Clark for FreeBSD http://www.alianwebserver.com/perl/smoke/5.9.0/
Re: Test::Smoke::Database
The uploaded file Test-Smoke-Database-1.5.tar.gz has entered CPAN as file: $CPAN/authors/id/A/AL/ALIAN/Test-Smoke-Database-1.5.tar.gz size: 28921 bytes 1.5 2003/02/10 00:58:05 alian - Add feature of graph - Correct Irix report parsing (no os/archi version) - Correct number of failed test - Read archi from 1.16 report - Update parsing of error of HM Brand reports - Update display for cgi - Remove --update-archi option. Done after every import Here an example of this last week smoke: http://www.alianwebserver.com/perl/smoke/smoke_db.cgi?smoke_fil=Last&last_smoke_fil=18601 Graph are added on navigation bar. See "stats". 3 parts: since beginning of smoke, since perl-5.9, last 50 smoke. Idea / comment welcome. -- Alain BARBET
Re: Test::Smoke::Database
Just one knit: The new 1.16_15+ reports with locale (6 columns) are not parsed correctly :-( http://www.alianwebserver.com/perl/smoke/smoke_db.cgi?last=1&smoke=18680#2614 The uploaded file Test-Smoke-Database-1.6.tar.gz has entered CPAN as file: $CPAN/authors/id/A/AL/ALIAN/Test-Smoke-Database-1.6.tar.gz size: 32774 bytes md5: 53f3e152ed9696ecf5caca5c21f4e4f1 1.6 2003/02/16 18:47:04 alian - FORMAT OF DATABASE UPDATED ! - Update summary table: add number of configure failed, number of make failed. - Add legend after summary table - Add parsing/display of matrice, as Test::Smoke 1.16_15+ can report more than 4 columns - Correct a bug that add a 'Failure:' in HM Brand Report - Add stats about CPAN testers - Change default limit for CGI: 18600. - Update test files, add some reports. A quick relay with CPAN tester: Here some stats about CPAN testing: http://www.alianwebserver.com/perl/smoke/cpan.html Perl 5.8 with 6 month of life is the perl most tested with CPAN modules! Thanks for CPANPLUS for that :-)
Test::Harness::Straps
Hi, First, thanks for your QA job ! Next, I ask you an advice for Test::Harness::Straps. (project /1) I today use a lot of script that use Test::More methods. I run them from several ways: from cron or from cgi. I use my own function to parse result, and display them as I want. Yesterday (project /2), I see the long know crlf bug in cygwin in strap-analyse.t and try to find it. I see the doc of Test::Harnness::Straps and think that I can use in 1/ But I miss something in result ... there is no info about why some tests fails. reason field are only for skip or todo block. This is what you want or ? Can you speak about this ? Thanks, -- Alain BARBET
Re: testers.cpan.org
Out of interest, is there a reason http://testers.cpan.org/ is so slow? Huge size of database ?
About Devel::Cover
Hi, First thanks for your talk on YAPC 2003 at Paris on this subject. The first thing i've do when I was come back was to test your module. Here feedback. In the README: Requirements: - Perl 5.6.1 or 5.7.1. * May be 5.8.0|1 ? ;-) * Or 5.6.1 or greater ? - The -delete flag of cover isn't documented I want use this module to see the cover of my cpan module, and think a lot of people using your module want do same thing. But if I ran test scripts, I've only cover for the test script, not the module. I found this, after a lot of try and with the help of cpancover. May be this can be added to the doc: -- For test the cover of a CPAN module, cd in the directory with your Makefile.PL is, and then do: $ for i in $(ls t/*.t); do \ perl -I$PWD/blib/lib -I$PWD/blib/arch \ -MDevel::Cover=-ignore,t/,-select,blib/lib $i; done; or the easy way, use the HARNESS_PERL_SWITCHES: (with bash) $ export HARNESS_PERL_SWITCHES=\ "-MDevel::Cover=-ignore,t/,-select,blib/lib" $ make test. But with this method, i've two cover_db created: ls -l cover_db/ t/cover_db/ cover_db/: total 40 -rw-r--r--1 alianusers 39217 jui 27 18:48 cover.4 t/cover_db/: total 96 -rw-r--r--1 alianusers 91374 jui 27 18:49 cover.4 The first one is in fact empty. So I need to cd to t, and then cover -report html. But I receive this: $ cover -report html Reading database from cover_db -- -- -- -- -- Filestmt branch cond time total -- -- -- -- -- blib/lib/Test/Smoke/Database.pm31.06 29.62 31.44 100.00 30.64 blib/lib/Test/Smoke/Database/Graph.pm 0.00 0.00 0.00n/a 0.00 Total 22.93 24.13 26.29 100.00 23.78 -- -- -- -- -- HTML output sent to /home/alian/project/CPAN/Test/Smoke/Database/t/cover_db/cover_db.html Unable to open blib/lib/Test/Smoke/Database.pm: Aucun fichier ou répertoire de ce type Exiting subroutine via next at /usr/local/perl-5.8.0/lib/site_perl/5.8.0/i686-linux-thread-multi-64all-ld/Devel/Cover/Report/Html.pm line 95. Unable to open blib/lib/Test/Smoke/Database/Graph.pm: Aucun fichier ou répertoire de ce type Exiting subroutine via next at /usr/local/perl-5.8.0/lib/site_perl/5.8.0/i686-linux-thread-multi-64all-ld/Devel/Cover/Report/Html.pm line 95. Yes, normal, the blib is one level up in the directory. So I delete the empty one, mv the t/cover_db directory to his parent, and cover become happy. Thanks for your module. May be more feedback later. -- Alain BARBET
Re: About Devel::Cover
But with this method, i've two cover_db created: saturne:~/project/CPAN/Test/Smoke/Database/t$ grep chdir * 2normal_report.t:chdir(dirname($0)); 3hmbrand_report.t:chdir(dirname($0)); It doesn't seem like this ! With another module without chdir, I've only one cover_db database created.
Re: About Devel::Cover
Sorry, an another part of doc missing: How use Pod::Coverage from Devel::Cover ? Thank you, -- Alain BARBET
Re: About Devel::Cover
Here a patch for what I just say ... HTH, -- Alain BARBET diff -ru Devel-Cover-0.20-old/BUGS Devel-Cover-0.20/BUGS --- Devel-Cover-0.20-old/BUGS Sat Oct 5 19:16:10 2002 +++ Devel-Cover-0.20/BUGS Wed Jul 30 11:29:07 2003 @@ -1,2 +1,6 @@ - Code in BEGIN and END blocks is not reported. - Doesn't play nicely with Perl's testsuite. +- Sometimes (!), cover_db database will be dropped +- If you have chdir call in your test, you need to +use an absolute path for cover db, else you will +have several database. diff -ru Devel-Cover-0.20-old/cover Devel-Cover-0.20/cover --- Devel-Cover-0.20-old/cover Sat Oct 5 19:16:10 2002 +++ Devel-Cover-0.20/cover Wed Jul 30 11:30:07 2003 @@ -157,6 +157,7 @@ -file filename- only report on the file (default all) -write [db] - write the merged database(default off) + -delete - drop selected cover database -coverage criterion - report on criterion (default all available) diff -ru Devel-Cover-0.20-old/lib/Devel/Cover.pm Devel-Cover-0.20/lib/Devel/Cover.pm --- Devel-Cover-0.20-old/lib/Devel/Cover.pm Sat Oct 5 19:16:10 2002 +++ Devel-Cover-0.20/lib/Devel/Cover.pm Wed Jul 30 11:42:38 2003 @@ -571,6 +571,10 @@ perl -MDevel::Cover=-db,cover_db,-coverage,statement,time yourprog args + # of for a CPAN module + export HARNESS_PERL_SWITCHES="-MDevel::Cover=-ignore,t/,-select,blib/lib" + make test + =head1 DESCRIPTION This module provides code coverage metrics for Perl. @@ -604,7 +608,7 @@ Requirements: - Perl 5.6.1 or 5.7.1. + Perl 5.6.1 or greater. The ability to compile XS extensions. Pod::Coverage if you want pod coverage. Template Toolkit 2 if you want HTML output. @@ -612,7 +616,10 @@ =head1 OPTIONS -coverage criterion - Turn on coverage for the specified criterion. - -db cover_db- Store results in coverage db (default cover_db). + Criterion can be in: all, none, statement, branch, + condition, default all) + -db cover_db- Store results in coverage db (default cover_db + in current directory). -inc path - Set prefixes of files to ignore (default @INC). +inc path - Append to prefixes of files to ignore. -ignore RE - Ignore files matching RE. @@ -640,6 +647,7 @@ =head1 BUGS Did I mention that this is alpha code? +See the BUGS file. =head1 VERSION
Re: Testers & PASS
My point is that the CPAN Testers reports for fails have the output of make test, eg: http://nntp.x.perl.org/group/perl.cpan.testers/96865 ... but passes don't: http://nntp.x.perl.org/group/perl.cpan.testers/96886 This is inconsistent and we're throwing information away. I say put it in all the time. In the same order (may be I think more important), is to set TEST_VERBOSE=1 as default for cpan tester. I think too as cpan tester and as cpan author that browse ok report can be great. But there is a serious problem with CPAN test database. There is like 100 000 reports in the CPAN db. Perl 5.8.0 has been 50% more tested than perl-5.6.0, and 150% more than perl-5.003. What will be the futur for this database ? Too big. Too slow. What do now ? Add more informations without take a look on the size of the database ? Regards, -- Alain BARBET
Re: Testers & PASS
Thus, I think that my website is correct in sorting the version
numbers. 1.2 should be later than 1.18. I think your versioning system
is wrong ;-)
Ok as this is not *my* versioning system, I will explain.
I use CVS, and extract version from something like:
$VERSION = ('$Revision: 1.18 $ ' =~ /(\d+\.\d+)/)[0];
I don't remember where I see this first time.
But after a google search, it seems I must do in fact:
$VERSION = sprintf("%d.%02d", q$Revision ... $);
Re: Testers & PASS
Comments welcome, Leon About order of v-string: It would be good in details results that 1.18 would be first before 1.2. Eg: 1.18 (2 PASSes) 97757 PASS sun4-solaris 97976 PASS i586-linux 1.17 (1 FAIL) 97620 FAIL sun4-solaris 1.16 (1 FAIL) 97484 FAIL sun4-solaris in place of 1.3 (3 PASSes) 22914 PASS sparc-openbsd 22988 PASS sun4-solaris 23106 PASS i686-linux 1.2 (5 PASSes) 20449 PASS PA-RISC1.1 20450 PASS ppc-linux 20451 PASS sun4-solaris 20461 PASS MSWin32-x86-multi-thread 20622 PASS sparc-openbsd 1.18 (2 PASSes) 97757 PASS sun4-solaris 97976 PASS i586-linux 1.17 (1 FAIL) 97620 FAIL sun4-solaris Regards, -- Alain BARBET
Re: Testers & PASS
http://search.cpan.org/author/LBROCARD/CPAN-WWW-Testers/ http://testers.astray.com/ I just see that testers.cpan.org now use your interface. Cl. Faaast. But some problems ... We use in CPANPLUS old interface, old url to fetch reports about a dist. So there is no longuer report from CPANPLUS testers since yesterday. Ok, I patch this for use the great yaml I found one the new site, but there is missing info into the yaml file: When reports are failed, we just know: "this fail". I need another field like 'detailed_results' who is an url to detailed report (today http://nntp.x.perl.org/group/perl.cpan.testers/xxx). Can you add this ? Regards, -- Alain BARBET
Re: Testers & PASS
But some problems ... We use in CPANPLUS old interface, old url to fetch reports about a dist. So there is no longuer report from CPANPLUS testers since yesterday. Which interface is this? We can probably fake it with a mod_rewrite rule if you tell me the details. The patch is already done. But if there is others apps that use that: http://testers.cpan.org/search?request=dist&dist=$name who is now: http://testers.cpan.org/show/$name But as the format has change, I don't know if this is really needed. Ok, I patch this for use the great yaml I found one the new site, but there is missing info into the yaml file: When reports are failed, we just know: "this fail". I need another field like 'detailed_results' who is an url to detailed report (today http://nntp.x.perl.org/group/perl.cpan.testers/xxx). If you replace the xxx with the ID then you have the correct URL. Great, I didn't see that. The next release of CPAN testers will have a report_url key containing this. Thank you, -- Alain BARBET
Re: Testers & PASS
Ok, I patch this for use the great yaml I found one the new site, but there is missing info into the yaml file: Can't you include os name and os version that can be found in all reports: osname=solaris, osvers=2.8, archname=sun4-solaris This is displayed in the old interface with all result, now you only display archname, ok, as you want, but in the yaml file it would be great to find this info. Regards, -- Alain BARBET
Re: Phalanx has started, and I need perl-qa's help
Hi, I just start smoke on this "top100" with cpanplus. Why, as you display docs, bugs and rating, you didn't display link to cpan testers results ? Regards, -- Alain BARBET
Re: Test::Smoke::Database
Test-Smoke-Database-1.12 http://cpanplus.keradel.com/cgi-bin/smoke_db.cgi Comments welcome, -- Alain BARBET
Test-Smoke-Database-1.14
The uploaded file Test-Smoke-Database-1.14.tar.gz has entered CPAN as file: $CPAN/authors/id/A/AL/ALIAN/Test-Smoke-Database-1.14.tar.gz size: 45076 bytes md5: a5aac8a73f06e47af430307cd94b199b 1.14 2003/08/19 10:37:24 alian - FORMAT OF DATABASE UPDATED ! (two cols added, one moved). - Add a 'version' field to filter/parser (Eg: All perl-5.8.1 report) - Use the field 'date' into filter/parser (Eg: All report after 07/2003) - Add an author field to parser, and a smoker HTML page about recent smokers and their available config. - Change how nbte (number of failed tests) is calculate - Graph are done by month, no longuer with patchlevel - Only rewrite cc if gcc. Else we lost solaris info - Remove ccache info for have less distinct compiler - Add another report to tests - Update FAQ.pod for last Test::Smoke version - Save only wanted headers for each nntp articles (and save From: field). - Move away last varchar field from builds to data Here example of new feature: "All smoke for perl-5.8.1 this month": http://cpanplus.keradel.com/cgi-bin/smoke_db.cgi?os_fil=All&osver_fil=All&archi_fil=All&cc_fil=All&ccver_fil=All&smoke_fil=Last&last_smoke_fil=All&version_fil=5.8.1&date_fil=2003-08-01+00%3A00%3A00&.submit= Regards, -- When the speaker and he to whom he is speaks do not understand, that is metaphysics. -- Voltaire
Re: FreeBSD on Alpha.
I may be able to lay my hands on an alpha running FreeBSD (I don't yet know version numbers). Is there any interest in running smokes on that platform combination, or is that already well covered? http://cpanplus.keradel.com/cgi-bin/smoke_db.cgi?os_fil=All&osver_fil=All&archi_fil=All&cc_fil=All&ccver_fil=All&smoke_fil=Last&last_smoke_fil=All&version_fil=5.8.1&date_fil=2003-08-01+00%3A00%3A00&.submit= shows me no such combo yet, so yes please I just update to 1.15, that parse 5.6.1 reports with their only 2 cols in reports. (for you Rafael ;-) http://cpanplus.keradel.com/cgi-bin/smoke_db.cgi?os_fil=All&osver_fil=All&archi_fil=All&cc_fil=All&ccver_fil=All&smoke_fil=Last&last_smoke_fil=All&version_fil=5.6.1&date_fil=All&.submit= and add a top20 smokers ;-) at end of page http://cpanplus.keradel.com/cgi-bin/smoke_db.cgi?smokers=1 But I've some problems with graph since www cpan-testers has changed. I can't no longuer make this: http://cpanplus.keradel.com/perl/smoke/cpan.html with this url: http://testers.cpan.org/search?request=by-config which give today 404. Leon can you make something for that ? Regards, -- Jone's Motto: Friends come and go, but enemies accumulate.
