Re: run C++ TAP output? (even even even easier)

2007-03-09 Thread Eric Wilhelm
# from Eric Wilhelm # on Friday 09 March 2007 06:26 pm: >However, if you feel the need for speed, I *think* you could add > support in runtests for "--exec ''" to turn into @exec=() rather than > @exec=(''). Darn! That sounded like so much fun. I just couldn't resist. http://svn.hexten.net/t

Re: run C++ TAP output? (even even easier)

2007-03-09 Thread Eric Wilhelm
# from Julien Beasley # on Friday 09 March 2007 03:39 pm: >Thanks Ovid! This may be exactly what I'm looking for (since I'm going > to have tests in libtap and perl). However, and I apologize if I'm > wrong about this, doesn't your proposed solution have to start a new > perl interpreter for every

Re: The most important TAP tasks.

2007-03-09 Thread Andy Lester
On Mar 9, 2007, at 7:54 PM, Michael G Schwern wrote: Better yet, I can make it available in public Subversion so that you can edit it, if you're so inclined. Public as in world-writable? Or public as in read-only and then I have to write up a patch and email it and wait for it to be acc

Re: The most important TAP tasks.

2007-03-09 Thread Michael G Schwern
Andy Lester wrote: > > On Mar 9, 2007, at 5:17 PM, Michael G Schwern wrote: > >> since. There's mistakes, typos, ommissions, you name it. Work on >> it. Put >> it on a POD wiki to allow easier collaborative editing. Talk with Ovid >> about where the ambiguities are, he's had to work them out

Re: Test::Files messes with TODO tests in Test::Harness?

2007-03-09 Thread Phil Crow
--- Michael G Schwern <[EMAIL PROTECTED]> wrote: > Julien Beasley wrote: > > Hi, > > > > I've found that using Test::Files in a test script > changes the output of > > TODO tests in Test::Harness. > > Here's the problem. > > $ perl -wle 'use Test::Files; print > Test::Builder->new->exported_t

Re: The most important TAP tasks.

2007-03-09 Thread Andy Lester
On Mar 9, 2007, at 5:17 PM, Michael G Schwern wrote: since. There's mistakes, typos, ommissions, you name it. Work on it. Put it on a POD wiki to allow easier collaborative editing. Talk with Ovid about where the ambiguities are, he's had to work them out for TAP::Parser. Better yet,

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Michael G Schwern
Julien Beasley wrote: > Well at my last job, we had hundreds of test files.. and most of them were > really fast because we wanted to keep the total time to a minimum. Even > then, it took over five minutes to run all of our tests, and that was > getting to be Too Long. So I could definitely see in

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Andy Armstrong
On 9 Mar 2007, at 23:57, Julien Beasley wrote: Well at my last job, we had hundreds of test files.. and most of them were really fast because we wanted to keep the total time to a minimum. Even then, it took over five minutes to run all of our tests, and that was getting to be Too Long. So I

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Julien Beasley
Well at my last job, we had hundreds of test files.. and most of them were really fast because we wanted to keep the total time to a minimum. Even then, it took over five minutes to run all of our tests, and that was getting to be Too Long. So I could definitely see in a case like that that the ov

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Andy Armstrong
On 9 Mar 2007, at 23:39, Julien Beasley wrote: Thanks Ovid! This may be exactly what I'm looking for (since I'm going to have tests in libtap and perl). However, and I apologize if I'm wrong about this, doesn't your proposed solution have to start a new perl interpreter for every single tes

Re: run C++ TAP output? (even easier)

2007-03-09 Thread chromatic
On Friday 09 March 2007 15:39, Julien Beasley wrote: > However, and I apologize if I'm wrong about > this, doesn't your proposed solution have to start a new perl interpreter > for every single test file? If so, that might up being too slow for > practical use. That would surprise me; I would exp

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Julien Beasley
Thanks Ovid! This may be exactly what I'm looking for (since I'm going to have tests in libtap and perl). However, and I apologize if I'm wrong about this, doesn't your proposed solution have to start a new perl interpreter for every single test file? If so, that might up being too slow for practi

Re: Please use the wiki.

2007-03-09 Thread Michael G Schwern
David Golden wrote: > Maybe you want to post the wiki address to remind people? ;-) Doesn't Google know everything by now? Ok, http://perl-qa.yi.org/index.php/Main_Page

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-09 Thread Michael G Schwern
chromatic wrote: > On Friday 09 March 2007 14:50, Michael G Schwern wrote: > >> We can leverage any existing status system we want. HTTP status. Exit >> status. Throw them all in! Don't find TAP's existing statuses rich >> enough? Add your own extension keys! A particular status code not

The most important TAP tasks.

2007-03-09 Thread Michael G Schwern
There's a lot of noise and very little signal about TAP lately. Absolutely no real TAP extensions have been implemented. The TAP document remains in exactly the same form it was 9 months ago. http://search.cpan.org/~petdance/TAP-1.00/TAP.pm We're spinning our wheels and going nowhere. I'm going

Please use the wiki.

2007-03-09 Thread Michael G Schwern
Before I go completely off the handle, I'm going to say this once. Lately there's been a tremendous amount of discussions about the same old crap over and over again every few months. The same seemingly good but actually flawed ideas are proposed, the same arguments are raised and the same soluti

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-09 Thread chromatic
On Friday 09 March 2007 14:50, Michael G Schwern wrote: > We can leverage any existing status system we want. HTTP status. Exit > status. Throw them all in! Don't find TAP's existing statuses rich > enough? Add your own extension keys! A particular status code not make > sense for your ap

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-09 Thread Michael G Schwern
Adam Kennedy wrote: >> I propose that we prefix lines from STDERR with '! ' in the same way >> that '# ' is used for diagnostics. wstat and exit can just be >> >> wstat 256 >> exit 1 > > The problem with STDERR and exit is that we can't actually use them for > anything significant. > > Otherwise

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Michael G Schwern
Andy Armstrong wrote: > That looks pretty usable. I wonder if it might be better to package > those YAMLish annotations as a special kind of diagnostic > > # Normal diagnostic > ## got: 1 > ## expected: 0 Please read the bottom of this page. http://perl-qa.yi.org/index.php/TAP_Proposals > That

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-09 Thread Michael G Schwern
Andy Armstrong wrote: > On 8 Mar 2007, at 22:47, Eric Hacker wrote: >>> I propose that we prefix lines from STDERR with '! ' in the same way >>> that '# ' is used for diagnostics. wstat and exit can just be >>> >>> wstat 256 >>> exit 1 >> >> How about this? >> >> wstat: 256 >> exit: 1 >> >> YAML,

Re: Paying for TAP 2.0

2007-03-09 Thread Michael G Schwern
Andy Armstrong wrote: > On 8 Mar 2007, at 16:34, Nicholas Clark wrote: >>> But how do you know "23 ok" if you were never told that it ran ok? >> >> Surely one can post-process a regular TAP file to "sparse" output? >> And only do so if the TAP file is valid non-sparse output. > > Or post process

Re: Customizing Your Test Modules

2007-03-09 Thread Michael G Schwern
Ovid wrote: > @EXPORT = ( > @Test::More::EXPORT, > @Test::Differences::EXPORT, > @Test::Exception::EXPORT, > ); Don't forget @EXPORT_OK

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Andy Armstrong
On 9 Mar 2007, at 21:57, Michael G Schwern wrote: http://perl-qa.yi.org/index.php/TAP_diagnostic_syntax There's already a proposal for this which nobody's found serious fault with. Its, oh, 9 months old now. Its backwards compatible with existing TAP. It both solves the problems of associ

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Michael G Schwern
http://perl-qa.yi.org/index.php/TAP_diagnostic_syntax There's already a proposal for this which nobody's found serious fault with. Its, oh, 9 months old now. Its backwards compatible with existing TAP. It both solves the problems of associating diagnostics with a specific test and making them m

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Eric Hacker
On 3/9/07, Andy Armstrong <[EMAIL PROTECTED]> wrote: On 9 Mar 2007, at 16:15, Eric Hacker wrote: > I know. It probably isn't hard for a human to figure it out. A TAP > consumer can only really make a best guess though. It wouldn't be hard to information to the TAP to associate diagnostics with t

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Eric Hacker
On 3/9/07, Andy Armstrong <[EMAIL PROTECTED]> wrote: On 9 Mar 2007, at 13:37, Eric Hacker wrote: > Current TAP diagnostics appear after the the test results in the > standard TAP producers. D'you mean messages produced with diag() ? They appear in same order relative to the test results in the T

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Andy Armstrong
On 9 Mar 2007, at 16:15, Eric Hacker wrote: I know. It probably isn't hard for a human to figure it out. A TAP consumer can only really make a best guess though. It wouldn't be hard to information to the TAP to associate diagnostics with tests but where would that information come from? Cur

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Geoffrey Young
Ovid wrote: > If you want things to be *really* easy to run test suites in multiple > languages, do this. another option is this: http://people.apache.org/~geoff/test-more-separately.tar.gz which illustrates how to separate planning, etc in perl but use a foreign tap producing faile - somethin

Re: TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Andy Armstrong
On 9 Mar 2007, at 13:37, Eric Hacker wrote: Current TAP diagnostics appear after the the test results in the standard TAP producers. D'you mean messages produced with diag() ? They appear in same order relative to the test results in the TAP as they do in the test. Disclaimer: I often oper

TAP.ng Collecting stdout/stderr and diagnostics with Test runs

2007-03-09 Thread Eric Hacker
Current TAP diagnostics appear after the the test results in the standard TAP producers. If one is capturing application output like stdout/stderr, the relevant output would appear before the test. I am often using the test for debugging and have some of my modules use Test::More's diag instead o

Re: run C++ TAP output? (even easier)

2007-03-09 Thread Ovid
If you want things to be *really* easy to run test suites in multiple languages, do this. First, make sure that all test programs are executable. Then use this driver program: $ cat bin/run.pl #!/usr/bin/perl use strict; use warnings; my $prog = shift; unless ( -e $prog &&

Re: How would I use TAPx::Harness to run C++ TAP output?

2007-03-09 Thread Ovid
--- Julien Beasley <[EMAIL PROTECTED]> wrote: > I'm trying to get my project to move to TAP -- we have some perl test > files > and some C++ test files. Let's say I have 2 files: test.pl, and > test.exe, > the former is a perl program and the latter is a compiled program > that > outputs TAP. > >