Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-09 Thread Michael G Schwern
chromatic wrote: On Friday 09 March 2007 14:50, Michael G Schwern wrote: We can leverage any existing status system we want. HTTP status. Exit status. Throw them all in! Don't find TAP's existing statuses rich enough? Add your own extension keys! A particular status code not make

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Michael G Schwern
Michael G Schwern wrote: Stop. Stop stop stop! Stop right there. Umm, people might not realize that I tend to be a bit over dramatic and didn't actually mean to shut down the discussion.

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Andy Armstrong
On 8 Mar 2007, at 21:55, Michael G Schwern wrote: Michael G Schwern wrote: Stop. Stop stop stop! Stop right there. Umm, people might not realize that I tend to be a bit over dramatic and didn't actually mean to shut down the discussion. He's not the Messiah. He's a very naughty boy.

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Andy Armstrong
On 8 Mar 2007, at 22:47, Eric Hacker wrote: I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 How about this? wstat: 256 exit: 1 YAML, YAML, do! Doesn't look like TAP though :) -- Andy

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Adam Kennedy
I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 The problem with STDERR and exit is that we can't actually use them for anything significant. Otherwise when dealing with TAP streams that

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Andy Armstrong
On 9 Mar 2007, at 00:28, Adam Kennedy wrote: I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 The problem with STDERR and exit is that we can't actually use them for anything significant.

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Eric Hacker
On 3/8/07, Andy Armstrong [EMAIL PROTECTED] wrote: I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 How about this? wstat: 256 exit: 1 YAML, YAML, do! ;)

Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)

2007-03-08 Thread Eric Hacker
On 3/8/07, Adam Kennedy [EMAIL PROTECTED] wrote: Andy Armstrong wrote: Otherwise when dealing with TAP streams that don't have a concept of an exit code or a seperate error channel, the most common example being web testing, we're left high and dry. In which case you'd just omit them, no? I

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 13:01, Eric Hacker wrote: Exit code or Status code? Well let's generalise it and discuss the specifics: any useful information that's available when the test script terminates The RFC Status codes might not be a perfect fit for test status, but like the SIP protocol,

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 13:48, Eric Hacker wrote: I think it was Ovid who recently called it the Test Anything Protocol. If really what is desired, then some additional complexity is required. Sure - I'm completely in favour of being able to test anything and capture everything that might be

Re: Should TAP capture exit codes

2007-03-07 Thread Eric Hacker
On 3/7/07, Andy Armstrong [EMAIL PROTECTED] wrote: Ovid's post about TAP::Tests has reminded me: would it be useful to have a TAP statement that conveys the exit code of a test script? At the moment in a hypothetical situation where there's some distance between the harness and the test script -

Re: Should TAP capture exit codes

2007-03-07 Thread Eric Hacker
On 3/7/07, Andy Armstrong [EMAIL PROTECTED] wrote: On 7 Mar 2007, at 13:01, Eric Hacker wrote: Exit code or Status code? Well let's generalise it and discuss the specifics: any useful information that's available when the test script terminates Ok The RFC Status codes might not be a

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 16:26, Eric Hacker wrote: [snip] The first digit can be a grouping by success/failure. Yes, I see where you're going with this :) So then if I'm not too far off base with the above, then to use something different than HTTP::Status type codes would be reinventing. 1xx Info

Re: Should TAP capture exit codes

2007-03-07 Thread Michael G Schwern
Andy Armstrong wrote: On 7 Mar 2007, at 16:26, Eric Hacker wrote: [snip] The first digit can be a grouping by success/failure. Yes, I see where you're going with this :) So then if I'm not too far off base with the above, then to use something different than HTTP::Status type codes would be

Re: Should TAP capture exit codes

2007-03-07 Thread demerphq
On 3/7/07, Michael G Schwern [EMAIL PROTECTED] wrote: Andy Armstrong wrote: On 7 Mar 2007, at 16:26, Eric Hacker wrote: [snip] The first digit can be a grouping by success/failure. Yes, I see where you're going with this :) So then if I'm not too far off base with the above, then to use

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 18:18, demerphq wrote: If you want to say Temporary Redirect don't say 307 say Temporary Redirect! If you want to put lots of information into one value, like categorization, use a hash! { type = Redirect, permanent = 0 } Numeric response codes have the advantage that

Re: Should TAP capture exit codes

2007-03-07 Thread demerphq
On 3/7/07, Andy Armstrong [EMAIL PROTECTED] wrote: On 7 Mar 2007, at 18:18, demerphq wrote: If you want to say Temporary Redirect don't say 307 say Temporary Redirect! If you want to put lots of information into one value, like categorization, use a hash! { type = Redirect, permanent = 0

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 18:59, demerphq wrote: Neither to me to be a very convincing reason to redesign something as well thought out as the HTTP response code schema. With it you have a well documented, well designed language agnostic response structure. It seems to me youd have to work hard to come

Re: Should TAP capture exit codes

2007-03-07 Thread demerphq
On 3/7/07, Andy Armstrong [EMAIL PROTECTED] wrote: On 7 Mar 2007, at 18:59, demerphq wrote: Neither to me to be a very convincing reason to redesign something as well thought out as the HTTP response code schema. With it you have a well documented, well designed language agnostic response

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 19:21, demerphq wrote: I guess it comes down to whether you can anticipate the possibility that you will need new codes, and having a framework to put them into. OK, well we can talk about that now and at least get an idea of what kind of future we're proofing ourselves

RE: Should TAP capture exit codes

2007-03-07 Thread Gary Hawkins
OK, well we can talk about that now and at least get an idea of what  kind of future we're proofing ourselves against. What do people  envisage that we might want / be able to capture about a test run? 1. Access to both STDERR and STDOUT, in proper order ('prove' jumbles them), capturable

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 19:39, Eric Hacker wrote: Now, I know you are thinking about exit status on test scripts and I'm thinking individual tests (of which running another test script might be an instance), but in the distributed functional testing space, one really can't rely on independent test

Re: Should TAP capture exit codes

2007-03-07 Thread Eric Hacker
On 3/7/07, Andy Armstrong [EMAIL PROTECTED] wrote: On 7 Mar 2007, at 13:48, Eric Hacker wrote: I think it was Ovid who recently called it the Test Anything Protocol. If really what is desired, then some additional complexity is required. Sure - I'm completely in favour of being able to test

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 19:50, Gary Hawkins wrote: 1. Access to both STDERR and STDOUT, in proper order ('prove' jumbles them), capturable into a variable (T::B snatches away STDERR) TAP's a line oriented protocol so I imagine the best we can do is to keep /lines/ from STDERR and STDOUT in the

RE: Should TAP capture exit codes

2007-03-07 Thread Gary Hawkins
2. Option to inject a clearcut delimiter between tests Distinct from, say, outputting a diagnostic between groups of tests? By 'tests' I'm thinking 'file' with its subtests, so-to-speak, so yes, anything that is clearly delineatable (LOL) programmatically where one 'file' output stops and

Re: Should TAP capture exit codes

2007-03-07 Thread Andy Armstrong
On 7 Mar 2007, at 20:35, Gary Hawkins wrote: 2. Option to inject a clearcut delimiter between tests Distinct from, say, outputting a diagnostic between groups of tests? By 'tests' I'm thinking 'file' with its subtests, so-to-speak, so yes, anything that is clearly delineatable (LOL)

Re: Should TAP capture exit codes

2007-03-07 Thread Eric Hacker
On 3/7/07, Andy Armstrong [EMAIL PROTECTED] wrote: It sounds as if you're doing monitoring rather than testing though. Although they're related the requirements are quite different. Poor explaining on my part then. Monitoring has similar needs, but us usually much more shallow. Consider a web

Re: Should TAP capture exit codes

2007-03-07 Thread Michael G Schwern
demerphq wrote: If you want to say Temporary Redirect don't say 307 say Temporary Redirect! If you want to put lots of information into one value, like categorization, use a hash! { type = Redirect, permanent = 0 } Numeric response codes have the advantage that they are language agnostic.