Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
chromatic wrote: On Friday 09 March 2007 14:50, Michael G Schwern wrote: We can leverage any existing status system we want. HTTP status. Exit status. Throw them all in! Don't find TAP's existing statuses rich enough? Add your own extension keys! A particular status code not make sense for your application? Leave it out! Frequently Unasked Question: But Schwern, if we run all of our tests through Test::Harness, not only will it not report that extra diagnostic information but it doesn't run non-Perl tests very easily! Therefore we can't use TAP! Frequently Unanswered Answer: You can *run* the tests and *analyze* the results in separate steps. That's why they're separate steps. I think people get hung up on the idea that there is or has to be only one way to run-and-analyze tests in one step. That's great, put it on the wiki please.
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
Michael G Schwern wrote: Stop. Stop stop stop! Stop right there. Umm, people might not realize that I tend to be a bit over dramatic and didn't actually mean to shut down the discussion.
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
On 8 Mar 2007, at 21:55, Michael G Schwern wrote: Michael G Schwern wrote: Stop. Stop stop stop! Stop right there. Umm, people might not realize that I tend to be a bit over dramatic and didn't actually mean to shut down the discussion. He's not the Messiah. He's a very naughty boy. Oops, wrong language. Do we agree that the ability to capture STDERR (ordered correctly wrt STDOUT) and test script exit status are desirable? I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 We also spoke about a hierarchical test numbering scheme to allow groups of tests to be nested. Test::Block and Test::Group already give a taste of how that might work. I'm +1 on that. Also tags. I haven't thought through how they might be used. Anyone got any ideas about that? -- Andy Armstrong, hexten.net
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
On 8 Mar 2007, at 22:47, Eric Hacker wrote: I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 How about this? wstat: 256 exit: 1 YAML, YAML, do! Doesn't look like TAP though :) -- Andy Armstrong, hexten.net
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 The problem with STDERR and exit is that we can't actually use them for anything significant. Otherwise when dealing with TAP streams that don't have a concept of an exit code or a seperate error channel, the most common example being web testing, we're left high and dry. That's always been the main reason to intentionally ignore the exit code at least. Adam K
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
On 9 Mar 2007, at 00:28, Adam Kennedy wrote: I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 The problem with STDERR and exit is that we can't actually use them for anything significant. We currently have the option of spooling parse TAP to a file during testing for later consumption. If we store STDERR + exit in that TAP it becomes possible to recreate the test run pretty accurately. Otherwise when dealing with TAP streams that don't have a concept of an exit code or a seperate error channel, the most common example being web testing, we're left high and dry. In which case you'd just omit them, no? I didn't imagine they'd be mandatory. -- Andy Armstrong, hexten.net
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
On 3/8/07, Andy Armstrong [EMAIL PROTECTED] wrote: I propose that we prefix lines from STDERR with '! ' in the same way that '# ' is used for diagnostics. wstat and exit can just be wstat 256 exit 1 How about this? wstat: 256 exit: 1 YAML, YAML, do! ;)
Re: You cannot predict what TAP will be used for (was Re: Should TAP capture exit codes)
On 3/8/07, Adam Kennedy [EMAIL PROTECTED] wrote: Andy Armstrong wrote: Otherwise when dealing with TAP streams that don't have a concept of an exit code or a seperate error channel, the most common example being web testing, we're left high and dry. In which case you'd just omit them, no? I didn't imagine they'd be mandatory. mock disbeliefWeb testing doesn't have an error channel? /mock disbelief HTTP has lot's of channels. They are called Status Codes. :) There are a whole slew of them for errors. Way more than you'd ever need for testing, or so I'm told. Peace, Hacker
