Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread Andy Armstrong
On 19 Feb 2009, at 20:01, Michael G Schwern wrote: Andy Armstrong wrote: On 18 Feb 2009, at 22:44, Michael G Schwern wrote: The thing which most takes advantage of this is TODO tests. They send their failure diagnostics to STDOUT so the user is not spammed with passing test information

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread Andy Armstrong
On 19 Feb 2009, at 21:16, Michael G Schwern wrote: This makes me think that -1 should actually be normal, from what Schwern has said, and 0 should include the failures and diagnostics and messages and whatnot (# stuff), as Andy seems to have expected in the past. But I can't really figure out

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread Ovid
- Original Message From: Ovid publiustemp-perl...@yahoo.com Remember Ovid and I going at it like Godzilla and Rodan over merging STDOUT and STDERR when TH3 was being put together? Yup. This has bitten me today at work. Badly :( All the more reason why we need TB2 to be done

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread David E. Wheeler
On Feb 20, 2009, at 3:18 AM, Andy Armstrong wrote: RENUMBER Won't that fuck up existing users of the library? The description for verbose should really be show the raw TAP stream. Patches / commits welcome - but I'm not going to have time to do anything more than review said patches /

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread Andy Armstrong
On 20 Feb 2009, at 16:52, David E. Wheeler wrote: On Feb 20, 2009, at 3:18 AM, Andy Armstrong wrote: RENUMBER Won't that fuck up existing users of the library? Yeah, I was making a BASIC joke :) The description for verbose should really be show the raw TAP stream. Patches / commits

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread David E. Wheeler
failed, which is a big help, but not any diagnostics. So I think I'll add a `diagnostics` parameter. Make sense? Thanks, David

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread Andy Armstrong
On 20 Feb 2009, at 20:25, David E. Wheeler wrote: Well, almost. I see which test failed, which is a big help, but not any diagnostics. So I think I'll add a `diagnostics` parameter. Make sense? Yeah, I think that's reasonable - although it would be nice at some point to do something

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread Michael G Schwern
Andy Armstrong wrote: On 20 Feb 2009, at 16:52, David E. Wheeler wrote: On Feb 20, 2009, at 3:18 AM, Andy Armstrong wrote: RENUMBER Won't that fuck up existing users of the library? Yeah, I was making a BASIC joke :) A very basic joke. -- 44. I am not the atheist chaplain. -- The

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread David E. Wheeler
On Feb 20, 2009, at 12:34 PM, Andy Armstrong wrote: Yeah, I think that's reasonable - although it would be nice at some point to do something about the option proliferation that seems to afflict us. That's not your fault of course :) Thanks. You you should probably subscribe to

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-20 Thread David E. Wheeler
On Feb 20, 2009, at 12:34 PM, Andy Armstrong wrote: Yeah, I think that's reasonable - although it would be nice at some point to do something about the option proliferation that seems to afflict us. That's not your fault of course :) Thanks. You you should probably subscribe to

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-19 Thread Andy Armstrong
On 18 Feb 2009, at 22:11, David E. Wheeler wrote: Of course if I use -v, it passes `verbosity = 1` to TAP::Harness, but I'd love to be able to see the failure diagnostics without having to see all of the passing test TAP output, too. Is there some way to get TAP::Harness to pass through

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-19 Thread Andy Armstrong
On 18 Feb 2009, at 22:44, Michael G Schwern wrote: The thing which most takes advantage of this is TODO tests. They send their failure diagnostics to STDOUT so the user is not spammed with passing test information. I didn't know that... The current behaviour also means that prove's

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-19 Thread David E. Wheeler
On Feb 19, 2009, at 10:07 AM, David E. Wheeler wrote: As a way of dealing with the immediate need, I'd love to see a way to just tell TAP::Harness to emit all diagnostics, whether failure diagnostics or freeform output, as you say. It should be off by default, as you pointed out

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-19 Thread Michael G Schwern
are running). -2 really quiet Suppress everything but the tests summary. -3 silent Suppress everything. This makes me think that -1 should actually be normal, from what Schwern has said, and 0 should include the failures and diagnostics and messages and whatnot (# stuff

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-19 Thread David E. Wheeler
On Feb 19, 2009, at 1:16 PM, Michael G Schwern wrote: What prove is referring to in -1 is suppressing its own messages about test failure, not TAP comments. Ah, okay. What you want is a .5 (didn't we figure out in BASIC that you don't closely space your numeric sequences?) which is show

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-19 Thread Ovid
- Original Message From: Michael G Schwern schw...@pobox.com The current behaviour also means that prove's --merge switch hides diagnostics: Remember Ovid and I going at it like Godzilla and Rodan over merging STDOUT and STDERR when TH3 was being put together? Yup. This has

Make TAP::Harness Output Failures Diagnostics?

2009-02-18 Thread David E . Wheeler
=316, 0 wallclock secs ( 0.05 usr 0.00 sys + 0.09 cusr 0.00 csys = 0.14 CPU) Result: FAIL pgTAP, however, has no access to STDERR, so diagnostics and failure information is sent to STDOUT along with everything else. This means that if I run tests in non-verbose mode, I don't see any

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-18 Thread David E . Wheeler
On Feb 18, 2009, at 2:04 PM, David E. Wheeler wrote: Of course if I use -v, it passes `verbosity = 1` to TAP::Harness, but I'd love to be able to see the failure diagnostics without having to see all of the passing test TAP output, too. Is there some way to get TAP::Harness to pass through

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-18 Thread Michael G Schwern
David E. Wheeler wrote: On Feb 18, 2009, at 2:04 PM, David E. Wheeler wrote: Of course if I use -v, it passes `verbosity = 1` to TAP::Harness, but I'd love to be able to see the failure diagnostics without having to see all of the passing test TAP output, too. Is there some way to get TAP

Re: Make TAP::Harness Output Failures Diagnostics?

2009-02-18 Thread David E. Wheeler
the axiom that passing tests should be quiet. The thing which most takes advantage of this is TODO tests. They send their failure diagnostics to STDOUT so the user is not spammed with passing test information. Fair enough. That said, a show me the comments switch for TAP::Parser would

Re: [tap] JSON TAP Diagnostics?

2008-08-22 Thread Lisa Dusseault
From a personal perspective, I'm an admirer of JSON and its clarity and simplicity, and also prefer only-one-format. It's hard to negotiate between two formats over the wire and get that right. From a standardization perspective, since JSON is already an RFC, and because IETF reviewers

Re: JSON TAP Diagnostics?

2008-08-21 Thread chromatic
On Mon, Aug 18, 2008 at 03:36:15PM +0200, Aristotle Pagaltzis wrote: * Michael Peters [EMAIL PROTECTED] [2008-08-18 15:30]: YAML does support things that JSON does not (types, embedded documents, etc) but I've been in doubt that we'd ever need those things for TAP anyway. That would be

Re: JSON TAP Diagnostics?

2008-08-21 Thread Ricardo SIGNES
* chromatic [EMAIL PROTECTED] [2008-08-20T13:59:14] Aren't these two separate concerns, human versus machine readability? The latter rarely respects ambiguity. Yes. Right now, there seem to be two pro-YAML arguments. (1) It's easier to for humans read. Sure. I will admit that. It is

Re: JSON TAP Diagnostics?

2008-08-21 Thread Ovid
a serialization language) Since the primary issue with diagnostics today is that it's not machine-readable and that's the primary reason we need it, I should think this is a slam-dunk issue. If that's not enough to convince folks, though, I have a final argument that I (hope) should put

Re: JSON TAP Diagnostics?

2008-08-21 Thread Eric Wilhelm
# from Ovid # on Thursday 21 August 2008 09:28: (2) YAML is better suited for complex serialization than JSON. 1. YAML is prettier. 2. JSON, unlike YAML, is stable. Let's not forget that the debated requirement for diagnostics is that the generators and consumers speak the same language

Re: JSON TAP Diagnostics?

2008-08-21 Thread Ricardo SIGNES
* Eric Wilhelm [EMAIL PROTECTED] [2008-08-21T12:46:59] # from Ovid 1. YAML is prettier. 2. JSON, unlike YAML, is stable. Let's not forget that the debated requirement for diagnostics is that the generators and consumers speak the same language Does it have to be just one? Now

Re: JSON TAP Diagnostics?

2008-08-21 Thread Ovid
PROTECTED] wrote: Let's not forget that the debated requirement for diagnostics is that the generators and consumers speak the same language Does it have to be just one? Now and forever? I originally argued that we should allow more than one, but I was wrong. We don't know the source of our

Re: JSON TAP Diagnostics?

2008-08-21 Thread Andy Armstrong
On 21 Aug 2008, at 17:57, Ricardo SIGNES wrote: Ovid (and I) would like it to be JSON, pending any better idea (that we agree is better). I'm in the JSON camp too. -- Andy Armstrong, Hexten

Re: TAP Diagnostics

2008-08-21 Thread Eric Wilhelm
# from Ricardo SIGNES # on Thursday 21 August 2008 09:57: Does it have to be just one?  Now and forever? Some people want the spec to say that diagnostics (not free-form additional data blocks) are always readable by any TAP consumer, which means they need a declared format.  It needs

Re: JSON TAP Diagnostics?

2008-08-21 Thread David E. Wheeler
On Aug 21, 2008, at 09:57, Ricardo SIGNES wrote: Schwern would like it to be YAML (a superset of JSON), with the phrasing consumers MUST understand JSON and SHOULD understand YAML. +1 David

Re: TAP Diagnostics

2008-08-21 Thread chromatic
* with this information? If the TAP protocol is simply a matter of reliably communicating the diagnostic info to the consumer (and associating it with a result?), does specifying the consumer's usage of the diagnostics over-reach? Let me answer the final question: yes. TAP should specify the description

Re: TAP Diagnostics

2008-08-21 Thread chromatic
On Thursday 21 August 2008 13:07:22 Eric Wilhelm wrote: Well, exactly how are we defining sack, potato, and wad here, and how does it have anything to do with what people want? Specifying a serialization format in TAP diagnostics says Here's a serialization format you can use to store

Re: TAP Diagnostics

2008-08-21 Thread David E. Wheeler
On Aug 21, 2008, at 13:58, chromatic wrote: I wonder why anyone wants a test so complex that its diagnostic requires you to serialize and deserialize objects and/or nested data structures to and from custom TAP producers and consumers, and, if you really need to do that, if you should

Re: TAP Diagnostics

2008-08-21 Thread Ovid
that the acronym has nothing to do with the reality, so they started with a huge mess o' confusion to start with. For the most part, we're just looking for is taking the information that's ALREADY in diagnostics and making it machine readable. Remember, this is mostly just a list of key/value pairs

Re: TAP Diagnostics

2008-08-21 Thread Ovid
--- On Thu, 21/8/08, chromatic [EMAIL PROTECTED] wrote: I've written a handful of TAP producers and consumers myself, as well as software which other people have used in ways I never intended. The more complexity you add to a system to reduce the complexity people have to manage to use

IETF list? (was Re: JSON TAP Diagnostics?)

2008-08-21 Thread Michael G Schwern
Ovid wrote: Folks, this really, really needs to go to the IETF list. What IETF list? -- Ahh email, my old friend. Do you know that revenge is a dish that is best served cold? And it is very cold on the Internet!

Re: [tap] TAP Diagnostics

2008-08-21 Thread Andy Armstrong
On 21 Aug 2008, at 22:41, chromatic wrote: I've written a handful of TAP producers and consumers myself, as well as software which other people have used in ways I never intended. The more complexity you add to a system to reduce the complexity people have to manage to use your system as

Re: IETF list? (was Re: JSON TAP Diagnostics?)

2008-08-21 Thread Andy Armstrong
On 21 Aug 2008, at 23:37, Michael G Schwern wrote: What IETF list? https://www.ietf.org/mailman/listinfo/tap -- Andy Armstrong, Hexten

Re: JSON TAP Diagnostics?

2008-08-21 Thread Aristotle Pagaltzis
* Eric Wilhelm [EMAIL PROTECTED] [2008-08-21 18:50]: Does it have to be just one? Now and forever? It doesn’t have to be *just* one, but it needs to be *at least* one, and specifically at least one that *everyone* supports, so that you can count on having a way to make an emitter and consumer

Re: TAP Diagnostics

2008-08-21 Thread Aristotle Pagaltzis
. someone who uses TAP in testing a web app will probably have use for an HTTP response status key and also for a hash of response headers. The killer app example for diagnostics that I always come back to is statical analysis over an archive of your TAP streams over time. It makes sense to preserve

Re: JSON TAP Diagnostics?

2008-08-19 Thread Ricardo SIGNES
* Ovid [EMAIL PROTECTED] [2008-08-18T06:50:00] JSON is fairly well implemented and new implementations are trivial. This is not true for YAML. Trying to define a minimum standard of YAML for extended TAP is a quagmire. With JSON, we can punt and just point to a fairly well-established JSON

Re: JSON TAP Diagnostics?

2008-08-19 Thread Ricardo SIGNES
* David Golden [EMAIL PROTECTED] [2008-08-18T09:27:57] What's the latest consensus on the best pure-perl JSON module? And ditto for JSON via XS? JSON and JSON::XS, most likely. Certainly JSON::XS. -- rjbs

Re: JSON TAP Diagnostics?

2008-08-19 Thread Ricardo SIGNES
* Ovid [EMAIL PROTECTED] [2008-08-18T11:17:25] Oh, definitely agreed. I cannot assert that non-Perl implementations of JSON are any better, but JSON is simple enough that I'm pretty damned sure they are. However, YAML is so problematic that I *CAN* state that non-Perl versions are often as

Re: JSON TAP Diagnostics?

2008-08-19 Thread Ricardo SIGNES
* Michael G Schwern [EMAIL PROTECTED] [2008-08-18T12:26:54] YAML types can be little more than local tags which only have meaning to that particular document. name: !customer Evil Business Guy Made Of Butter Yeah, that's neat and everything, but there aren't any Perl implementations that

JSON TAP Diagnostics?

2008-08-18 Thread Ovid
Hi all, One issue which arose at YAPC::EU was the problem with machine-readable TAP diagnostics. Since they're not yet implemented, we can change them. The problem we wound up with was that we have two things to specify: core TAP and extended TAP. Core TAP is simple (well, uh, mostly

Re: JSON TAP Diagnostics?

2008-08-18 Thread Paul Johnson
On Mon, Aug 18, 2008 at 03:50:00AM -0700, Ovid wrote: Thoughts? Agreement. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net

Re: JSON TAP Diagnostics?

2008-08-18 Thread David Golden
On Mon, Aug 18, 2008 at 6:50 AM, Ovid [EMAIL PROTECTED] wrote: Thoughts? Likewise, agreed. What's the latest consensus on the best pure-perl JSON module? And ditto for JSON via XS? David

Re: JSON TAP Diagnostics?

2008-08-18 Thread Michael Peters
Ovid wrote: Thoughts? ++ There are some other things to work out though, like how do we decide that a JSON doc has begun (YAML has the nice --- thing), etc. YAML does support things that JSON does not (types, embedded documents, etc) but I've been in doubt that we'd ever need those things

Re: JSON TAP Diagnostics?

2008-08-18 Thread Aristotle Pagaltzis
* Ovid [EMAIL PROTECTED] [2008-08-18 12:55]: First of all, read that thoroughly. That should take you a few days. I know, right? When I mention that I always that the YAML spec is much more complex than the XML spec and the XML Namespaces spec put together. (Despite the XML and Namespaces specs

Re: JSON TAP Diagnostics?

2008-08-18 Thread Aristotle Pagaltzis
* Michael Peters [EMAIL PROTECTED] [2008-08-18 15:30]: YAML does support things that JSON does not (types, embedded documents, etc) but I've been in doubt that we'd ever need those things for TAP anyway. That would be useful if any of the YAML producers were capable of serialising tricky data

Re: JSON TAP Diagnostics?

2008-08-18 Thread David Golden
On Mon, Aug 18, 2008 at 9:31 AM, Aristotle Pagaltzis [EMAIL PROTECTED] wrote: Are we still considering human readability a goal for TAP? That For basic TAP, I think it should be a goal. For extended TAP, I think the goal is more about machine-readable output so that diagnostics can be collected

Re: JSON TAP Diagnostics?

2008-08-18 Thread Ovid
--- On Mon, 18/8/08, Michael Peters [EMAIL PROTECTED] wrote: ++ There are some other things to work out though, like how do we decide that a JSON doc has begun (YAML has the nice --- thing), etc. YAML does support things that JSON does not (types, embedded documents, etc) but I've been

Re: JSON TAP Diagnostics?

2008-08-18 Thread Ovid
--- On Mon, 18/8/08, Aristotle Pagaltzis [EMAIL PROTECTED] wrote: And for those who would argue for YAML::Tiny as our spec, it already has limitations that hit us at the BBC. In what way, and why would that be relevant to TAP? Would JSON not have those same limitations? I was about to

Re: JSON TAP Diagnostics?

2008-08-18 Thread Dominique Quatravaux
On Mon, Aug 18, 2008 at 4:12 PM, Ovid [EMAIL PROTECTED] wrote: YAML::Tiny seems to do everything that JSON does, so I must now eat crow (nom, nom, nom, gag). Well, hope you found it tasty, but JSON is still a reasonable alternative to consider if non-Perl implementations are better than

Re: JSON TAP Diagnostics?

2008-08-18 Thread Michael G Schwern
Ovid wrote: One issue which arose at YAPC::EU was the problem with machine-readable TAP diagnostics. Since they're not yet implemented, we can change them. The problem we wound up with was that we have two things to specify: core TAP and extended TAP. Core TAP is simple (well, uh, mostly

Re: jpeg TAP Diagnostics?

2008-08-18 Thread Eric Wilhelm
# from Ovid # on Monday 18 August 2008 03:50: JSON is fairly well implemented and new implementations are trivial.  This is not true for YAML.  Trying to define a minimum standard of YAML for extended TAP is a quagmire.  With JSON, we can punt and just point to a fairly well-established JSON

Re: JSON TAP Diagnostics?

2008-08-18 Thread David E. Wheeler
On Aug 18, 2008, at 07:03, Ovid wrote: Those are certainly important issues, but JSON will make some of them trivial. The YAML types, embedded documents and the one format to rule them all concept is precisely what makes it unsuitable for TAP. That's a damned shame because if there was

Re: JSON TAP Diagnostics?

2008-08-18 Thread Michael G Schwern
Ovid wrote: --- On Mon, 18/8/08, Michael G Schwern [EMAIL PROTECTED] wrote: YAML has several important things that JSON is lacking. Without going into detail, I'll just say that you raise some valid points. I agree with some and not with others, but we should defer this discussion

Re: JSON TAP Diagnostics?

2008-08-18 Thread Ovid
--- On Tue, 19/8/08, Michael G Schwern [EMAIL PROTECTED] wrote: I think we should start the process by specifying TAP version 12 aka core TAP. The stuff we all agree on and is in wide use. Extension discussion should be orthogonal so as not to stall the standardization process. That's the

Re: JSON TAP Diagnostics?

2008-08-18 Thread Michael G Schwern
Ovid wrote: --- On Tue, 19/8/08, Michael G Schwern [EMAIL PROTECTED] wrote: I think we should start the process by specifying TAP version 12 aka core TAP. The stuff we all agree on and is in wide use. Extension discussion should be orthogonal so as not to stall the standardization

Re: JSON TAP Diagnostics?

2008-08-18 Thread Aristotle Pagaltzis
* Andy Armstrong [EMAIL PROTECTED] [2008-08-18 17:35]: I prefer JSON aesthetically apart from any technical considerations. I don't actually find YAML all that readable. To programmers' eyes JSON looks more like code - presumably because it is :) YAML requires less quoting and backslashing.

Re: JSON TAP Diagnostics?

2008-08-18 Thread Eric Wilhelm
# from Michael G Schwern # on Monday 18 August 2008 16:55:  The stuff we all agree on and is in wide use.  Extension discussion should be orthogonal so as not to stall the standardization process. That's the stance I took in Copenhagen last week.  I was unanimously voted down. Seeing as

User-Supplied Diagnostics Are *Important*

2008-04-18 Thread Ovid
--- chromatic [EMAIL PROTECTED] wrote: (Given that even unstructured diagnostics have never actually appeared in TAP documents before, my guess is No, everyone's arguing out of ignorance, This is not true at all. If you rephrase this slightly: Given that even unstructured diagnostics

Re: TAP YAML Diagnostics

2008-04-07 Thread Aristotle Pagaltzis
* Eric Wilhelm [EMAIL PROTECTED] [2008-04-06 18:45]: (perhaps allow non-first numbers too?) /^[a-z][a-z0-9_]*$/ ? ++ Preceding the key with an underscore or some kind of squiggle should be enough to make it safe. Having ToWrite AllPrivateKeys InStudlyCaps

Re: TAP YAML Diagnostics

2008-04-07 Thread Shlomi Fish
On Monday 07 April 2008, Aristotle Pagaltzis wrote: * Eric Wilhelm [EMAIL PROTECTED] [2008-04-06 18:45]: (perhaps allow non-first numbers too?) /^[a-z][a-z0-9_]*$/ ? ++ Preceding the key with an underscore or some kind of squiggle should be enough to make it safe. Having ToWrite

TAP YAML Diagnostics

2008-04-06 Thread Ovid
Hi all, Mostly an FYI: I'm in Oslo right now and we're working out (and implementing) new features in TAP. One of them is the new YAML diagnostics. They look like this (these aren't all we support): not ok 2 We ain't got foobar! --- file: t/something.t line: 14 tags

Re: TAP YAML Diagnostics

2008-04-06 Thread Michael Peters
Eric Wilhelm wrote: # from Ovid # on Sunday 06 April 2008 08:41: file: t/something.t If the filename is the same on every test in the stream, why is this being attached to the test and not the stream? It's the default (when attached to a stream), but can be overridden by each individual

Re: TAP YAML Diagnostics

2008-04-06 Thread Eric Wilhelm
# from Nicholas Clark # on Sunday 06 April 2008 09:33: reserved (and it's a parse error to use an unknown lower-case key). Are there any strange Unicode issues where we might get confused about what is upper and lower case?) I believe that there are code points which would be considered word

Re: TAP YAML Diagnostics

2008-04-06 Thread Ovid
--- Eric Wilhelm [EMAIL PROTECTED] wrote: # from Ovid # on Sunday 06 April 2008 08:41: file: t/something.t If the filename is the same on every test in the stream, why is this being attached to the test and not the stream? Or, the filename *could* change between tests in a stream?

Re: TAP YAML Diagnostics

2008-04-06 Thread Nicholas Clark
On Sun, Apr 06, 2008 at 08:41:11AM -0700, Ovid wrote: Currently you can shove anything in there you want, but you must use upper-case keys for your personal use and all lower-case keys are reserved (and it's a parse error to use an unknown lower-case key). Are there any strange Unicode

Re: TAP YAML Diagnostics

2008-04-06 Thread Eric Wilhelm
# from Ovid # on Sunday 06 April 2008 08:41: file: t/something.t If the filename is the same on every test in the stream, why is this being attached to the test and not the stream? Or, the filename *could* change between tests in a stream? Does it? Should it? --Eric -- A counterintuitive

Re: Diagnostics

2008-02-13 Thread Ovid
--- Michael G Schwern [EMAIL PROTECTED] wrote: David Landgren wrote: I wish you'd s/Got/Actual/ or Received. Got must die. Why's that? Got is the simple past of to get. Informally it's often used as the past tense of to have -- as in Test::More diagnostics. While it's not exactly on par

Re: Diagnostics

2008-02-13 Thread Philippe Bruhat (BooK)
On Wed, Feb 13, 2008 at 12:34:44AM -0800, Ovid wrote: As for received/expected, I really like it because it reads very nicely (the verbs aren't quite the opposite of each other, but I still like 'em). Oh, and they line up nicely too, in case you want to do some ascii formatting:

Re: Diagnostics

2008-02-13 Thread Matisse Enzer
On Feb 12, 2008, at 1:44 PM, David Landgren wrote: I wish you'd s/Got/Actual/ or Received. Got must die. I like Got/Expected - I understood them right away when i first encountered them, and they seem like plain english to me: I only got 5 dollars when I expected 10! Expected an array,

Smoking private dists? was Re: Diagnostics was Re: Smoking private dists?

2008-02-13 Thread Paul Johnson
On Wed, Feb 13, 2008 at 07:51:24AM -0800, Matisse Enzer wrote: And of course, in the Java world, it is Expected/Got (JUnit wants the expected value as the first argument to assertions.) Which is, of course, the right way round, however you spell Got. (Hint: it's Received.) Haven't we had

Re: Smoking private dists? was Re: Diagnostics was Re: Smoking private dists?

2008-02-13 Thread Michael Peters
Paul Johnson wrote: But did we ever get an answer to the original question? I have a similar requirement and the answers currently seem to be 1. Buildbot (which seems to be overkill in my situation) 2. Smolder and some scripts Is that a fair summary? I'd add a #3 which is

Re: Diagnostics

2008-02-13 Thread Chris Dolan
On Feb 13, 2008, at 9:51 AM, Matisse Enzer wrote: And of course, in the Java world, it is Expected/Got (JUnit wants the expected value as the first argument to assertions.) ;-) TestNG (testng.org) reversed the JUnit order, so Java unit test assert arguments are now the same order as

Re: Diagnostics

2008-02-12 Thread chromatic
On Tuesday 12 February 2008 13:44:06 David Landgren wrote: I wish you'd s/Got/Actual/ or Received. Got must die. The Received/Expected pair gets a rare +2 from me. -- c

Re: Diagnostics

2008-02-12 Thread David Landgren
Ovid wrote: --- Philippe Bruhat (BooK) [EMAIL PROTECTED] wrote: The biggest trouble I had was for diagnostics. I ended up considering that diagnostics output after a test result belong to the test result (as a comment to it), and that diagnostics appearing before the first test result

Re: Diagnostics

2008-02-12 Thread Michael G Schwern
David Landgren wrote: I wish you'd s/Got/Actual/ or Received. Got must die. Why's that? -- Hating the web since 1994.

Diagnostics

2008-02-07 Thread Ovid
--- Philippe Bruhat (BooK) [EMAIL PROTECTED] wrote: The biggest trouble I had was for diagnostics. I ended up considering that diagnostics output after a test result belong to the test result (as a comment to it), and that diagnostics appearing before the first test result are global

Re: Diagnostics

2008-02-07 Thread Andy Armstrong
On 7 Feb 2008, at 10:29, Ovid wrote: This is, in my opinion, the single most important thing that we need to resolve in Perl's testing world: moving people to TAP version 13 along with full diagnostics in TAP that software can actually read. +1 -- Andy Armstrong, Hexten

Re: Diagnostics

2008-02-07 Thread Philippe Bruhat (BooK)
On Thu, Feb 07, 2008 at 02:29:18AM -0800, Ovid wrote: --- Philippe Bruhat (BooK) [EMAIL PROTECTED] wrote: The biggest trouble I had was for diagnostics. I ended up considering that diagnostics output after a test result belong to the test result (as a comment to it), and that diagnostics

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Geoffrey Young [EMAIL PROTECTED] wrote: There are two usual rebuttals. the third being just add it and let me decide :) Right in one! I'll go fix that diagnostic thing now. Unfortunately, I think I'll have to violate encapsulation :( As a side note: after reading everything

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
Ovid wrote: I'll go fix that diagnostic thing now. Unfortunately, I think I'll have to violate encapsulation :( If you know how to fix it let me know, because other than enumerating each testing module you might use and lex-wrapping all the functions they export, I'm not sure how to do it.

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
The whole idea of halting on first failure was introduced to me by some XUnit folks. Their rationale was not to avoid spewing output, they had no such problem since it's all done via a GUI, but that once one failure has happened the failing code might hose the environment and all following

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Michael G Schwern [EMAIL PROTECTED] wrote: The whole idea of halting on first failure was introduced to me by some XUnit folks ... As any field scientist knows, there's no such thing as uncontaminated data. As any tester knows, a one size fits all suit often doesn't fit. Let people

Preserving diagnostics when dieing on test failure

2008-01-12 Thread Ovid
. # got: '2' # expected: '3' Test failed. Halting at MyTestMore.pm line 54. 1..3 # Looks like you failed 1 test of 3. # Looks like your test died just after 3. So we can preserve diagnostics, but we need help in cleaning up those damned line numbers. Hook::LexWrap didn't

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
Ovid wrote: --- Michael G Schwern [EMAIL PROTECTED] wrote: The whole idea of halting on first failure was introduced to me by some XUnit folks ... As any field scientist knows, there's no such thing as uncontaminated data. As any tester knows, a one size fits all suit often doesn't fit.

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Michael G Schwern
Aristotle Pagaltzis wrote: * Michael G Schwern [EMAIL PROTECTED] [2008-01-12 12:00]: Ovid wrote: I'll go fix that diagnostic thing now. Unfortunately, I think I'll have to violate encapsulation :( If you know how to fix it let me know, because other than enumerating each testing module you

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-12 Thread Ovid
--- Michael G Schwern [EMAIL PROTECTED] wrote: Set a flag that T::B should quit when the next test result is about to be recorded? I guess it works, but it leaves you dead halfway through another test function which is weird. The code I sent is conceptually similar, but since a failure

Re: Preserving diagnostics when dieing on test failure

2008-01-12 Thread Michael G Schwern
Ovid wrote: So we can preserve diagnostics, but we need help in cleaning up those damned line numbers. Hook::LexWrap didn't have the magic I thought it would. ok() is now inside a wrapper so you're one level further down then it thinks. Just add one to $Level and then take it back off again

Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-11 Thread Michael G Schwern
; is 23, 42' not ok 1 # Failed test at /usr/local/perl/5.8.8/lib/Test/More.pm line 329. Test failed. Halting at OurMore.pm line 44. 1..1 Dude, where's my diagnostics? In Test::Builder, the diagnostics are printed *after* the test fails. So dying on ok() will kill those very important diagnostics

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-11 Thread demerphq
On 12/01/2008, Michael G Schwern [EMAIL PROTECTED] wrote: -- 184. When operating a military vehicle I may *not* attempt something I saw in a cartoon. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/?page_id=3 That was one of the

Re: Dude, where's my diagnostics? (Re: Halting on first test failure)

2008-01-11 Thread Geoffrey Young
There are two usual rebuttals. the third being just add it and let me decide :) --Geoff

Re: TAP datetime (was Re: Current state of TAP::Diagnostics)

2007-09-07 Thread Ovid
--- A. Pagaltzis [EMAIL PROTECTED] wrote: That reasoning leads me to conclude that we can reduce the cost of this SHOULD over time by making it no formal requirement at all for app data and a MUST for TAP metainformation right off the bat. And if we go down this road, we can halt this navel

Re: TAP datetime (was Re: Current state of TAP::Diagnostics)

2007-09-07 Thread A. Pagaltzis
* Michael G Schwern [EMAIL PROTECTED] [2007-09-07 06:40]: We're not prohibiting other formats, just establishing a baseline. I think we specifically want to prohibit other formats in TAP metainformation. For app data, we could advise authors to use the same format, but I don’t think It's

Re: TAP datetime (was Re: Current state of TAP::Diagnostics)

2007-09-07 Thread Michael G Schwern
leads me to conclude that we can reduce the cost of this SHOULD over time by making it no formal requirement at all for app data and a MUST for TAP metainformation right off the bat. I'm down with changing it to MUST for TAP meta and MAY for diagnostics. I completely missed MAY in RFC 2119

Re: TAP datetime (was Re: Current state of TAP::Diagnostics)

2007-09-07 Thread Eric Wilhelm
# from Michael G Schwern # on Friday 07 September 2007 05:38 am: I'm down with changing it to MUST for TAP meta and MAY for diagnostics. +1 -- If the above message is encrypted and you have lost your pgp key, please send a self-addressed, stamped lead box to the address below

Re: TAP datetime (was Re: Current state of TAP::Diagnostics)

2007-09-07 Thread A. Pagaltzis
* Michael G Schwern [EMAIL PROTECTED] [2007-09-07 14:40]: I'm down with changing it to MUST for TAP meta and MAY for diagnostics. Works for me. Regards, -- Aristotle Pagaltzis // http://plasmasturm.org/

  1   2   >