Re: [tapx-dev] Tracking tests over time

2007-07-12 Thread Smylers
Andy Armstrong writes: > Michael Peters wrote: > > > Why can't you just use test names? > > > > ok($is_ok, 'HTML nested bullet list, open tags (bug 5497)'); > > You certainly could - but I tend to think of them as descriptions > rather than names and therefore not entirely ideal as the primar

Re: Tracking tests over time

2007-07-12 Thread Andy Armstrong
Michael Peters wrote: > By the way. Do you have a link to show this or is it just something that > prints > to the console when you run the tests? They had an accident with the mail server that's obviously supposed to be relaying the tests and about 12 months worth spewed forth onto the list toda

Re: Tracking tests over time

2007-07-12 Thread Michael Peters
Andy Armstrong wrote: > I've just noticed that the MediaWiki team have a test harness that > tracks the life of individual assertions over time: > > * HTML nested bullet list, open tags (bug 5497) [Has never passed] > * HTML nested ordered list, open tags (bug 5497) [Has never passed] >

Re: Tracking tests over time

2007-07-12 Thread Andy Armstrong
Ian Malpass wrote: > Requiring that test numbering doesn't change seems like an awfully large > burden. It would mean you couldn't add extra values to a loop with a > test in it, for example (unless it happened to be at the end of your > test). It would be difficult to enforce, easy to forget/ignor

Re: Tracking tests over time

2007-07-12 Thread Ian Malpass
Andy Armstrong wrote: With TAP that'd require either adding a persistent identifier to tests, requiring that the test numbering doesn't change over time (new tests added at the end) or that there's some heuristic that attempts to track assertions that have been renumbered. Requiring that test

Re: [tapx-dev] Tracking tests over time

2007-07-12 Thread Andy Armstrong
Michael Peters wrote: > Why can't you just use test names? > > ok($is_ok, 'HTML nested bullet list, open tags (bug 5497)'); You certainly could - but I tend to think of them as descriptions rather than names and therefore not entirely ideal as the primary key. I don't really want to be forced n

Re: [tapx-dev] Tracking tests over time

2007-07-12 Thread Michael Peters
Andy Armstrong wrote: > I've just noticed that the MediaWiki team have a test harness that > tracks the life of individual assertions over time: > > * HTML nested bullet list, open tags (bug 5497) [Has never passed] > * HTML nested ordered list, open tags (bug 5497) [Has never passed]

Tracking tests over time

2007-07-12 Thread Andy Armstrong
I've just noticed that the MediaWiki team have a test harness that tracks the life of individual assertions over time: * HTML nested bullet list, open tags (bug 5497) [Has never passed] * HTML nested ordered list, open tags (bug 5497) [Has never passed] * Fuzz testing: image with bog