On 02/07/06, Adam Kennedy <[EMAIL PROTECTED]> wrote:
Fergal Daly wrote: > On 02/07/06, Adam Kennedy <[EMAIL PROTECTED]> wrote: >> > There's no way to declare a top-level plan. That is, I can't say how >> > many groups of tests I'm going to run so there's effectively no plan, >> >> One point that Andy was extremely insistant on, and I think Schwern and >> I agree, is that the main plan is ALWAYS the total number of tests for >> the entire test script. >> >> In that case, groups form an additional set of checks, but do NOT alter >> the plan for the entire script. > > That contradicts #2 "I don't want to have to count up the total number > of tests in my > file but I do want the protection of the plan." but looking again, I > see that the example does include an overall plan that does count up > the total. There's four cases here. 1. Plan, no groups 2. No plan, no groups As is now 3. Plan, with groups The plan still is for the ENTIRE test script, but in addition within that total you can define groups to add extra protection or grouping information for diagnostics. 4. No plan, with groups In THIS case, the total of the script does not matter or may not be known, but you want protection of a sort of "miniplan" for specific sections. This does bring up a gap in the spec though (or I'm not remembering right). If you have the following, how do you tell where the end of the group is. Currently I think it would be implicit and unclear? (noplan) ok 1 ok 2 ..2 ok 3 ok 4 ok 5 ok 6
That seems like a problem too but the one I'm trying to get at is 4 no plan, with groups If your script exits prematurely after one of the groups, the harness will not notice because everything looks just fine. The solution to this is not to use "plan, with groups" because then you have to count all the tests individually which goes aginst objective #2, F
