> >3) Possibly for those cases that can't be handled by (2), use a file
> >naming convention, like axes01-xalanc.xml.
BTW, this should have been axes01-xalanc.out. Same as Shane's proposal.
-scott
David Marston/Cambridge/IBM <[EMAIL PROTECTED]> wrote on 12/02/2002
08:42:33 AM:
>
>
>
>
> Scott Boag writes:
> >It's not totally clear if these would be a complete reproduction of
> >the so-called gold files, or just the deltas (maybe I didn't read
> >carefully enough... from paragraph 2 I think you mean just the
> >deltas).
>
> Since "deltas" implies differences over time in this context, I
> don't think that's the word. Here's a more concrete description:
> 1. Most groups, like axes or boolean, do not need to be in the
> accept cluster at all.
> 2. Some groups need a few tests in accept. For example, the idkey
> group has about 63 positive tests, of which 6 give a problem
> with different versions. We can probably code around 4 and
> migrate 2 to accept.
> 3. The decision to migrate is made per test case. Typically, the
> test cannot be made into a universal conformance test in a
> reasonable way.
> 4. For those tests that go into accept, which might be 50 that
> migrate and another 20 that should be written, there would be
> multiple silver files under plan (2), or a set of tags for
> plan (1) to overwrite the gold tree as needed.
>
> >I would feel better if we:
> >1) Enumerated the differences that need to be processor dependent.
>
> That's underway.
>
> >2) See how many of these differences could be handled via better
> >comparators, probably with configuration options.
>
> I think that's being given full consideration. On the other hand,
> if we set requirements for emitting a named entity like under
> one option and the numbered entity under another, we really want to
> test that we do each of those things at the correct time.
>
> >3) Possibly for those cases that can't be handled by (2), use a file
> >naming convention, like axes01-xalanc.xml.
>
> This breaks way more automation than the other proposals! To you,
> accept-gold/axes/axes01-xalanc.xml
> and
> accept-silver-xalanc/axes/axes01.xml
> should be close enough that you can defer to others' needs. Under
> proposal (2), you will need to set a couple parameters that affect
> the smoke test. (Can we make the processor setting an environment
> variable, for people who only work on one?) You need to set those
> parameters regardless of which naming scheme applies to the silver
> files. Under proposal (1), you need to take explicit action to set
> up the proper reference outputs.
> .................David Marston
>
>
