The purpose of this exercise is to make sure that Tuscany behaves as the
specification says under various situations. We can only build the
confidence that Tuscany is conforming if we have fairly precise assertions
for these items. Associating the error messages with the conformance items
(at least the ids) definitely help us to verify the test results.
Thanks,
Raymond
--------------------------------------------------
From: "Simon Laws" <[email protected]>
Sent: Tuesday, August 11, 2009 6:26 AM
To: <[email protected]>; <[email protected]>
Subject: Re: [2.x] otest - Tuscany specific error checking
IMO suggesting that a test passes just because it raises an exception,
without being specific about what that exception might be, is at best
missleading. We can't say that our code base accurately reflects the
content of the SCA specifications based such a loose assertion.
Remember that the conformance tests are just that. I would expect
there to be many scenarios that the otests themselves don't cover. We
need extra tests in Tuscany for the cases that the otests don't cover.
The least we can do is wring as much value as we can out of the set of
tests that the otest suite represents. Otherwise we just have to
re-write the test ourselves.
Simon
