On Tue, Aug 11, 2009 at 2:26 PM, Simon Laws<[email protected]> wrote: > IMO suggesting that a test passes just because it raises an exception, > without being specific about what that exception might be, is at best > missleading. We can't say that our code base accurately reflects the > content of the SCA specifications based such a loose assertion. >
Not all the tests are expecting an exception some are expecting a successful invocation, of the ones that are presently failing we can fix the ones that aren't making a successful invocation and we can fix the ones where the test is expecting an exception but tuscany is not failing too. I don't think we are trying to say Tuscany "accurately reflects the content of the SCA specifications" all we're trying to say Tuscany passes the OASIS SCA conformance tests, and being able to say that would be of big benifit, we'd be able to blog and brag and get some much needed publicity and, start encouraging people to start using the 2.x code. There will be bugs discovered after that which aren't picked up by the conformance tests that we'll need to fix but thats just business as usual isn't it? > Remember that the conformance tests are just that. I would expect > there to be many scenarios that the otests themselves don't cover. We > need extra tests in Tuscany for the cases that the otests don't cover. > The least we can do is wring as much value as we can out of the set of > tests that the otest suite represents. Otherwise we just have to > re-write the test ourselves. > I'm not saying we should never look at tests which pass because of the wrong failure, it just makes a lot more sense to me to first look at the ones that don't even pass. That way we get those benefits of saying we're conformant sooner and also its much easier to measure fixing something thats broken than it is to measure thinking about if something is working for the wrong reasons so by fixing the broken ones first we should make faster progress and be better able to measure that progress. ...ant
