Hi Kelvin, Robbie's main concern seems to be the ability to run the tests within a J2EE or service container and he mentions "classloader hierarchy" issues in his description. I'm guessing that the junit test runner uses a custom class loader that caused problems when used within a J2EE container. Now that we have established that the test cases themselves can be invoked using reflection it seems that users should be able to write code that calls the tests directly from their own, or from some other testing framework, without any dependency on the junit test runner. Thanks, Andy.
________________________________ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of kelvin goodson Sent: 17 April 2007 16:59 To: Andy Grove Cc: [email protected] Subject: Re: [Java SDO CTS] Junit 4.1 pattern for calling setUp when classes don't inherit from TestCase Andy, that's really helpful. I guess my problem was, and still is to some extent, knowing whether reliance upon the junit.jar is acceptable to the set of people we are trying to cater for. I think your example has made something clear to me though, which is that our prime aim would be to cater for the junit harness or anyone's home grown harness, but not necessarily another 3rd party harness. Do you think this is what the original intention of the meaning of "harness agnostic" is? Regards, Kelvin. On 17/04/07, Andy Grove <[EMAIL PROTECTED]> wrote: To better understand this myself, I just put a simple test case together using junit 4.1 with annotations and made use of the junit assertion calls e.g. public class MyTest { @Test public void testSomething() { // this test will fail assertEquals( "numbers are same", 1, 2 ); } } I then wrote a simple test harness to load the test class using reflection and invoke any methods starting with "test". public static void main(String[] args) throws Exception { Class testClass = Class.forName( "test.MyTest" ); Object testObject = testClass.newInstance(); Method method[] = testClass.getMethods(); for (int i = 0; i < method.length; i++) { if (method[i].getName().startsWith("test")) { System.out.println("Running " + method[i].getName()); try { method[i].invoke( testObject ); } catch (Throwable th) { th.printStackTrace(); } } } } This ran the above test method and caught the following exception: java.lang.AssertionError: numbers are same expected:<1> but was:<2> For me, this seems to demonstrate that using junit 4.1 style tests will allow people to call their tests from their own test harnesses without requiring junit-users to have to go to any special effort. The junit Assert.* methods simply check some criteria and then call fail() if that criteria is false. The fail() method simply throws an AssertionError. Writing the CTS tests using junit with annotations (rather than extending TestCase) does seem to provide everything required to allow users to run the tests outside of junit, albeit with a dependency on junit.jar but I don't see why that would be a problem for anyone? We could write our own versions of the assert* methods but they would do exactly the same thing as the junit implementation so this seems rather pointless. My conclusion is that we should continue writing SDO CTS tests using junit, but ensure we use the annotation pattern rather than extending TestCase. Is everyone happy with this? Thanks, Andy. -----Original Message----- From: kelvin goodson [mailto:[EMAIL PROTECTED] Sent: 17 April 2007 14:53 To: [email protected] Subject: Re: [Java SDO CTS] Junit 4.1 pattern for calling setUp when classes don't inherit from TestCase Yes, I was about to write to the list on this subject. I'd like to understand more of how the test harness agnosticism was intended, and whether its really practical. As it stands there is still junit through and through, in particular, each test method still references junit assertion calls. Even if we could do assertions from annotations (as per JML pre and post conditions), we still have all the junit imports. If that were possible, then I guess each of the test methods could be invoked by something other than the junit test harness, but they would have trouble asserting anything about the success of the method invocation, since there are no artifacts available before or after the method invocation to make assertions about. Kelvin On 17/04/07, Simon Nash <[EMAIL PROTECTED]> wrote: > > If the goal is to make the tests "harness agnostic", then I don't see > much difference between a JUnit-specific inheritance dependency and a > JUnit-specific annotation dependency. Is the annotation dependency > less troublesome for some reason? > > Simon > > kelvin goodson wrote: > > > Thanks for this Andy, I'll play with it tomorrow. > > > > Regards, Kelvin. > > > > On 16/04/07, Andy Grove < [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > wrote: > > > >> > >> > >> I believe you just need to annotate the setUp method with @Before. > >> This is described in the junit cookbook, here: > >> > >> http://junit.sourceforge.net/doc/cookbook/cookbook.htm > >> > >> I'm currently working on submitting some more XSD test cases in the > >> CTS so I'll try this method out. Hopefully I can then remove the > >> current dependency on TestCase in those tests. > >> > >> Thanks, > >> > >> Andy. > >> > >> > >> -----Original Message----- > >> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] ] On > Behalf > >> Of kelvin goodson > >> Sent: 16 April 2007 14:42 > >> To: [email protected] > >> Cc: Robbie Minshall > >> Subject: [Java SDO CTS] Junit 4.1 pattern for calling setUp when > classes > >> don't inherit from TestCase > >> > >> Hi, > >> I'm looking at doing some work in the CTS. I was looking back at > >> Robbie's attached description about how to keep the tests "harness > >> agnostic". I'm assuming that this is still a goal of the CTS > >> although > I > >> may have missed something in my catching up. In my quest to make > >> the > CTS > >> better I note that a number of the test case classes still extend > >> the junit TestCase class. > >> This is true for all test classes that have a setUp() method. One > >> that doesn't inherit from TestCase is XSDSerializationTest, and > >> adding a setUp method to this class doesn't cause junit to invoke > >> it in my eclipse environment. I'm trying to work out whether I > >> should I expect a 4.1environment to discover and execute the setUp > >> method when junit is used in this way. I seem to have Eclipse junit > >> plugins for 3.8.1 and > >> 4.1.0.1 and the preferences tab for Junit doesn't seem to offer > >> much in the way of configuration, so I can't be sure I'm using 4.1 behaviour. > >> > >> I really would like to be XSDSerializationTests to execute setUp so > that > >> we can have a fresh HelperContext per test, and I guess the easy > >> way out is to make the test class inherit from TestCase like the > >> others, but I'd prefer not to introduce the explicit dependency on > >> Junit if I can avoid it. > >> > >> Regards Kelvin. > >> > >> > >> On 07/12/06, Robbie Minshall < [EMAIL PROTECTED]> wrote: > >> > > >> > This sounds quite good. > >> > > >> > I have written some test cases with Brian Murray which I would be > >> > happy to contribute to tuscany. Identifying duplication and > >> > differences in similar tests would probably be an intersting > excercise > >> right off the bat. > >> > > >> > One decision that we spent a little time mulling over was the > >> > framework to use for our test suite. Originally we used the much > >> > loved junit harness which worked well. Different runtimes ( > >> > command line, J2EE Application Server, a Service Container ) have > >> > different > >> classloader hierarchies etc. > >> > Without many modifications to the junit code it was difficult and > >> > quite ugly testing SDO within the context of a variety of > >> > runtimes which the SDO APIs will be used. > >> > > >> > We took the approach of writing general test libraries which can > >> > then simply be called from a variety of test frameworks such as > >> > junit or a simple J2EE or SCA Application test harness. I like > >> > this approach > for > >> > >> > keeping the actual test code very simple, allowing for > >> > integration a variety of test frameworks, and providing ability > >> > to test directly within the different runtimes people care about. > >> > > >> > Any thoughts on this ? > >> > > >> > Robbie > >> > > >> > > >> > > >> > > >> > On 12/1/06, kelvin goodson < [EMAIL PROTECTED] > wrote: > >> > > > >> > > Andy, > >> > > please attach them to the JIRA for this work and one of us > >> > > can pick them up, thanks. > >> > > Best Regards, Kelvin. > >> > > > >> > > On 01/12/06, Andy Grove (Contractor) < [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > wrote: > >> > > > > >> > > > > >> > > > Hi Dan, > >> > > > > >> > > > I was previously working with Kelvin Goodson to donate some > >> > > > junit > >> > > tests > >> > > > on behalf of Rogue Wave Software. > >> > > > > >> > > > These tests are written purely to the SDO API and I have > validated > >> > > that > >> > > > the tests do run against Tuscany as well as Rogue Wave's > >> > > implementation. > >> > > > > >> > > > > >> > > > Should I send the tests to Kelvin? > >> > > > > >> > > > Thanks, > >> > > > > >> > > > Andy. > >> > > > > >> > > > -----Original Message----- > >> > > > From: Dan Murphy [mailto:[EMAIL PROTECTED] ] > >> > > > Sent: 30 November 2006 17:44 > >> > > > To: Tuscany Developers; Tuscany Users > >> > > > Subject: Proposal for a (Java) community test suite for SDO > >> > > > > >> > > > I would like to propose starting a community test suite for > >> > > > service > >> > > data > >> > > > objects (SDO CTS) implementations written in Java. Based on > >> > > > feedback from an earlier post this seems to be the first > >> > > > logical step in getting interoperable SDO implementations in > >> > > > all languages. I can see this leading to an interoperability > >> > > > test suite to check serialisation between implementations > >> > > > also works (across languages and implementations). > >> > > > > >> > > > Proposal for Community Test Suite (CTS) for SDO Develop a > >> > > > test suite to validate an SDO implementation behaves as > >> > > > expected, according to the community's understanding of the > >> > > > SDO > >> specification. > >> > > > Should > >> > > > the specification appear ambiguous or unclear then the > >> > > > community will decide what to do; it may decide to test the > >> > > > area with an agreed expected behaviour, or decide not to test this area. > >> > > > Ambiguities will be fed > >> > > back > >> > > > to > >> > > > the specification group for clarification. Although we will > >> > > > run this against Tuscany, the test suite will only test > >> > > > things that > we > >> > >> > > > think any implementation should support. > >> > > > > >> > > > The SDO CTS will enable developers to choose or switch SDO > >> > > > implementations without the concern of having to re-code a > >> > > > significant proportion of their application due to > >> > > > differences between implementations. This community test > >> > > > suite will first focus on areas identified important to > >> > > > developers of > >> > > > >> > > > SDO > >> > > > applications. SDO users feedback and involvement will be > >> > > > crucial to > >> > > the > >> > > > success of this effort. Over time this may grow to include a > large > >> > >> > > > proportion of the SDO specification, however the suite should > grow > >> > >> > > > according to the community's desire, rather than attempting > >> > > > to be a validation > >> > > or > >> > > > compliancy suite. > >> > > > > >> > > > To encourage everyone with an interest in SDO to contribute > >> > > > and use > >> > > the > >> > > > suite, I propose we : > >> > > > > >> > > > 1. Create a separate module in SVN to separate this from > >> Tuscany > >> > > > components and testcases. > >> > > > 2. Make use of a java package namespace that is not > >> attributable to > >> > > > either Tuscany or any other SDO implementation: test.sdo > >> > > > 3. Refactor some of the existing Tuscany SDO Java test > >> > > > cases > to > >> > >> > > > remove > >> > > > any Tuscany specific coding and re-package these to the > >> test.sdo > >> > > > namespace. > >> > > > 4. Accept tests from anyone who wishes to contribute them > under > >> > >> > > > normal > >> > > > Apache contribution conditions. > >> > > > > >> > > > > >> > > > SDO users involvement will be crucial to this effort, > >> > > > developers of > >> > > SDO > >> > > > implementations will benefit by contributing to and consuming > >> > > > a community test suite, rather than working on their own. > >> > > > > >> > > > Who's up for working on this with me ? > >> > > > > >> > > > If you are interested in joining this effort; have any > >> > > > concerns, comments or suggestions please append them... > >> > > > > >> > > > Thanks in advance to all those who volunteer :) Dan > >> > > > > >> > > > > ------------------------------------------------------------------ > >> > > > --- To unsubscribe, e-mail: > >> > > > [EMAIL PROTECTED] > >> > > > For additional commands, e-mail: > >> > > > [EMAIL PROTECTED] > >> > > > > >> > > > > >> > > > >> > > > >> > > >> > > >> > -- > >> > * * * Charlie * * * > >> > Check out some pics of little Charlie at > >> > http://www.flickr.com/photos/[EMAIL PROTECTED]/sets/ > >> > > >> > Check out Charlie's al crapo blog at > >> > http://robbieminshall.blogspot.com > >> > > >> > * * * Addresss * * * > >> > 1914 Overland Drive > >> > Chapel Hill > >> > NC 27517 > >> > > >> > * * * Number * * * > >> > 919-225-1553 > >> > >> ------------------------------------------------------------------- > >> -- To unsubscribe, e-mail: [EMAIL PROTECTED] > >> For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
