Hi Tom, a few quick thoughts:
- I would expect that I could use multiple test frameworks (e.g. JUnit and TestNG), and Gradle would still run all of my tests. Will this be supported? - Javassist and Mina sound like very heavyweight dependencies "just" to run tests. Ideally, 20KB of ASM would be enough. - If you are looking for JUnit annotations on a class, the logic should be "classHasRunWithAnnotation() || anyMethodHasTestAnnotation()". (I'm mentioning this because most IDEs initially got this wrong and only looked for @Test.) - I'm not sure if skipping classes with "$" in their name is completely safe, even though people probably won't write unit tests as static inner classes. Is this done for performance reasons? My experience with ASM is that scanning for annotations is incredibly fast. - I've done auto-detection of JUnit tests with ASM for Spock. You can have a look at it here: http://code.google.com/p/spock/source/browse/#svn/trunk/spock-core/src/main/java/org/spockframework/buildsupport. I chose the same ASM version that Groovy uses, even though it's not the latest one. Cheers, Peter Tom Eyckmans wrote: > > Hi guys, > > I'm working on the native test framework execution stuff, I just want to > let > you guys know how I'm trying to implement this so you can contribute > ideas/ > better ways of doing things. > > Global overview: > - recurse through compiled test classes, filter out class files that > contain > a $ sign => testClassFilesQueue ( BlockingQueue ) > - testClassFilesQueue => scan for test classes (first found test class > determines test framework) => testInfosQueue ( BlockingQueue ) > - testServer / testClient > - server controls the forked test process: > - test process requests work (which test to execute next > (dequeued > from testInfosQueue -> work/memory throthle), no more work -> terminate) > - client sends progress events to the server => progressEventQueue => > these events get duplicated to BuildListener notifications and Test Report > output, which both need to get executed serially. > > Smarter test detection. This is done on the compiled test classes. Only > test > classes that don't contain a $ sign are queued on a blocking queue for > processing. > > Queued classes are checked by using javassist to get to the information ( > annotations, parent classes ). The logic required to detect test classes > is > located in implementations of the TestFramework interface. I've added some > base logic to scan for annotations in the AbstractTestFramework class; the > logic in here checks for a number of annotations on methods. As Adam > mentioned to me the test methods can be inherited and so if no methods are > found on the current class it will try and scan upwards on the inheritance > tree and stop when the parent class is java.lang.Object. Scanning up the > inheritance tree is currently only done when the super test class is > available in the test-classes directory (scanning upwards should be done > eventually in the complete classpath). > > When a class is identified as an actual test class, an implementation of > TestInfo is queued testInfosQueue, this object is a reference to the test > class and the type of implementation determines the way the test class > needs > to be processed. This is needed to support executing JUnit TestCases and > TestSuites or TestNG Test classes and xml suite files. > > When the first test class is identified the TestFramework that identified > the test class is used. This causes only that framework to be used for > test > detection from that point on. > > The only way I could come up with a mechanism to provide the most > 'real-time' progress notifications and a way to control the forked vm was > with something networkish => a client-server communication process, > currently I've implemented this with a java.nio socket client-server with > Apache MINA. To control the ports used by the server processes (over > multiple gradle builds running at the same time) I'm currently using a > single file in ~/.gradle/internal/testing/ports.used that is exclusively > locked while a new port is being determined or when a port is no longer > used. Currently I'm starting to use ports from 2000 on, I've just picked > this port so this may not be the best starting point. Ideally I want to > detect free ports so there is really no need for configuration for ports. > > With this client-server in place the forked vm gets work from the server > that dequeues work from the testInfosQueue (dequeueing in blocks of > 100(just > to name a number) tests). We can also provide a way for the forked vm to > wait until we want it to start executing tests ( this is usefull when a > user > wants to debug the tests, the user specifies -Dtest.debug Gradle asks the > user to confirm, which may be a first way of solving GRADLE-388, ideally > I'd > want to detect when the debugger is attached and proceed then). > > Progress events are send by the client and queued on the server side, this > can be used to notify BuildListeners and for test output. I'm currently > undecided where to do the output processing. I'm currently in favour of > doing this in the gradle build process as to limit the classpath of the > forked vm. I think we can re-use some of the Ant JUnit output code but not > sure if this is something that we want. > > I'd love to receive feedback on this. > > Thx, > > Tom > > -- View this message in context: http://www.nabble.com/Native-test-framework-integration-tp22815266p22820115.html Sent from the gradle-dev mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
