Cool feedback - good ideas. No doubt we will iterate and improve the way we test the daemon.
So far I've been leaning towards below strategy for daemon testing (it's personal and very biased - you've been warned :). 1. I'd like to have functional tests that specifically target the daemon infrastructure and are decoupled from gradle builds (for better maintainability). This probably means we need to aim to decouple daemon infrastructure (as much as possible) from 'executing gradle builds'. Example of such test: DaemonFunctionalTest (gives some headache but we will iterate until it's perfected). I'm not very interested in running those tests in 'embedded daemon' mode because they are focused on the daemon infrastructure and the essence of this infrastructure is cross-process synchronization and communication. 2. Integration tests that actually run gradle builds and use our precious GradleExecuter. Example of such test: EnvironmentVariablesIntegrationTest (very interesting especially when executed with the Daemon!). Other examples: all other integ tests. 3. unit tests The combination of those 3 testing strategies gives excellent coverage and good maintainability. The idea is something that clarified for me after some time of working with the daemon code. Initially I started differently: using daemon distribution executor to run mini-builds, synchronizing builds via files, etc. However, that didn't work very well (high maintenance, slow cycles, difficult debugging, etc.) Let's talk about daemon testing sometime today or tomorrow. I have couple of questions. Cheers! On Mon, Sep 12, 2011 at 12:18 AM, Adam Murdoch <[email protected]>wrote: > > On 11/09/2011, at 6:09 PM, [email protected] wrote: > > Branch: refs/heads/master > Home: https://github.com/gradle/gradle > > Commit: 05308efa20db9712f2def074e9e2cb3577dbb86b > > https://github.com/gradle/gradle/commit/05308efa20db9712f2def074e9e2cb3577dbb86b > Author: Szczepan Faber <[email protected]> > Date: 2011-09-11 (Sun, 11 Sep 2011) > > Changed paths: > M > subprojects/integ-test/src/integTest/groovy/org/gradle/integtests/daemon/DaemonFunctionalTest.groovy > > > I think this should become a real integration test. If you look at the > method names, they reflect features we want Gradle to have regardless of > which interface you use: daemons expire, stops all daemon, starts a new > daemon if daemon is busy, etc. This is stuff we want coverage for, so it > would be nice if we can reuse this test in different contexts. > We should introduce a daemon integ test fixture, with 2 implementations: an > 'embedded' mode, which bolts directly onto DaemonClient and DaemonRegistry, > and a 'forking' mode, which runs real builds to start the daemon, and uses > 'gradle --stop' to stop the daemons. > It would need to use the DaemonRegistry for now, to probe the status of the > daemons. But this is something we want to add to the command-line, at some > point, eg http://issues.gradle.org/browse/GRADLE-1462. > > We should also get rid of the Sleep command, and just run a build that > sleeps for the appropriate amount of time. Plus find a way of synchronising > between the test and the test build that doesn't involve hard-coded sleeps, > perhaps using a marker file. > > > -- > Adam Murdoch > Gradle Co-founder > http://www.gradle.org > VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting > http://www.gradleware.com > > -- Szczepan Faber Principal engineer@gradleware Lead@mockito
