Hugh, > Having now got Selenium running I'm looking at some unit testing stuff. > > I've looked at the test runner at > http://qooxdoo.org/documentation/0.7/frame_apps_testrunner which seems > quite good. > > There are one or two problems which I've reported (issue 821, 822) > Yep, got them. One is fixed, the other we'll see... > I've been thinking about the way we could use Selenium to run unit tests, > automating the whole process. The testRunner app is great but it has to > be run manually. > But in general, this could be exactly the app you drive through Selenium. Would be a matter of "clicking" tree nodes and checking for the right outcome... > We can evaluate arbitrary code using the Selenium assertEval command. I > did a short test script for a tricky helper class I wrote yesterday, and > Selenium can run it and test the result while testing our application: > > tr> > <td>assertEval</td> > <td>var rm = new window.abling.util.CRangeManager(); > rm.AddRange(1,5,"1","5");rm.AddRange(6,10,"6" > ;,"10");String(rm);</td> > <td>1-10</td> > </tr> > > Output: > > [info] Executing: |assertEval | var rm = new > window.abling.util.CRangeManager(); > rm.AddRange(1,5,"1","5");rm.AddRange(6,10,"6","10");String(rm); | 1-10 | > Interesting approach. You could in principle pass your unit test classes code to assertEval and call their methods, but this is inconvenient since you have to maintain your test code in an HTML table, rather than a proper class file. Also, the unit test throw exceptions on failure which might be an issue in an assertEval.
Or, you could mimic the TestRunner and pass code to assertEval that just instantiates test classes (included from a normal class file) and runs their "test*" methods, catching exceptions and making corresponding "outputs" (e.g. <div>'s on the hosting web page). Those could be easily checked for in your Selenium test script. But now you are already close to the Testrunner itself, so it might be worth a try to Selenium-control the Testrunner directly, the biggest challenge probably being to catch and evaluate test results from the "Test Results" output pane. > One advantage of this is that we could add some simple methods to our > classes to enable the current state to be inspected while running our > whole application test. Therefore in addition to testing what's on the > screen we can verify other information internal to the application. > Every unit test can do that, and many qooxdoo unit tests already do it. (Actually, unit tests lend themselves much better to return values and internal state than GUI updates). > But also we can then build up and run arbitrary unit tests created in our > Python environment for running Selenium tests. > > However that separates the unit tests from the code which isn't very good, > and it would be difficult to debug the unit tests. > Indeed. If you were to avoid the Testrunner for Selenium tests, I would rather take my second approach, maintain proper Javascript unit test classes and drive them through a separate Javascript layer that replaces the Testrunner framework (instantiating, invoking, catching exceptions). > If the test runner tests were incorporated in a "make source" build > (perhaps with "make source-test") then we could invoke them from a source > version using injected code from Selenium; alternatively if they were > included in a "make build-test" version as well then we could test the > build compilation (probably quite important given the compression that is > done). > > Mh, I'm not sure I get the whole picture but I'll throw in a couple of remarks, see if it rings something with you... > A rough outline of what is needed is: > > - test runner testcases and basic stuff incorporated into main qooxdoo > code tree. > The trunk version has qx.dev.unit.*, that probably covers that. > - normal source and build targets exclude any unit testing code. > You can still include them if you like. > - additional make targets "source-test" and "build-test" which include > > the unit testing code but don't invoke it. The user could add an > arbitrary interface to run them from their UI, using qooxdoo variants. > The new generator will allow you to define arbitrary targets (aka "jobs" in the new speak) so it will be easy for anybody to produce various "source" and "build" versions of their app, with as many classes included as you wish for. > - add a standard API for invoking the test cases from Selenium more > easily. > Right, we could probably provide boilerplate code that runs standard qooxdoo unit test classes in a "Selenium-friendly" way (e.g. with the Simulator project) ... That's a nice thought :). > - "make test-source" and "make test" remain as they are to create the > testRunner application for manual testing. As above, Selenium API created > for running any individual test, or group of tests so that can be > automated as well (maybe just the same as when using the application). > Or run 'make simulator' and have a (possible non-GUI) app generated that includes app classes, unit test classes and the Simulator-friendly 'runner' classes. You could run this page immediately through e.g. Selenium RC, the tests start onload and the SRC client script checks the output of the <div>s. Doesn't sound so bad, does it?! The important point here is that the unit test classes do all the real work, interacting with the application classes. Ok, that would be my first brain dump so far... Cheers, Thomas ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ qooxdoo-devel mailing list qooxdoo-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
