Hi Jerome, On Jan 17, 2012, at 3:48 PM, Jerome Velociter wrote:
> Hi, > > As anyone who went trough the hassle of creating manually a whole > distribution for the sake of having functional tests for an > application, I can only be enthusiastic about this proposition :) > > Big +1 > > Another topic would be how do we make it easier/faster to write tests > ? I'm playing with the thought that it would be easier to write > functional tests in groovy (maybe not the page objects, but the tests > themselves). Or maybe something like geb (http://www.gebish.org/) all > the way would make sense in that direction. > Another thing I would like to dig some day would be the possibility to > write tests in the wiki application itself and then run them either > from the browser or from maven. That might not be easy to achieve > though, since it requires starting another browser with a clean > profile... from the browser itself. It could be done with the help of > a RC daemon on the local machine, then the browser sends commands to > that daemon, which in turn runs commands WebDriver. > Something in the spirit of jasmine maven integration, where you start > a jasmine daemon with mvn jasmine:bdd > (http://searls.github.com/jasmine-maven-plugin/) > Food for thought. I'd like that too. I even started an experiment back in 2007 (time flies!): http://extensions.xwiki.org/xwiki/bin/view/Extension/Selenium+Application The only problem is that "Make it fast to write test" is not really compatible with "Write the tests inside the wiki". The main problem is that or wiki is not yet a full-fledged IDE and we don't have syntax coloring but more importantly auto-completion, ability to easily search in code and follow methods calls, find usages, etc So right now writing a test in the IDE is much faster than in the browser. Note that with our Page Objects strategy it's really easy to write a test in java using our "DSL". Unfortunately we often have to add new PO since often new tests are about new features… ;) Note also that I've tried in the past to use recording tool and so far I've not found one that worked well. They always had issues. It's easy to understand. Imagine for example the following scenario: you got to a page and then click on the attachment tab at the bottom. The recorded script will do just that… But when executed it'll fail very often. Why? Because the load of the attachment tab is asynchronous and you need to wait for it to be loaded before being able to assert its content. So recording helps a bit but the problem is that usually the generated script is hard to read and you need to fully understand it in order to be able to fix the script so that it works. All in all it in lots of cases (if not all) it takes longer than to write the test from the beginning… I'm all for doing experiments though :) We just need to address the issues mentioned above. Thanks -Vincent PS: Note that we have a license for Greenpepper which does exactly that: writing tests in XWiki ;) http://www.greenpeppersoftware.com/confluence/display/GPW/Home/ > Jerome > > On Sun, Jan 15, 2012 at 11:27 AM, Vincent Massol <[email protected]> wrote: >> Hi devs, >> >> I've started an experiment to have colocated functional tests (CFT), which >> means having the functional tests located where the functional domain >> sources are located instead of in XE. >> >> For example for the linkchecker module we have the following directories: >> >> xwiki-platform-linkchecker/ >> |_ xwiki-platform-linkchecker-refresher (JAR) >> |_ xwiki-platform-linkchecker-ui (XAR) >> |_ xwiki-platform-linkchecker-tests (functional tests) >> >> The rationale for this was: >> * Have everything about a functional domain self-contained (source and all >> tests) >> * Making it easy to run only tests for a given functional domain >> * Move page objects to the functional domain too >> >> Here are some findings about this experiment: >> >> A - It takes about 30 seconds to generate the adhoc packaging and start >> XWiki. This would be done for each module having functional tests compared >> to only once if all tests were executed in XE >> B- The package mojo created to generate a full packaging is quite nice and I >> plan to reuse it in lots of other places in our build (distributions, >> database, places where we need XWiki configuration files) >> C- We will not be able to run platform builds in Maven multithreaded mode >> since it would mean that several XWiki instance could be started at the same >> time on the same port >> D- The colocated functional test module >> >> Solutions/ideas: >> >> * One idea to overcome A and C would to have the following setup: >> ** Keep functional test modules colocated but have them generate a test JAR >> ** Still allow running functional tests from the colocated module (this >> makes it easy to verify no regression was introduced when making changes to >> a given domain) >> ** Have functional tests in XE depend on the colocated functional test >> module JARs and configure Jenkins to run all functional tests from XE only >> >> * Another solution to overcome C is to auto-discover the port to use in our >> XWiki startup script (and save it in a file so that the stop script can use >> it). >> >> I think the first proposal is the best one and brings the best of the 2 >> worlds. >> >> WDYT? >> >> Thanks >> -Vincent >> >> _______________________________________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/mailman/listinfo/devs > > > > -- > Jérôme Velociter > Winesquare > http://www.winesquare.net/ > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
