+1 Caleb
Marius Dumitru Florea wrote: > +1 > > Thanks, > Marius > > On 06/25/2010 07:08 PM, Alex Busenius wrote: >> Hello devs, >> >> >> I'd like to add a new test framework aimed on catching XML escaping bugs >> that we currently have almost everywhere. Escaping bugs cause broken, >> unusable pages when special characters are used in >> space/page/category/user/whatever names and often present serious >> security issues. This test framework will run automatic tests and also >> allow for writing manual tests. >> >> >> Currently, escaping tests are usually written using selenium2, see e.g.: >> http://svn.xwiki.org/svnroot/xwiki/enterprise/trunk/distribution-test/ui-tests/src/test/it/org/xwiki/it/ui/EscapeTest.java >> This is a bad solution, slow and tricky to write. >> >> Writing unit tests for such bugs is also not a good solution, because >> most of the time they originate in velocity templates. It is hard to >> test a single template in isolation, they usually require access to many >> API functions. >> >> >> The solution I propose works similar to webstandards tests. The >> framework searches for all templates and documents in XWiki Enterprise >> *.xar and *.war and generates automatic escaping tests for each of them. >> Each test then requests URLs from XWiki server with different >> parameters and runs a custom validator that searches for not escaped >> data in the response. Manual tests can also be added easily. >> >> The implementation can be found in the sandbox: >> >> https://svn.xwiki.org/svnroot/xwiki/contrib/sandbox/xwiki-enterprise-test-escaping/ >> >> It currently implements some simple automated escaping tests on all >> templates and applications from the XAR. The parameters to use are >> chosen based on the source, using matching with regular expressions >> (searches for things like $request.get("something")). >> >> Full test run takes 8 minutes on my laptop. Current results: Tests run: >> 499, Failures: 166, Errors: 3, Skipped: 0 (there are still some false >> positives that I need to track down though). >> >> Files and test classes can be excluded from the tests using various >> properties set in maven (see pom.xml). Usual -Dtest and -Dpattern are >> also supported, e.g. to test all pages from "Blog" space, run: >> >> mvn -Dtest=ApplicationTest -Dpattern='^Blog/' install >> >> >> Some implementation details: >> >> org.xwiki.escaping.suite >> It is a custom JUnit4 test suite. It reads given zip file, then creates >> and initializes instances of the test class for each found file. The >> test class must implement FileTest (defines initialize()) and annotated >> with @RunWith(ArchiveSuite.class). Test classes can decide based on >> file name and content whether they can test the given file or not. >> ApplicationTest and TemplateTest use this suite. >> >> org.xwiki.escaping.framework >> Contains base classes for the escaping tests (implement downloading data >> from server, regex-based "parsing" etc.), escaping validator etc. >> >> org.xwiki.escaping >> Contains the tests (2 automatic and 1 manual). >> Instances of the test classes (built by ArchiveSuite or manual ones) a >> run using the default JUnit4 test runner, so all fancy test method >> annotations are supported. >> >> >> I propose to add this test framework to the main repository starting >> with 2.5 branch. This will give us more time to fix current issues >> (problematic files can be easily excluded to generate less noise). >> >> >> WDYT? >> >> >> Regards, >> >> Alex >> _______________________________________________ >> devs mailing list >> [email protected] >> http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
