+1 Sounds great!
- Asiri On Fri, Jun 25, 2010 at 9:38 PM, Alex Busenius <[email protected]>wrote: > Hello devs, > > > I'd like to add a new test framework aimed on catching XML escaping bugs > that we currently have almost everywhere. Escaping bugs cause broken, > unusable pages when special characters are used in > space/page/category/user/whatever names and often present serious > security issues. This test framework will run automatic tests and also > allow for writing manual tests. > > > Currently, escaping tests are usually written using selenium2, see e.g.: > > http://svn.xwiki.org/svnroot/xwiki/enterprise/trunk/distribution-test/ui-tests/src/test/it/org/xwiki/it/ui/EscapeTest.java > This is a bad solution, slow and tricky to write. > > Writing unit tests for such bugs is also not a good solution, because > most of the time they originate in velocity templates. It is hard to > test a single template in isolation, they usually require access to many > API functions. > > > The solution I propose works similar to webstandards tests. The > framework searches for all templates and documents in XWiki Enterprise > *.xar and *.war and generates automatic escaping tests for each of them. > Each test then requests URLs from XWiki server with different > parameters and runs a custom validator that searches for not escaped > data in the response. Manual tests can also be added easily. > > The implementation can be found in the sandbox: > > > https://svn.xwiki.org/svnroot/xwiki/contrib/sandbox/xwiki-enterprise-test-escaping/ > > It currently implements some simple automated escaping tests on all > templates and applications from the XAR. The parameters to use are > chosen based on the source, using matching with regular expressions > (searches for things like $request.get("something")). > > Full test run takes 8 minutes on my laptop. Current results: Tests run: > 499, Failures: 166, Errors: 3, Skipped: 0 (there are still some false > positives that I need to track down though). > > Files and test classes can be excluded from the tests using various > properties set in maven (see pom.xml). Usual -Dtest and -Dpattern are > also supported, e.g. to test all pages from "Blog" space, run: > > mvn -Dtest=ApplicationTest -Dpattern='^Blog/' install > > > Some implementation details: > > org.xwiki.escaping.suite > It is a custom JUnit4 test suite. It reads given zip file, then creates > and initializes instances of the test class for each found file. The > test class must implement FileTest (defines initialize()) and annotated > with @RunWith(ArchiveSuite.class). Test classes can decide based on > file name and content whether they can test the given file or not. > ApplicationTest and TemplateTest use this suite. > > org.xwiki.escaping.framework > Contains base classes for the escaping tests (implement downloading data > from server, regex-based "parsing" etc.), escaping validator etc. > > org.xwiki.escaping > Contains the tests (2 automatic and 1 manual). > Instances of the test classes (built by ArchiveSuite or manual ones) a > run using the default JUnit4 test runner, so all fancy test method > annotations are supported. > > > I propose to add this test framework to the main repository starting > with 2.5 branch. This will give us more time to fix current issues > (problematic files can be easily excluded to generate less noise). > > > WDYT? > > > Regards, > > Alex > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
