Mark, I agree with your definition that the proposed functional test framework is developer centric. It's meant to be run by developers. We've also been using selenium for tests with vireo. And I wouldn't argue against it's use, but it fills a slightly different need. The biggest deciding factor I see is who's willing to write what tests in which method. If we all adopt a test framework and no one rights tests then it's all for nothing.
Here's a few points about our experience with selenium tests with Vireo. - They were quicker and easier to create tests, but those tests were more brittle. By brittle I mean they tended to depended on inconsequential markup changes so you couldn't run the tests with another theme or caused excessive amounts of false positives. - They environment to run selenium was a pain to set up. In practice It wasn't a tool that a developer could use to validate the the system before committing a change. It's more applicable in the use-case where you're about to release a product and you dust off the old selenium scripts to give them a test instead of constant integration. The nice thing about the developer centric approach is that a developer is able to relatively easily test a change. This lets them address a problem more quickly while they still remember everything about the change, until waiting to later in the release cycle. Scott-- On Apr 7, 2010, at 7:28 PM, Graham Triggs wrote: > Ahhh... testing. It's an interesting subject. Because we can all see > the importance in being able to say that something is tested and > working - but we all have different experiences (or simply take > different things away from the same experiences) of what works, and > what doesn't. > > On 7 April 2010 23:58, Mark Diggory <mdigg...@atmire.com> wrote: > 2.) We need to get the modules properly broken up and out from the > synchronous release cycle like we discussed previously in our IRC > meetings. This means moving more of the dspace-xxx code into the > modules space and organizing accordingly. > > That's an important process, and can bring some improvements that > will enable more testing. But I don't see that it's explicitly linked. > > 3.) So a Testing Harness, even if called from Ant should be a > separate project that can be called independently against any DSpace > URL and I think you are getting us a good start here. > > HERE COMES THE IMPORTANT BIT!! > > Or rather, this is where it directly relates to Scott's original > approach. I totally agree that for system testing we need to hold > that as separate from the main codebase, and be able to run it > against an installation of choice. > > My one observation of Scott's patch is that as a specific > implementation, it's a very developer centric approach to writing > and performing tests in an area that isn't otherwise the exclusive > domain of a developer. > > I would favour the use of something like Selenium for testing at the > user level: > http://seleniumhq.org/ > > Which would open up the ability to author and run tests to a wider > field than developers. It's probably also just plain faster to work > with for most developers too (although I can't in any way quantify > that statement). > > 4.) However, work like dspace-discovery will completely violate > these test classes as they replace significant Search and Browse > logic. > > In theory, testing at the UI level, it ought to be possible to > replace significant logic - like switching search and/or browse out > to dspace-discovery - without invalidating the test scripts. That > depends on what you expect to have out of the final system. > > But really, it's fine if the UI tests have to change. It's probably > fine if true unit tests have to change. You can't evolve if you > don't remove, alter, try new things. The system has to change, and > therefore so do the tests. > > The alternative is that we declare DSpace is 'complete', > congratulate ourselves on a job well done and enjoy a jolly good > drink at Open Repositories 2010. > > G > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev_______________________________________________ > Dspace-devel mailing list > Dspace-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/dspace-devel ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
