Mark, Yes. I agree that a central point to this implementation is the Ant install scripts. I advocate for DSpace to move to a pure java-based install process. However addressing that issue wasn't in the scope of this project. I do however disagree that we should link these two issues and say that we should wait on functional tests until we have a pure-java install. They are relatively independent, if the install process is ant-based we should use an ant-based install process for tests. On the other hand if its java-based then we should use the same for tests. If someone is currently working on a pure-java install for 1.6.1 then we might want to hold off till then.
I think one of the key points about this framework is that it installs DSpace using the exact same install process that a user would go through when installing DSpace. It ultimately call's ant's target for a fresh_install, and I think that it's important that the functional tests use an install process that is as close to production as we can make it. It's also important to note that the test cases themselves aren't effected by how DSpace is installed (ant vs java). So if the framework is in place and test cases are written they should still be effective when the install process is updated. Scott-- On Apr 6, 2010, at 11:50 AM, Mark Diggory wrote: > Scott, > > This was one of the projects proposed in the GSoC 2010 ideas and > something that I think the community would be interested in. I > suspect though there will be some design considerations in terms of > where files are going if we continue to have an interest in dropping > use of Ant in the installation process as has been discussed in recent > years. > > We also have considerations in that we have a great body of work in > Elliot Metsgers Mock DSpace object and JUint testing framework,. > > I really think it would be valuable to have something that is > standalone and separate from the core codebase for use in testing. It > would be especially valuable to have it be runnable against existing > new dspace deployment without having to patch it. With these > considerations I would recommend > > 1.) Not packaging testing classes with the dspace-api or other > libraries. Either place them within a separate test directory in those > projects or create a new project called something like "test-harness" > > 2.) Avoid using Ant build.xml files for this process if at all > possible, look to setting up everything in java directly so that it > can be actionable from maven, ant, CLI or Standalone. > > I agree, we need a focus point for this in JIRA, WIKI and I also > suggest that GSoC has some activities in this area that we should try > to connect the dots in. > > Mark > > > On Sun, Apr 4, 2010 at 1:21 PM, Scott Phillips > <scott.a.phill...@gmail.com> wrote: >> DSpace Functional Tests? >> >> The Texas Digital Library has been focusing on testability for our projects. >> Since DSpace is related too or part of most of our projects we’ve been >> looking for a way to increase DSpace’s testability. Traditionally this would >> mean adding unit tests and integration tests. However as DSpace currently >> stands is hard to break it up into individual components that can be tested >> in isolation. You’ll quickly find that writing tests for DSpace pull in the >> entire system, plus databases, and a file system. To address this problem >> we’ve created a simple framework for adding both integration tests and >> functional tests which improve the reliability of our projects. I’m >> interested to see if this is something the greater DSpace community would be >> interested in? >> >> The goals of our project were to create a mechanism where we could run >> complete functional tests. Functional tests evaluate the entire system as >> the end user would use it, so think of it as opening a web browser and >> evaluating the output – but completely automated. They test everything all >> together. Ideal it would be better to test each component individual, but >> this is in practical for DSpace for two reasons 1) DSpace is highly >> integrated and nearly impossible to separate from the database and file >> systems, 2) Creating unit test for all of DSpace is very time consuming it >> is simpler to write a few functional tests that cover a wide set of features >> over the whole application. It gets you to a point where you can reliably >> verify the software quicker. If you’re working on unit tests for DSpace >> please do not let this stand in your way. >> >> The main concept is to script the install of a test DSpace, with a full >> configuration and setup. Then we start DSpace in an embedded webserver and >> then run through several scenarios just as a normal user would. This tests >> the whole application, using a database, a file system, and a full build. >> The ant script where you normally run “ant fresh_install” has a new target >> “ant test”. You pass it a few parameters such as what database to use. The >> script will then run through a fresh install of DSpace into a local /test >> directory, setup some communities and collections, and import some basic >> items. Then JUnit-based tests are run against the embedded webserver using >> HtmlUnit to simplify verifying the HTML output. >> >> Here is how to run it. After compiling using a “mvn package”, cd into >> target/dspace-*-build.dir/ directory. Then run “ant test” you may need to >> pass it some parameters as listed below. Each parameter has a default so if >> you configure you’re database connections the same way then it can be as >> simple as running “ant test” without any parameters. >> >> -Dtest.db.driver="org.postgresql.Driver" >> -Dtest.db.url="jdbc:postgresql://localhost:5432/dspacetest" >> -Dtest.db.username="dspacetest" >> -Dtest.db.password="dspacetest" >> -Dtest.dspace.dir=”./test/" >> -Dtest.config=”./test/config/dspace.cfg" >> >> We’ve used this approach rather successfully for two of our DSpace-based >> projects here at TDL: an ETD submission system called vireo, and a learning >> object repository. These projects haven’t moved to 1.6 yet, but I do have a >> patch available for DSpace 1.5.2. Most of the test cases we’ve created so >> far are specific to the project we’re working on. However the patch includes >> 4 manakin tests, which are really just an example of how tests work within >> this framework. >> >> Download the >> patch: >> http://scott.phillips.name/wp-content/uploads/2010/04/DSpace-1.5.2-FunctionalTest.patch.txt >> >> The question is, is this something that the DSpace community would like? >> >> Scott-- >> >> ------------------------------------------------------------------------------ >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> _______________________________________________ >> Dspace-devel mailing list >> Dspace-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/dspace-devel >> >> > > > > -- > Mark R. Diggory > Head of U.S. Operations - @mire > > http://www.atmire.com - Institutional Repository Solutions > http://www.togather.eu - Before getting together, get t...@ther ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Dspace-devel mailing list Dspace-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-devel
