We use a combination of tools for test automation. We could have purchased vendor tools, we chose the tools we have because they fit our needs.
The tools we use are: JUnit for programmer-facing tests (we practice TDD) FitNesse for customer-facing tests behind the GUI (and occasionally, programmer-facing tests, when it's easier to do TDD for a particular bit of code that way) WebTest for regression tests using the GUI WATIR for some GUI regression testing and scripts to set up for manual tests, where we needed a lot of flexibility and particular features (WebTest probably provides some of the features through using Groovy or similar, but for this it was easier for us to ramp up on Ruby) We chose WebTest 3 years ago when we started an agile development approach. We had no automated tests at all. We looked for a tool that was programmer-friendly, but easy for a tester with some technical skills to ramp up on, fairly independently. The fact that it integrates so easily with CruiseControl was another big 'selling' point. Within 8 months, we had a suite of smoke tests than covered all the critical parts of our app. This suite first ran as part of a nightly build, now it runs as part of a continual 'full' build that also runs FitNesse suites. At some point we had a lot of trouble with Javascript so we started using WATIR for scripts where we couldn't get WebTest to work. I think this was a couple years ago. My fellow tester was a Perl programmer so it was easy for him to get up to speed on Ruby and he wrote some nice scripts that accepted lots of command line parameters to set up particular data scenarios for manual testing. He preferred WATIR to WebTest, just a personal preference, so he tended to automate all his GUI tests there. We still used our extensive WebTest suite. The WATIR tests are very difficult and expensive to maintain. The tests have to be tested because they contain complex logic, whereas the WebTest scripts are specified and we rarely use any logic such as the ifStep. Also, we haven't gone to the trouble to try to integrate WATIR scripts with our build process (since it runs on Windows). At Agile 2006 I attended Paul King's WebTest tutorial and realized there were lots of robust new features. Meanwhile, the author of most of our WATIR scripts left and the maintenance had become a big burden. Everyone was having to learn Ruby (I enjoyed learning it myself, but it is difficult for me) and try to understand some complex code in order to work on the WATIR scripts. We looked back to WebTest and discovered all the Javascript issues we had with WebTest scripts in the past were solved, and we've been automating pretty much all of our GUI tests in WebTest since then. The tests are much quicker to write and cheap to maintain. We have to sharpen up some of our HTML coding but we should do that anyway. We will keep our WATIR script and maintain it, but I expect we will write most tests for new features in WebTest. We are very happy with our combination of tools covering our different needs. We're always open to new tools - a couple of the programmers saw Selenium at a conference and are eager to explore it more. But as the changes in WebTest have shown, good tools grow and change, and the team's needs or skill sets may change too, so it's important to always keep all options open. Our WebTest scripts, while quite simple, find regression bugs quite often. We don't have to change them often, and when we do (due to UI changes) it goes quickly. The ROI on them is tremendous. I've used many vendor tools successfully over the past 10 - 12 years, and the ROI on WebTest is many orders of magnitude higher than any tool I've used. Our programmers are able and willing to update scripts when necessary (with much less complaining than they do about the Ruby scripts) and our entire team is happy with it. -- Lisa On 11/23/06, Marc Guillemot <[EMAIL PROTECTED]> wrote:
Hi all, why do you use WebTest rather than some other test tool (commercial or open source)? Of course I have some ideas on this question ;-) but as WebTest committer I'm surely not objective when comparing WebTest to others. While presenting WebTest I've already encountered difficulties to convince that it was the tool of choice. I think that it occurs particularly when people don't have any real experience of automated web application tests and therefore can't understand "real" arguments. From your experience as WebTest user, what would you say to persuade someone wanting to start test automation for a webapp to use WebTest rather than something else? Marc. _______________________________________________ WebTest mailing list [email protected] http://lists.canoo.com/mailman/listinfo/webtest
-- Lisa Crispin Co-author, Testing Extreme Programming http://lisa.crispin.home.att.net
