Yes, I do have an execution manager script which reads the xml files where all settings are stored (duration, rampu up, start time, number of users, etc, etc,) and then takes care of things like waiting for the desired start time. The actual virtual users are started with a seperate thread for each user and with keeping the desired rampup in mind. The first one is always the watir script, every next one is a http user. When the thread finishes , the execution manager checks if the test duration has been reached. If not it restarts the thread. And at the end all reporting duties are done. So this is the master script.
Scenarios are defined in XML files, and with that it is possible to have for instance 10 HTTP users doing a user registration, 25 HTTP users browsing the catalag, etc. The Watir script runs all areas covered in the HTTP scipts to do the measurements as a single user. The actual measuring of response times and other data is done with the watir script. I measure the end user performance with setting timers in the script just before and after doing some action, like clicking a link. Besides that, I use the HTTPwatch API to get HTTP response times, page sizes, object sizes, overview of all objects on a page, etc. So with both the end-user performance and the HTTP performance, I know what the client side execution time was (taking into account some script overhead ofcourse). It is still in development, has some rough edges, but has proven already to be quite valuable in my current project. This is mainly because it is a website with lots of ajax and client-side execution of javascript which loads additional data. I was very unhappy with LoadRunner, because it measures only the initial HTTP performance. Not the client-side execution, not the ajax parts without putting a lot of extra effort. Using Watir for this was pretty easy to do and a joy to think out. Of course I have the benefit I have a lot of experience with Watir already, because our regression tests are already automated with Watir. Cheers, John On Apr 16, 6:02 pm, Chuck van der Linden <sqa...@gmail.com> wrote: > that sounds like a really interesting project. > > how are you instrumenting and capturing performance counter data for > the system under test? and what perf counters are you looking at? > > do you have some kind of master controller script that is launching > the load threads, so you can do ramping of the load, and correlate how > many vusers are running at a given time? perhaps have different > vusers running different load scenarios? > > On Apr 15, 10:21 pm, JArkelen <johnvanarke...@gmail.com> wrote: > > > I'm currently using watir to develop a end-user performance testing > > tool (with x number of users generating http requests and 1 user > > running a watir script and do the actual measuring). For reporting, I > > store all raw data in csv files and then at the end of the test this > > raw data is used to generate a html report, or a PDF report or an > > aggregated excel file, or an excel dashboard or added to a mysql > > database, based on the reporting settings. This way I can report my > > test results in any kind of format I like, because it depends very > > much on the specific situation what kind of reporting I want. The base > > is however always the same: raw csv data. > > > On Apr 16, 12:57 am, Paul Rogers <paul.rog...@shaw.ca> wrote: > > > > if you look in Lisa Crispins book, page 323, there is an example of a test > > > reporting app like you describe. This was written in rails and was in use > > > ( > > > and still is ) at financial internet company. Disclaimer - I wrote this > > > app > > > ;-) > > > > I think there is a big opportunity for someone to write n open source > > > version of this, many people like the idea. But it does depend on the > > > nature > > > of your company. Where Im working things like that are interesting to the > > > managers or development group. > > > > Paul > > > > On Wed, Apr 15, 2009 at 4:19 PM, Tiffany Fodor <tcfo...@comcast.net> > > > wrote: > > > > > Hi! > > > > > I've been using ci_reporter to generate xml reports of my Test::Unit > > > > results (also works with RSpec) and then I use xsl to transform them > > > > to html. It's not the best and I'm working on better reporting in my > > > > next framework. I'll be using Rails, so I'll likely store results in > > > > MySQL and then make reporting pages in the Rails app. > > > > > -Tiffany > > > > > On Apr 15, 3:56 pm, Jason Shelton <jas.shel...@hotmail.com> wrote: > > > > > All, > > > > > > I have always used Excel as my test output/results format, but I would > > > > like to enhance this. I have spoken with our DBA, and, if necessary he > > > > will > > > > create a table in our database for my test automation results. What I > > > > am > > > > asking is if you all can recommend any techniques and/or methodologies > > > > for > > > > reporting your test results. What worked for you? What looks nice? > > > > As I > > > > said, I have always used Excel, with the green or red cells(pass/fail), > > > > but > > > > I would like to take my test results to the next level. Thanks in > > > > advance > > > > for all of your help. > > > > > > - Shelton > > > > > > _________________________________________________________________ > > > > > Internet Explorer 8 – Get your Hotmail Accelerated. Download free! > > > >http://clk.atdmt.com/MRT/go/141323790/direct/01/-Hide quoted text - > > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Watir General" group. To post to this group, send email to watir-general@googlegroups.com Before posting, please read the following guidelines: http://wiki.openqa.org/display/WTR/Support To unsubscribe from this group, send email to watir-general-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/watir-general -~----------~----~----~----~------~----~------~--~---
