I created my own tool which generates "dumb" HTTP requests to the server, combined with one watir user at the same time doing the actual measuring of response times. Because I also use the HTTPwatch API, I can report on both HTTP response times and real-user response times. Best of both worlds!
Cheers, John On Apr 3, 4:33 am, Chuck van der Linden <[email protected]> wrote: > On Apr 2, 2:01 pm, Paul Rogers <[email protected]> wrote: > > > most load tools do direct http posts/gets/put requests. jmeter does this > > using a java library, to get the high levels of virtual users. > > Browsermob uses amazon cloud running ms windows 2003 server and a real > > browser. > > Paul is absolutely right in this regard. This is because the overhead > of rendering and running client side code means if you use anything > remotely resembling a 'real browser' it won't scale well, and you will > need a small fleet of systems to generate the load. > > It's also un-necessary. some vendors who don't support protocol level > load will try to justify their tool as being 'more realistic' which I > say is utter one-hundred percent hogwash. The server has NO idea who > is sending those packets, and it can't tell IE or Firefox from a good > tool claiming to be IE or Firefox.. the 'realism' is all in the mind > of the salesman because the web server hasn't a clue. > > > > > Ive think the best way of doing load tests is to use jmeter or other tool. > > You may be able to record a script via the jmeter proxy when watir is > > running. I can see a lot of advantages to this, but it takes a lot of hand > > editing to go and make the recorded jmeter script usable for more than one > > user. > > > Ive found load testing to be a particulary difficult thing todo. Mostly > > because of the question "what are we trying to do here?" - find out what bit > > breaks first? Find out how many users we can support before we need a new > > bit of hardware? Find out what the slowest operastion is? Find out how to > > make the slowest operation better? There is a book that Scott Barber was a > > coauthor of, that is available as a free pdf that is good reading. > > I agree 100% Paul. load testing and perf testing is some of the more > 'scientific' testing you can do, it requires close attention to > detail, tight control over variables, tests that are highly > reproducable, and as you point out you have to start out by asking the > right question. A loadtest is basically an experiment, and like any > experiment it's only going to be as successful as the hypothesis it's > trying to prove or disprove. If you don't know the question you are > trying to ask, you're not likely to design loadtest that will give you > results you can use. > > As for myself, I've used both loadrunner (really good but really > expensive) and Microsoft's Visual Studio Team System2008 (both the > "test edition" and "team suite" versions have a HTTP level web-test > and loadtest capability) I found in my work that it scales well and > blows loadrunner away in terms of bang for the buck. I've used it > with just a single machine as the load generator (also running the > controller as wel) and brought both webservers and db servers to their > virtual knees and had they crying uncle! There's also an excellent > MSDN forum with some of the main pm's etc providing advice and > support, and a small collection of videos and such. > > The downside is that it does all the SUT monitoring via WMI > (Microsoft's implementation of WBEM) which works fabulous on windows > servers, but because the *nix community never implemented WBEM (a > multi vendor DMTF standard) there's no performance monitoring daemon > that allows you to connect to the machine using WBEM, and hence you > can't monitor the performance of non windows servers, and without good > (really good) data on how the systems are being affected by the load, > corrilated with what the scripts are doing, the loadtests are of much > less value. > > Sorry but this is a bit of a soapbox for me. it's really ironic in a > way that the "closed' windows OS, because MS implemented an open > standard is more accessable in terms of remote monitoring and > rathering of system information (out of the box without having to > installl anything, and merely configuring some accounts that are > allowed to access the data) than the supposedly 'open' os's which > never adopted that standdard and are thus a relative black hole in > terms of being able to remotely monitor performance counters > > --Chuck --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Watir General" group. To post to this group, send email to [email protected] Before posting, please read the following guidelines: http://wiki.openqa.org/display/WTR/Support To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/watir-general -~----------~----~----~----~------~----~------~--~---
