Hi Chaitanya, I guess I am looking at this from a browser view rather than from the server side. What I tend to find is that current load testing does not properly reflect the browser load times on our site. The reports from Google (in webmasters tools) and newrelic.com give a very different picture from that done during load/performance testing.
We use HP Load Runner for our in-house load testing and the numbers we get (even when we download 'all' external resources) does not come close to the numbers reported by the other tools. I guess I am attempting to address a gap that I see in our in-house load/performance testing, and the integration of webdriver into JMeter could solve this problem. At this stage it is an attempt to try to provide a means to simulate what our production servers see (i.e. using a real browser that will spend time executing JS/CSS). Yes, I agree that the complexity and cost of running a browser instance is higher than using the standard HTTP Sampler, but with modern services such as Amazon AWS, I would think that these problems would be addressable/solvable. regards, CP On 22 April 2012 06:30, chaitanya bhatt <[email protected]> wrote: > *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <[email protected] > >wrote: > >Once you have more load > >applied to the servers/services different characteristics tend to emerge, > >which is why I was thinking that the use of JMeter would be useful. > > *I may totally sound cynical while saying this but honestly the reason why > I am criticising your idea is mainly due to the week rational behind your > motivation to create this feature: > > 1. First of all browser induced latency on the total workload is grossly > overrated. Having said that, all that an accurate simulation of browser > latency would do to your workload is that it would marginally reduce the > total hits made on the server, which I think is bad from a fundamental load > testing perspective. > 2.To accommodate in-memory containers to hold 100s-1000s of virutal user > browser sessions is practically impossible without testers having to > significantly size up their test lab resources. Think about the cost > factor. > 3. Looking at todays trend of browser adoption your web driver would have > to be equipped with simulation capability multiple types of browsers such > as IE, Chromer, Safari etc. etc. and also you would have to consider > simulating various device characteristics on the generated load. Think > about the complexity. > > @Shmuel: The loadrunner protocol you are talking about is Ajax > TrueClient. HP came out with Ajax TrueClient not with an intention > of generating real browser traffic but it was mainly developed to reduce > the complexity of scripting apps that employ unintelligible data > serialization like Google GWT. With this HP's protocol you wouldn't have a > necessity of correlating session variables at all and there by you would be > significantly reducing the time to develop scripts. However, this protocol > currently has poor user adoption due to various limitations. > > Thanks > Chaitanya M Bhatt > http://www.performancecompetence.com > > On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <[email protected] > >wrote: > > > Hi, > > > > The tools mentioned help a great deal, however they generally look at it > > from a single browser to a single server scenario. Once you have more > load > > applied to the servers/services different characteristics tend to emerge, > > which is why I was thinking that the use of JMeter would be useful. > > > > Furthermore, if our primary audience is hitting the site using a browser, > > then would it not make sense to use the same tool to simulate the load? > > > > Just my thoughts. > > > > regards, > > CP > > > > On 21 April 2012 17:26, chaitanya bhatt <[email protected]> > wrote: > > > > > Pleanty of awesome tools are already available for free to meet your > > goals. > > > > > > Free: > > > https://developers.google.com/pagespeed/ > > > http://yslow.org/ > > > > > > Free or Premium: > > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx > > > > > > > > > Selenium has a webdriver which you can plan on extending - if you will. > > > IMHO Jmeter is not the right platform for this. > > > Thanks > > > Chaitnaya M Bhatt > > > http://www.performancecompetence.com > > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim < > [email protected] > > > >wrote: > > > > > > > Hi, > > > > > > > > After looking at the current HTTP Sampler, I would like to propose a > > > > WebDriver based implementation. The problem is that with a lot of > > modern > > > > websites using AJAX, advanced CSS3 (transitions/transforms) to do the > > > > rendering, the performance problem may not be 'visible' to the > standard > > > > HTTP Sampler. The idea is that if we could use JMeter to control a > > > browser > > > > and collect the time it takes to render each 'page' it would give the > > > users > > > > an understanding of how long a page would take to render. > > > > > > > > Some of the high level goals are as follows: > > > > > > > > 1. Allow any WebDriver supported browser to be used to perform the > > > > Sampling. > > > > 2. Provide a BeanShell/BSF style pane where the user can script the > > > > behaviour of WebDriver. > > > > > > > > I've got a very basic implementation working here: > > > > https://github.com/cplim/JMeter - you'll have to switch to the > > > 'webmeter' > > > > branch to see the changes. There are limitations with this proof of > > > > concept, mainly: > > > > > > > > 1. It only creates Firefox instances > > > > 2. I've only provided Javascript lang support using BSF. > > > > > > > > Some outstanding questions: > > > > > > > > 1. How to support Phone (Android/iOS) based WebDriver, as these > seem > > to > > > > expect to only connect to a single phone. > > > > > > > > I would be happy to contribute the changes back to JMeter. Can I get > > > some > > > > feedback on this proposal, and what I need to do to contribute my > > changes > > > > back to the main JMeter codebase? > > > > > > > > regards, > > > > CP Lim > > > > > > > > > >
