Hi Shmuel, I've yet to try the WebSampler implementation on our test environment. Unfortunately I'm away for the next 2 weeks, but when I get back to work I can try out the WebSampler and see if I can replicate the statistics I'm getting in our production environment.
regards, CP On 22 April 2012 23:49, Shmuel Krakower <[email protected]> wrote: > I'd suggest to HP to drop C language scripting in order to make scripting > easier / shorter. > > As Chaitanya said - browser response times is overrated. > > Anyhow, I agree that implementing load tests with hundreds / thousands of > virtual users with that kind of a sampler(which running a browser) will > require much more hardware but as CP said, with today's cloud services this > is very affordable, thus this is not the issue. > I think that there are services on the web which already doing this ( > https://browsermob.com/website-load-testing-features), but I never tried > those. > > CP, if you get different numbers between you monitoring solution and JMeter > reports, you should try to understand why that happens. > It might be for many reasons. Did developing this sampler resolved your > original problem? Do you get more close numbers now? > > Shmuel. > > > On Sun, Apr 22, 2012 at 5:27 AM, Cheen-Pin Lim <[email protected] > >wrote: > > > Hi Chaitanya, > > > > I guess I am looking at this from a browser view rather than from the > > server side. What I tend to find is that current load testing does not > > properly reflect the browser load times on our site. The reports from > > Google (in webmasters tools) and newrelic.com give a very different > > picture > > from that done during load/performance testing. > > > > We use HP Load Runner for our in-house load testing and the numbers we > get > > (even when we download 'all' external resources) does not come close to > the > > numbers reported by the other tools. I guess I am attempting to address > a > > gap that I see in our in-house load/performance testing, and the > > integration of webdriver into JMeter could solve this problem. At this > > stage it is an attempt to try to provide a means to simulate what our > > production servers see (i.e. using a real browser that will spend time > > executing JS/CSS). > > > > Yes, I agree that the complexity and cost of running a browser instance > is > > higher than using the standard HTTP Sampler, but with modern services > such > > as Amazon AWS, I would think that these problems would be > > addressable/solvable. > > > > regards, > > CP > > > > On 22 April 2012 06:30, chaitanya bhatt <[email protected]> > wrote: > > > > > *On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim < > [email protected] > > > >wrote: > > > >Once you have more load > > > >applied to the servers/services different characteristics tend to > > emerge, > > > >which is why I was thinking that the use of JMeter would be useful. > > > > > > *I may totally sound cynical while saying this but honestly the reason > > why > > > I am criticising your idea is mainly due to the week rational behind > your > > > motivation to create this feature: > > > > > > 1. First of all browser induced latency on the total workload is > grossly > > > overrated. Having said that, all that an accurate simulation of browser > > > latency would do to your workload is that it would marginally reduce > the > > > total hits made on the server, which I think is bad from a fundamental > > load > > > testing perspective. > > > 2.To accommodate in-memory containers to hold 100s-1000s of virutal > user > > > browser sessions is practically impossible without testers having to > > > significantly size up their test lab resources. Think about the cost > > > factor. > > > 3. Looking at todays trend of browser adoption your web driver would > have > > > to be equipped with simulation capability multiple types of browsers > such > > > as IE, Chromer, Safari etc. etc. and also you would have to consider > > > simulating various device characteristics on the generated load. Think > > > about the complexity. > > > > > > @Shmuel: The loadrunner protocol you are talking about is Ajax > > > TrueClient. HP came out with Ajax TrueClient not with an intention > > > of generating real browser traffic but it was mainly developed to > reduce > > > the complexity of scripting apps that employ unintelligible data > > > serialization like Google GWT. With this HP's protocol you wouldn't > have > > a > > > necessity of correlating session variables at all and there by you > would > > be > > > significantly reducing the time to develop scripts. However, this > > protocol > > > currently has poor user adoption due to various limitations. > > > > > > Thanks > > > Chaitanya M Bhatt > > > http://www.performancecompetence.com > > > > > > On Sat, Apr 21, 2012 at 4:07 AM, Cheen-Pin Lim <[email protected] > > > >wrote: > > > > > > > Hi, > > > > > > > > The tools mentioned help a great deal, however they generally look at > > it > > > > from a single browser to a single server scenario. Once you have > more > > > load > > > > applied to the servers/services different characteristics tend to > > emerge, > > > > which is why I was thinking that the use of JMeter would be useful. > > > > > > > > Furthermore, if our primary audience is hitting the site using a > > browser, > > > > then would it not make sense to use the same tool to simulate the > load? > > > > > > > > Just my thoughts. > > > > > > > > regards, > > > > CP > > > > > > > > On 21 April 2012 17:26, chaitanya bhatt <[email protected]> > > > wrote: > > > > > > > > > Pleanty of awesome tools are already available for free to meet > your > > > > goals. > > > > > > > > > > Free: > > > > > https://developers.google.com/pagespeed/ > > > > > http://yslow.org/ > > > > > > > > > > Free or Premium: > > > > > http://ajax.dynatrace.com/ajax/en/content/c-speed-page-load.aspx > > > > > > > > > > > > > > > Selenium has a webdriver which you can plan on extending - if you > > will. > > > > > IMHO Jmeter is not the right platform for this. > > > > > Thanks > > > > > Chaitnaya M Bhatt > > > > > http://www.performancecompetence.com > > > > > On Fri, Apr 20, 2012 at 11:55 PM, Cheen-Pin Lim < > > > [email protected] > > > > > >wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > After looking at the current HTTP Sampler, I would like to > propose > > a > > > > > > WebDriver based implementation. The problem is that with a lot > of > > > > modern > > > > > > websites using AJAX, advanced CSS3 (transitions/transforms) to do > > the > > > > > > rendering, the performance problem may not be 'visible' to the > > > standard > > > > > > HTTP Sampler. The idea is that if we could use JMeter to > control a > > > > > browser > > > > > > and collect the time it takes to render each 'page' it would give > > the > > > > > users > > > > > > an understanding of how long a page would take to render. > > > > > > > > > > > > Some of the high level goals are as follows: > > > > > > > > > > > > 1. Allow any WebDriver supported browser to be used to perform > > the > > > > > > Sampling. > > > > > > 2. Provide a BeanShell/BSF style pane where the user can script > > the > > > > > > behaviour of WebDriver. > > > > > > > > > > > > I've got a very basic implementation working here: > > > > > > https://github.com/cplim/JMeter - you'll have to switch to the > > > > > 'webmeter' > > > > > > branch to see the changes. There are limitations with this proof > > of > > > > > > concept, mainly: > > > > > > > > > > > > 1. It only creates Firefox instances > > > > > > 2. I've only provided Javascript lang support using BSF. > > > > > > > > > > > > Some outstanding questions: > > > > > > > > > > > > 1. How to support Phone (Android/iOS) based WebDriver, as these > > > seem > > > > to > > > > > > expect to only connect to a single phone. > > > > > > > > > > > > I would be happy to contribute the changes back to JMeter. Can I > > get > > > > > some > > > > > > feedback on this proposal, and what I need to do to contribute my > > > > changes > > > > > > back to the main JMeter codebase? > > > > > > > > > > > > regards, > > > > > > CP Lim > > > > > > > > > > > > > > > > > > > > >
