Zippy, What is the point you are trying to set? I agree with some of the stuff you mention while some of it is incorrect.
The other posts by Etsy and Steve Shouders are really misleading for people which are not familiar with APM and I wanted to post some feedback on that blog post by Steve for long time now, but haven't done so yet. My bottom line here is that it make no sense to compare RUM and Synthetic and trying to come up with numbers or any kind of a rule about the relation between them. The only true thing to say about the relation of them is that if you have synthetic monitoring trend that shows you an increase in response time, you will probably see the same trend in RUM and even this has exceptions(I.e. synthetic run from specific problematic network or monitoring specific problematic content). Do you have some new technology in mind to resolve the limits of load testing tools and bring the benefits from real browser profiling? Because as we use jmeter we try to understand what builds up the application and try to mimic that behavior as much as possible with our scripts.. nothing fancy about it. On Feb 7, 2013 6:17 AM, "David Luu" <[email protected]> wrote: > So based on the discussion, I take it the OP/Zippy has decided that > integrating Selenium with JMeter (e.g. calling Selenium from JMeter) or > running a Selenium test separately but in parallel/simultaneously as JMeter > to assess browser DOM/AJAX rendering & response time is not acceptable? > Because those 2 approaches will tackle the problem just recently being > discussed. You create the needed synthetic load while at same time be able > to more accurately gauge browser performance (w/o having to do it manually > by hand). > > Just curious to know why not? Too much work & lack of a team member with > proper expertise to devise the JMeter/Selenium solution? Granted it does > take some work to build, but there's never good/perfect free lunches, > make/customize it yourself "to personal taste" or pay $$$ for it. > > On Wed, Feb 6, 2013 at 7:41 PM, Deepak Shetty <[email protected]> wrote: > > > I notice you didnt actually say whats the difference between two browsers > > and one. > > > > In any case web test tools have always been in two categories > > Those that drive the http request/response (JMeter, Grinder,older > versions > > of load runner) and those that drive the browser (selenium, watir, qtp, > > newer versions of load runner). > > Do you seriously think that people who develop and/or use the first > > category are measuring wind speed with > > a wet finger or is it more likely you dont get what you need to do if you > > want to use one of these tools to still get "true" response times? > > > > > > > > > > On Wed, Feb 6, 2013 at 7:13 PM, Zippy Zeppoli <[email protected] > > >wrote: > > > > > It's the difference between measuring wind speed with an anemometer and > > > your wet finger in the air. > > > > > > On Wednesday, February 6, 2013, Deepak Shetty wrote: > > > > > > > >I think you may be missing the point. > > > > Heh - the feelings mutual > > > > >There is no DOM rendering happening...and it won't reflect the true > > > > response time > > > > If you need browser times , yes Jmeter cant help you directly. > > > > > > > > But browser render times are really irrelevant to a *load test*. Lets > > say > > > > using any tool you have loaded the server with some high load . Now > > Lets > > > > say you and I (assume the addition of two requests makes no > difference > > to > > > > the server). access this via a browser with similar conditions(same > > > > browser, network, cpu, memory etc). Is there any difference that you > > and > > > I > > > > will see? Do you really need two or many browsers to figure out how > > much > > > > time your DOM rendering is taking or will one browser suffice?(lets > > > ignore > > > > that you still arent getting "true" times - because browser times are > > > > dependent on what else the user is doing, what sort of network > > bandwidth > > > he > > > > has , what browser he is using, what are IE cache settings are and so > > > on). > > > > > > > > Pre - cloud , it was prohibitive to drive browsers to do load tests - > > now > > > > it is possible , but the amount of additional value that you get > over a > > > > http request/response load test and some browser analysis is minimal > to > > > > none. (Some types of scripts are easier to write with a browser > driven > > > tool > > > > though). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Feb 6, 2013 at 4:30 PM, Zippy Zeppoli < > [email protected] > > > <javascript:;> > > > > >wrote: > > > > > > > > > I think you may be missing the point. > > > > > Real load cannot be tested via HTTP interactions. > > > > > There is no DOM rendering happening. > > > > > I can make HTTP requests all day and it won't reflect the true > > response > > > > > time unless it's done through a browser. > > > > > > > > > > Recording a script in Jmeter proxy is trivial. Simulating *real* > user > > > > load > > > > > is not it requires a browser and interactions with a web > application. > > > > > > > > > > On Tue, Feb 5, 2013 at 6:51 PM, Deepak Shetty <[email protected]> > > > wrote: > > > > > > > > > > > >Actually that does matter it cannot do JavaScript. If a request > > > > requires > > > > > > >you need to be able to click a JavaScript button then the > request > > > will > > > > > > >never happen. > > > > > > The point is that what happens when the button is clicked? > Assuming > > > > its a > > > > > > server - ajax call then A HTTP call is made and some parameters > are > > > > > passed > > > > > > and some values are returned. Thats whats important for the load > > > test , > > > > > not > > > > > > the fact that javascript was executed. > > > > > > So when you record the script , you will be the person clicking > the > > > > > > button(you are recording your actions) , JMeter will record every > > > > > > interaction that makes a call to the server and will record this > > as a > > > > > > separate HTTP request and when you run the script the same > request > > > will > > > > > be > > > > > > made as if someone clicked the button! > > > > > > > > > > > > You dont need to use the recorder either , you can modify the > > script > > > > > > yourself. > > > > > > > > > > > > If the javascript didnt actually make any server side call - then > > it > > > > > doesnt > > > > > > matter because you dont want to load test this anyway. > > > > > > > > > > > > Have you actually tried this? It sounds as if you have a problem > > > > > recording > > > > > > your script and you probably have concluded that JMeter doesnt do > > > > > > javascript (true) and hence cant test websites that do > > > javascript/ajax > > > > > > (false) > > > > > > > > > > > > >Real browser is needed > > > > > > Not for a good deal of use cases - as many of the people on this > > > > mailing > > > > > > list can attest too. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Feb 5, 2013 at 6:44 PM, Zippy Zeppoli < > > > [email protected] > > > > > > >wrote: > > > > > > > > > > > > > Deepak, > > > > > > > Actually that does matter it cannot do JavaScript. If a request > > > > > requires > > > > > > > you need to be able to click a JavaScript button then the > request > > > > will > > > > > > > never happen. No request will ever be made. Also testing true > > web > > > > > > > performance requires rendering the DOM, not just initiating > HTTP > > > > > requests > > > > > > > and recording the response time, rps, etc. > > > > > > > > > > > > > > Real browser is needed, with JavaScript, and Jmeter doesn't > > > integrate > > > > > > well > > > > > > > with this, it isn't designed for this, which is understandable. > > The > > > > > > problem > > > > > > > is there is a gap between real browser testing (owned by third > > > party > > > > > > > companies) and open source tools (Jmeter). There's nothing in > > > between > > > > > for > > > > > > > real-browser based performance testing. I could go into why, > but > > > its > > > > > off > > > > > > > topic of this list, and I'd rather spare everyone the gas. > > > > > > > > > > > > > > Point being, Jmeter cannot solve my problem, without some > serious > > > > > > > customization. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Feb 5, 2013 at 6:33 PM, Deepak Shetty < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi > > > > > > > > You are getting too caught up in the JMeter doesnt do > > javascript > > > > > thing. > > > > > > > In > > > > > > > > most cases it doesnt matter. > > > > > > > > You have a webserver that is receiving HTTP requests - > whether > > > > those > > > > > > > > requests are generated via the user clicking a link or via > AJAX > > > or > > > > > via > > > > > > > > flash is hardly relevant to the webserver. It sees HTTP > > requests > > > > and > > > > > > > sends > > > > > > > > HTTP responses. > > > > > > > > JMeter deals with HTTP request and responses. As long as you > > can > > > > make > > > > > > the > > > > > > > > same request that your javascript is making (which you can > see > > > via > > > > > the > > > > > > > > record > > > > > >
