Agree - my background is in writing commercial test tools - was team lead for Compuware on the web team for QALoad, before that with Rational/Pure/Performix (way way back when). So the big tests 1000+ users are what I'm used to dealing with. Right now I've been asked to evaluate jmeter against the commercial tools and scalability is always the killer - esp when you look at the tools that drive the server by driving instances of a browser (yuck!). I'm really liking jmeter - much better than I expected it to be. I expected to find lot's of holes and limitations (the arrogance of the old-timer) but really haven't seen any that aren't there for pretty good reason - like the proxy/ssl issue.
To distill down something you said, and ignoring the network limitations (since that's such a variable): so in your experience you get about 100 threads per machine before it saturates it? Is it typically CPU bound or memory or...? J ----- Original Message ----- From: "Peter Lin" <[EMAIL PROTECTED]> To: "JMeter Users List" <[EMAIL PROTECTED]> Sent: Thursday, July 15, 2004 11:43 AM Subject: Re: Any other limitations of jmeter? How about scalability? > for general performance testing stuff, I have several performance > articles listed on the links page > http://wiki.apache.org/jakarta-jmeter/JMeterLinks. > > If I compare JMeter to other tools I've used, I would say it is > comparable. The tools I've used: > > 1. apache ab > 2. zdnet Webbench > 3. custom script perl > > The biggest challenge of stress testing is this, "what are you > testing?" Most of the time, the bottleneck for a website is the > connection to the internet. I have a explanation of it in the > articles. It doesn't really matter if your website can handle 80 > concurrent requests reliably, if you're server is on a T1. I would say > JMeter can easily produce enough stress to figure out what the upper > limit is for your web application. The situations where it won't be > suitable is if you're testing a system that has to support 5,000 > concurrent users. You still could use JMeter if you setup each > instance with 100 threads and use 50 machines. > > realistically, to do that kind of test, the server would have to be a > big Unix box or mainframe hooked up to several gigabit routers. Using > 100mbit ethernet, you wouldn't be able to test 5K concurrent requests > for a sustained period of time effectively. Sustained transfer rate > plays a huge role in how many concurrent requests a system can handle > reliably for a long time. > > peter > > > > On Thu, 15 Jul 2004 11:25:50 -0400, joelsherriff > <[EMAIL PROTECTED]> wrote: > > After Peter pointed out that the jmeter proxy doesn't do ssl, I thought I'd > > ask everyone if they know of other limitations they've run into and about > > what seems to be the cronic limitation of these tools - scalability. Not > > talking about bugs per se, just things you might have been able to do with > > other tools you can't do with jmeter. Like, does jmeter support NTLM > > authentication? (I realize that's a question, not a statement, but some > > other tools have problems with NTLM so I thought I'd throw it out there.) > > > > As to scalability...How many users can you emulate before saturating your > > driver machine given an "average" script (mix of text and graphics, etc)? > > Given something like a 3Ghz P4 with 1M of memory. > > > > I originally did intend to play with jmeter more and lurk on this list much > > longer before asking these types of questions, but, as always, it's faster > > to ask... > > > > J > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
