I should mention that in my case, I am brutal. I use ramp up, but I don't use timers. I tend to shoot for worse case scenario to expose flaws, weaknesses and memory leaks in the dev and staging environment.
I usually try to shoot for better performance than the requirements. If my stress tests handle the worse case test, I usually feel confortable standing by the performance of the application. If it just barely meets the requirements, I usually state a lower number for sanity sake. I might be paranoid. peter On Thu, 15 Jul 2004 13:51:57 -0400, Michael Stover <[EMAIL PROTECTED]> wrote: > If you are on 3Ghz p4 with 1G, I would expect you can run more than 100 > users. I've run up to 500 with reasonable success (you can't trust the > absolute timings on it, but it doesn't chug badly or crash). > > However, it's all highly dependent on the test details. A test that > parses responses for embedded resources is going to kill JMeter's > performance a lot faster than one that doesn't. Regular expression > parsers, assertions also add significant processing time for JMeter. A > complex test might have 100 users as a max on that machine. A simple > test might allow up to 500 users. > > It also depends on how fast the server is - a faster server puts more > stress on JMeter because each thread spends less time blocked on IO and > more time preparing the next request. > > And, of course, timers. A test without timers is brutal for both > sides. A few timers increases dramatically the number of users that can > effectively be simulated. > > I think the biggest lingering limitation of JMeter is that the remote > testing feature was not adequately made, and does not in fact get you > anything in terms of scaling tests up. > > -Mike > > > > On Thu, 2004-07-15 at 13:05, joelsherriff wrote: > > Agree - my background is in writing commercial test tools - was team lead > > for Compuware on the web team for QALoad, before that with > > Rational/Pure/Performix (way way back when). So the big tests 1000+ users > > are what I'm used to dealing with. Right now I've been asked to evaluate > > jmeter against the commercial tools and scalability is always the killer - > > esp when you look at the tools that drive the server by driving instances of > > a browser (yuck!). I'm really liking jmeter - much better than I expected > > it to be. I expected to find lot's of holes and limitations (the arrogance > > of the old-timer) but really haven't seen any that aren't there for pretty > > good reason - like the proxy/ssl issue. > > > > To distill down something you said, and ignoring the network limitations > > (since that's such a variable): so in your experience you get about 100 > > threads per machine before it saturates it? Is it typically CPU bound or > > memory or...? > > > > J > > > > ----- Original Message ----- > > From: "Peter Lin" <[EMAIL PROTECTED]> > > To: "JMeter Users List" <[EMAIL PROTECTED]> > > Sent: Thursday, July 15, 2004 11:43 AM > > Subject: Re: Any other limitations of jmeter? How about scalability? > > > > > > > for general performance testing stuff, I have several performance > > > articles listed on the links page > > > http://wiki.apache.org/jakarta-jmeter/JMeterLinks. > > > > > > If I compare JMeter to other tools I've used, I would say it is > > > comparable. The tools I've used: > > > > > > 1. apache ab > > > 2. zdnet Webbench > > > 3. custom script perl > > > > > > The biggest challenge of stress testing is this, "what are you > > > testing?" Most of the time, the bottleneck for a website is the > > > connection to the internet. I have a explanation of it in the > > > articles. It doesn't really matter if your website can handle 80 > > > concurrent requests reliably, if you're server is on a T1. I would say > > > JMeter can easily produce enough stress to figure out what the upper > > > limit is for your web application. The situations where it won't be > > > suitable is if you're testing a system that has to support 5,000 > > > concurrent users. You still could use JMeter if you setup each > > > instance with 100 threads and use 50 machines. > > > > > > realistically, to do that kind of test, the server would have to be a > > > big Unix box or mainframe hooked up to several gigabit routers. Using > > > 100mbit ethernet, you wouldn't be able to test 5K concurrent requests > > > for a sustained period of time effectively. Sustained transfer rate > > > plays a huge role in how many concurrent requests a system can handle > > > reliably for a long time. > > > > > > peter > > > > > > > > > > > > On Thu, 15 Jul 2004 11:25:50 -0400, joelsherriff > > > <[EMAIL PROTECTED]> wrote: > > > > After Peter pointed out that the jmeter proxy doesn't do ssl, I thought > > I'd > > > > ask everyone if they know of other limitations they've run into and > > about > > > > what seems to be the cronic limitation of these tools - scalability. > > Not > > > > talking about bugs per se, just things you might have been able to do > > with > > > > other tools you can't do with jmeter. Like, does jmeter support NTLM > > > > authentication? (I realize that's a question, not a statement, but some > > > > other tools have problems with NTLM so I thought I'd throw it out > > there.) > > > > > > > > As to scalability...How many users can you emulate before saturating > > your > > > > driver machine given an "average" script (mix of text and graphics, > > etc)? > > > > Given something like a 3Ghz P4 with 1M of memory. > > > > > > > > I originally did intend to play with jmeter more and lurk on this list > > much > > > > longer before asking these types of questions, but, as always, it's > > faster > > > > to ask... > > > > > > > > J > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > -- > Michael Stover <[EMAIL PROTECTED]> > Apache Software Foundation > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
