hehe I tend to think in terms of page views per X time. Usually, I look at the nightly logs. If I get 8million page views per day, I try to calculate the average. Then I look at the peak traffic to figure out the peak concurrent requests for that day/week/month.
Based on that, I design my test plan so it simulates the desired number of concurrent requests. Once the test is done, I process the accesslogs to see if it actually produced the rate I was shooting for. I've never actually tried to figure X threads with timer == X threads w/o timers. for me the aggregate performance is more important. peter On Thu, 15 Jul 2004 14:07:08 -0400, Michael Stover <[EMAIL PROTECTED]> wrote: > I have often wondered if removing timers is an efficient way of > simulating more users than otherwise possible. Ie, if you remove > timers, 4 threads is equivalent to X threads delayed 2 seconds between > requests. In terms of load generated on the server. The elimination of > extra threads probably helps reduce some of the context switching and > memory use. > > Then again, no timers might increasing context switching, so who knows? > > -Mike > > > > On Thu, 2004-07-15 at 13:49, Peter Lin wrote: > > I should mention that in my case, I am brutal. I use ramp up, but I > > don't use timers. I tend to shoot for worse case scenario to expose > > flaws, weaknesses and memory leaks in the dev and staging environment. > > > > I usually try to shoot for better performance than the requirements. > > If my stress tests handle the worse case test, I usually feel > > confortable standing by the performance of the application. If it just > > barely meets the requirements, I usually state a lower number for > > sanity sake. > > > > I might be paranoid. > > > > peter > > > > > > On Thu, 15 Jul 2004 13:51:57 -0400, Michael Stover <[EMAIL PROTECTED]> wrote: > > > If you are on 3Ghz p4 with 1G, I would expect you can run more than 100 > > > users. I've run up to 500 with reasonable success (you can't trust the > > > absolute timings on it, but it doesn't chug badly or crash). > > > > > > However, it's all highly dependent on the test details. A test that > > > parses responses for embedded resources is going to kill JMeter's > > > performance a lot faster than one that doesn't. Regular expression > > > parsers, assertions also add significant processing time for JMeter. A > > > complex test might have 100 users as a max on that machine. A simple > > > test might allow up to 500 users. > > > > > > It also depends on how fast the server is - a faster server puts more > > > stress on JMeter because each thread spends less time blocked on IO and > > > more time preparing the next request. > > > > > > And, of course, timers. A test without timers is brutal for both > > > sides. A few timers increases dramatically the number of users that can > > > effectively be simulated. > > > > > > I think the biggest lingering limitation of JMeter is that the remote > > > testing feature was not adequately made, and does not in fact get you > > > anything in terms of scaling tests up. > > > > > > -Mike > > > > > > > > > > > > On Thu, 2004-07-15 at 13:05, joelsherriff wrote: > > > > Agree - my background is in writing commercial test tools - was team lead > > > > for Compuware on the web team for QALoad, before that with > > > > Rational/Pure/Performix (way way back when). So the big tests 1000+ users > > > > are what I'm used to dealing with. Right now I've been asked to evaluate > > > > jmeter against the commercial tools and scalability is always the killer - > > > > esp when you look at the tools that drive the server by driving instances of > > > > a browser (yuck!). I'm really liking jmeter - much better than I expected > > > > it to be. I expected to find lot's of holes and limitations (the arrogance > > > > of the old-timer) but really haven't seen any that aren't there for pretty > > > > good reason - like the proxy/ssl issue. > > > > > > > > To distill down something you said, and ignoring the network limitations > > > > (since that's such a variable): so in your experience you get about 100 > > > > threads per machine before it saturates it? Is it typically CPU bound or > > > > memory or...? > > > > > > > > J > > > > > > > > ----- Original Message ----- > > > > From: "Peter Lin" <[EMAIL PROTECTED]> > > > > To: "JMeter Users List" <[EMAIL PROTECTED]> > > > > Sent: Thursday, July 15, 2004 11:43 AM > > > > Subject: Re: Any other limitations of jmeter? How about scalability? > > > > > > > > > > > > > for general performance testing stuff, I have several performance > > > > > articles listed on the links page > > > > > http://wiki.apache.org/jakarta-jmeter/JMeterLinks. > > > > > > > > > > If I compare JMeter to other tools I've used, I would say it is > > > > > comparable. The tools I've used: > > > > > > > > > > 1. apache ab > > > > > 2. zdnet Webbench > > > > > 3. custom script perl > > > > > > > > > > The biggest challenge of stress testing is this, "what are you > > > > > testing?" Most of the time, the bottleneck for a website is the > > > > > connection to the internet. I have a explanation of it in the > > > > > articles. It doesn't really matter if your website can handle 80 > > > > > concurrent requests reliably, if you're server is on a T1. I would say > > > > > JMeter can easily produce enough stress to figure out what the upper > > > > > limit is for your web application. The situations where it won't be > > > > > suitable is if you're testing a system that has to support 5,000 > > > > > concurrent users. You still could use JMeter if you setup each > > > > > instance with 100 threads and use 50 machines. > > > > > > > > > > realistically, to do that kind of test, the server would have to be a > > > > > big Unix box or mainframe hooked up to several gigabit routers. Using > > > > > 100mbit ethernet, you wouldn't be able to test 5K concurrent requests > > > > > for a sustained period of time effectively. Sustained transfer rate > > > > > plays a huge role in how many concurrent requests a system can handle > > > > > reliably for a long time. > > > > > > > > > > peter > > > > > > > > > > > > > > > > > > > > On Thu, 15 Jul 2004 11:25:50 -0400, joelsherriff > > > > > <[EMAIL PROTECTED]> wrote: > > > > > > After Peter pointed out that the jmeter proxy doesn't do ssl, I thought > > > > I'd > > > > > > ask everyone if they know of other limitations they've run into and > > > > about > > > > > > what seems to be the cronic limitation of these tools - scalability. > > > > Not > > > > > > talking about bugs per se, just things you might have been able to do > > > > with > > > > > > other tools you can't do with jmeter. Like, does jmeter support NTLM > > > > > > authentication? (I realize that's a question, not a statement, but some > > > > > > other tools have problems with NTLM so I thought I'd throw it out > > > > there.) > > > > > > > > > > > > As to scalability...How many users can you emulate before saturating > > > > your > > > > > > driver machine given an "average" script (mix of text and graphics, > > > > etc)? > > > > > > Given something like a 3Ghz P4 with 1M of memory. > > > > > > > > > > > > I originally did intend to play with jmeter more and lurk on this list > > > > much > > > > > > longer before asking these types of questions, but, as always, it's > > > > faster > > > > > > to ask... > > > > > > > > > > > > J > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- > > > Michael Stover <[EMAIL PROTECTED]> > > > Apache Software Foundation > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > -- > Michael Stover <[EMAIL PROTECTED]> > Apache Software Foundation > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
