The conclusions that I made based on my tests. Maybe characteristic of tests that I've made is totally different from common jmeter usage case so I describe my tests: - long running tests: 12h-48h, the goal is to check how application behaviour on extremely high and long time load - no any delays between requests - no synchronization between jmeter worker threads - physical delay (calculated by ping command) between jmeter client and tested application below 0.2 ms - only one custom listener optimized to collect millions of requests (samples) and calculate avg, line 90%, line 99%, line 99.9%, throughput , min and max - in a peak application can by used by 10 000 unique users - average application throughput at a level 5 000 req/sec - average response (request) time below 20 ms - line 90% below 50 ms
W dniu 12 września 2009 00:50 użytkownik sebb <[email protected]> napisał: > On 11/09/2009, Jakub Białek <[email protected]> wrote: >> Obviously JMeter supports much more than 30 threads. But if you use >> JMeter to simulate e.g. 200 concurrent users to check load and >> response time of your application then you can use e.g.: >> - 1 jmeter instance with 200 threads or >> - 2 jmeter instances with 100 threads each or >> - 4 jmeter instances with 50 threads each or >> - 8 jmeter instances with 25 threads each >> ...... >> The last solution is the best if you want simulate real load that can >> be generated by 200 real users. The first solution is the worst. I'll >> explain it below. >> Let's assume that you have 1 core CPU. Some facts: >> - at the moment on your cpu can run only one process/thread to >> simulate multitasking OS gives small amount of time to each >> process/thread and changes the process/thread context >> - changing/switching process cost several cpu cycles because cpu >> registers have to be filled with corresponding data >> - changing/switching thread is cheaper >> - not only jmeter runs on your machine also many other >> application/process bounded to the OS, so jmeter is often dispossess >> and waiting for cpu cycles >> - not only workers threads of jmeter runs, also other java threads >> e.g. GCThread, all those threads compete for cpu >> - when one thread/process is running others are sleeping/waiting >> - process/thread can by disposed at any time of execution > > These are all true, but: > >> Conclusion: >> - 200 jmeter threads don't mean 200 concurrent users, load generated >> by those jmeter instances will be much less than this by 200 >> concurrent real users > > That's not a valid conclusion unless JMeter is running out of CPU. On my 8 core machine jmeter instance never have taken more than 150% (from 800% available) cpu, usually it has been below 100%, no matter how many worker threads I've using. Even if the generated load is much below the maximum. So in my test is true that 2 instances with 30 threads generate more throughput/load than 1 instance with 60 threads. I think that the problem is that in jmeter code is many synchronized keyword: synchronized methods, blocks and collections in objects from test plan (used concurrently by all worker threads), so more threads mean more time spent on synchronization and waiting to get a lock. > > In general, a single JMeter thread can generate *more* load than a > single user, because it can send requests much faster than a human. Yes it's true if you based on only several users/threads and you think about throughput. But you have to agree that almost always it easer for application/server to handle request by request (even with no delay between request) than handle at one moment 100 requests from different users and you cannot generate such concurrency by 100 worker threads in one jmeter (ok you can if you run jmeter instance on niagara platform that supports 128/256 concurrent threads ;)). One jmeter instance usually can send request faster than a user but 100 users can send 100 requests in the same moment of time (e.g. 10:00:00:000-10:00:00:001), do you agree? > >> - more jmeter threads more time spent on switching context, less time >> used by jmeter worker threads >> - any worker thread can be disposed (and wait when rest 199 threads >> will take their cpu time) after receiving answer from server but >> before calculate time that call to server took so this time will be >> incorrect >> >> So if your goal is to simulate as good as possible load generated by >> many users then use so much jmeter instances (computers) as you can >> with as less as possible threads on each jmeter instance. Otherwise >> your results can be far from reality and you can be surprised why your >> application can't handle with those users even though jmeter stress >> tests passed. > > Again, I disagree with this. If you agree with fact that process/thread can by disposed at any time of execution why do you don't agree that if you use many threads calculated response time can be incorrect (it will contain not only response time but also time that tread will waiting to get access to cpu)? > >> If someone is interested in comparison how performance of my >> application changes depending on jmeter configuration (number of >> threads and number of instances) I can do that test. I have 10 servers >> with 2 CPU each and each CPU with 4 core. Those servers can be used as >> a JMeter clients. >> >> >> Best regards, >> Jakub >> >> >> W dniu 11 września 2009 15:54 użytkownik sebb <[email protected]> napisał: >> >> > On 11/09/2009, Jakub Białek <[email protected]> wrote: >> >> How about tomcat is it on linux or windows machine? Make sure that OS >> >> doesn't limit the max number of inbound connection. >> >> >> >> Jakub >> >> >> >> BTW. I think that 200 thread on 1 jmeter instance are to many if you >> >> don't have cpus that support 64 threads simultaneously ;) . I usually >> >> use no more than 50 on 8 core system. >> >> >> > >> > A single JMeter instance can support many more than 50 threads, >> > depending on the test plan. >> > >> >> >> >> 2009/9/11 Mytho Logic <[email protected]>: >> >> > Hi, >> >> > >> >> > I tried with 550 threads in tomcat for 500 concurrent users. But not >> >> > succeed. >> >> > >> >> > Jmeter Client:- >> >> > System: Windows XP Professional Verison 2002 SP2 >> >> > Computer: Intel(R) Core(TM)2 CPU 6300 @ 1.86 Ghz 1.86 GHz, 1.97 GB of >> RAM >> >> > >> >> > Jmeter Server:- >> >> > System: Windows XP Professional x64 Edition Verison 2003 SP2 >> >> > Computer: Intel(R) Xeon(R) CPU 5130 @ 2.00 Ghz 2.00 GHz, 8.00 GB of >> RAM >> >> > >> >> > Thanks >> >> > Maumita >> >> > >> >> > >> >> > 2009/9/11 Jakub Białek <[email protected]> >> >> > >> >> >> If you have 500 (truly) concurrent users then you need at least 500 >> >> >> threads in tomcat, try with 550 thread. >> >> >> >> >> >> You run jmeter on windows or linux os? >> >> >> >> >> >> Best regards, >> >> >> Jakub >> >> >> >> >> >> 2009/9/11 Mytho Logic <[email protected]>: >> >> >> > Hi, >> >> >> > >> >> >> > I changed the maxthreads limit in tomcat server.xml to 400 and >> tried the >> >> >> > same testplan again..but few sample requests gets failed and throws >> >> >> > exception. >> >> >> > what should I do to solve the problem. >> >> >> > >> >> >> > Exception:- >> >> >> > >> >> >> > java.net.ConnectException: Connection refused: connect >> >> >> > at java.net.PlainSocketImpl.socketConnect(Native Method) >> >> >> > at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) >> >> >> > at >> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) >> >> >> > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) >> >> >> > at java.net.Socket.connect(Socket.java:519) >> >> >> > at java.net.Socket.connect(Socket.java:469) >> >> >> > at sun.net.NetworkClient.doConnect(NetworkClient.java:157) >> >> >> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:382) >> >> >> > at sun.net.www.http.HttpClient.openServer(HttpClient.java:494) >> >> >> > at sun.net.www.http.HttpClient.<init>(HttpClient.java:231) >> >> >> > at sun.net.www.http.HttpClient.New(HttpClient.java:304) >> >> >> > at sun.net.www.http.HttpClient.New(HttpClient.java:316) >> >> >> > at >> >> >> > >> >> >> >> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:817) >> >> >> > at >> >> >> > >> >> >> >> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:769) >> >> >> > at >> >> >> > >> >> >> >> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:694) >> >> >> > at >> >> >> > >> >> >> >> org.apache.jmeter.protocol.http.sampler.HTTPSampler.sample(HTTPSampler.java:431) >> >> >> > at >> >> >> > >> >> >> >> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1021) >> >> >> > at >> >> >> > >> >> >> >> org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1007) >> >> >> > at >> org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:290) >> >> >> > at java.lang.Thread.run(Thread.java:595) >> >> >> > >> >> >> > Thanks >> >> >> > Maumita >> >> >> > >> >> >> > >> >> >> > On Fri, Sep 11, 2009 at 3:07 PM, sebb <[email protected]> wrote: >> >> >> > >> >> >> >> On 11/09/2009, Mytho Logic <[email protected]> wrote: >> >> >> >> > Hi, >> >> >> >> > >> >> >> >> > In tomcat server.xml, I changed the maxtheads limit to 300 and >> kept >> >> >> >> > JAVA_OPTS=-Xms512m -Xmx1024m >> >> >> >> > >> >> >> >> > But not succeed.I am getting same "java.net.ConnectException: >> >> >> Connection >> >> >> >> > refused: connect" exception. Am I missing anything to change in >> >> >> >> server.xml >> >> >> >> > file. >> >> >> >> > The machine on which tomcat server is running has 2 GB RAM. >> >> >> >> > >> >> >> >> > Please help on this. >> >> >> >> >> >> >> >> This appears to be a Tomcat configuaration issue. >> >> >> >> >> >> >> >> Read the Tomcat documentation carefully, and if that does not >> answer >> >> >> >> your query, then post a question on the Tomcat user list. >> >> >> >> >> >> >> >> > Thanks >> >> >> >> > Maumita >> >> >> >> > >> >> >> >> > >> >> >> >> > On Fri, Sep 11, 2009 at 10:15 AM, Deepak Shetty >> <[email protected]> >> >> >> >> wrote: >> >> >> >> > >> >> >> >> > > Hi >> >> >> >> > > http://tomcat.apache.org/tomcat-5.5-doc/config/http.html >> >> >> >> > > There are three thread related properties in server.xml . >> You seem >> >> >> to >> >> >> >> have >> >> >> >> > > run into maxThreads limit >> >> >> >> > > Note that if your server cant handle the load that is a >> server/OS >> >> >> >> problem , >> >> >> >> > > you are more likely to get help on a tomcat forum. Jmeter is >> >> >> >> generating the >> >> >> >> > > load correctly. >> >> >> >> > > regards >> >> >> >> > > deepak >> >> >> >> > > >> >> >> >> > > On Thu, Sep 10, 2009 at 9:38 PM, Mytho Logic < >> >> >> >> [email protected] >> >> >> >> > > >wrote: >> >> >> >> > > >> >> >> >> > > > Hi, >> >> >> >> > > > >> >> >> >> > > > I am using jmeter version 2.3.2. >> >> >> >> > > > To test my application for 200 concurrent users I have used >> >> >> >> distibuted >> >> >> >> > > > testing set up. >> >> >> >> > > > It works fine for 200 users. >> >> >> >> > > > The test plan was - >> >> >> >> > > > 1. 200 concurrent users will first hit the home page >> >> >> >> > > > 2. After that 200 concurrent users will hit the products >> page >> >> >> >> > > > 3. After that 200 concurrent users will hit the product >> Detail >> >> >> page >> >> >> >> > > > >> >> >> >> > > > For concurrent user setup I have used 'Synchronizing >> Timer' above >> >> >> >> the >> >> >> >> > > > thread >> >> >> >> > > > groups and mentioned 200. >> >> >> >> > > > It works fine and I got the result. >> >> >> >> > > > >> >> >> >> > > > Setup:- >> >> >> >> > > > >> >> >> >> > > > Tomcat server, Jmeter client and Jmeter server all are in 3 >> >> >> >> different >> >> >> >> > > > machine. >> >> >> >> > > > >> >> >> >> > > > Tomcat Server >> >> >> >> > > > ------------- >> >> >> >> > > > JAVA_OPTS=-Xms1024m -Xmx1024m >> >> >> >> > > > >> >> >> >> > > > Jmeter Client >> >> >> >> > > > ------------- >> >> >> >> > > > HEAP=-Xms512m -Xmx1024m >> >> >> >> > > > >> >> >> >> > > > Jmeter Server >> >> >> >> > > > ------------- >> >> >> >> > > > HEAP=-Xms512m -Xmx4096m >> >> >> >> > > > >> >> >> >> > > > Using the same configuration and same setup,when i tried >> for 220 >> >> >> >> > > concurrent >> >> >> >> > > > users for few few sample requests I got >> >> >> >> > > > "java.net.ConnectException: Connection refused: connect: >> >> >> exception. >> >> >> >> > > > Sometime I succeded for 220 concurrent users without any >> >> >> errors.But >> >> >> >> next >> >> >> >> > > > time whne I again run the test plan for >> >> >> >> > > > 220 concurrent users I got exception. >> >> >> >> > > > >> >> >> >> > > > What is the reason behind this exception and how can I >> solve >> >> >> this? >> >> >> >> > > > I need to load my application upto 500 concurrent users. >> Please >> >> >> >> suggesst >> >> >> >> > > > how >> >> >> >> > > > can I do this? >> >> >> >> > > > >> >> >> >> > > > This is an urgent requirement. >> >> >> >> > > > >> >> >> >> > > > Thanks >> >> >> >> > > > Maumita >> >> >> >> > > > >> >> >> >> > > >> >> >> >> > >> >> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> >> To unsubscribe, e-mail: [email protected] >> >> >> >> For additional commands, e-mail: >> [email protected] >> >> >> >> >> >> >> >> >> >> >> > >> >> >> >> >> >> --------------------------------------------------------------------- >> >> >> To unsubscribe, e-mail: [email protected] >> >> >> For additional commands, e-mail: [email protected] >> >> >> >> >> >> >> >> > >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: [email protected] >> >> For additional commands, e-mail: [email protected] >> >> >> >> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: [email protected] >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
