Thanks very much for sharing this information. Is there any one out there who has similar information on struts+WSAD (websphere studio applicaton developer) of IBM? We would very much appreciate if someone can post any performance information on this combination.
Yanhui -----Original Message----- From: TKV Tyler VanGorder [mailto:[EMAIL PROTECTED]] Sent: Tuesday, March 05, 2002 11:01 AM To: '[EMAIL PROTECTED]' Subject: FW: Tomcat/Struts Profiling results Hi, For the last week and a half, a colleague and myself have been doing load/ scalability testing for our struts/jsp-based application. I would like to share our results in the hopes that it may help someone else faced with a similar task. We had a tough time finding real-world examples that we could learn from. This is a LONG post, so I will try to summarize first: ========================================================================== Yeah..the summary is pretty long as well. 8-) For JSP, tag-heavy applications, we found tuning parameters that may help others get better performance out of tomcat. We simulated a load of 50 concurrent users using automation tools that remove any human pauses. This is roughly equivalent to 250+ concurrent, human users. Yes...that is an estimation. Initially, using Tomcat 4.0.1, Stuts 1.0 Final: The IBM JDK performed MUCH MUCH better than Sun's Server Hotspot JVM (1.3.1) After profiling, we discovered that a significant amount of our time was being spent in the garbage collector with the hotspot JVM. Some of our garbage collections took a 12+ seconds! Ouch. We attempted tweaking the various hotspot JVM parameters to see if we could get the hotspot VM in line with IBM's performance. We did get better results but we still had some GCs that were taking 6 seconds. We had a large number of throw-away objects that were confusing the garbage collector in hotspot. We deployed the SAME EXACT application under Weblogic 6.1's servlet engine, using the hotspot JVM, and we saw much better performance and this was without any hotspot tuning parameters! So we switched back to tomcat and we ran the application, using OptimizeIt to profile the application. A majority of our time was being spent in the constructor of BodyContentImpl because of an initial 8K char buffer being created for EACH tag, embedded in our page. This was more pronounced in our application because we iterated over content that had tags nested in the iterator....that's 8K X Number of iterations X number of nested tags! We found that in our application most of our tags were only outputting a few characters. We recompiled the BodyContentImpl and changed the initial buffer size to 512 bytes. The results were dramatic and hotspot began outperforming the IBM JDK. Keep in mind that the hotspot server VM takes a while to tune its runtime so was necessary to run our scalability test on the hotspot VM several times. I submitted an enhancement to bugzilla, its ID is 6858. The final results were impressive! Our application has one workflow that is VERY database intensive. It takes a human user approximately two minutes to complete the workflow once they are familiar with it. The transaction does 20-40 database selects with result sets in the range of 10-40 rows each (on average). 20+ Inserts when the transaction is finally saved. All database activity happens in real time, while the user walks through our workflow. Tomcat 4.0.1 + (our hack), using Hotspot Server JVM 1.3.1: Hardware: EJB server - PII 300 (-Xms64m -Xmx64m) Servlet/web server - PIII Dual 550 (-Xms128m -Xmx128m) Simulated 50 automated users (No human pauses) - Average transaction speed 21 seconds, longest time 27 seconds! Of course this is our application...and each application will have its own load characteristics, but it is a good example of struts + tomcat. So..if our 50 automated users average 21 seconds and a human user takes roughly 2 minutes for the same workflow then the above platform *SHOULD* be able to support 6X the number of human users (300 users) Thanks and I hope this information might be useful to others trying to get a feel for how well JSP/struts will scale. Tyler Van Gorder [EMAIL PROTECTED] Reed Roberts Landacorp. ========================================================================= Details....and the actual different tuning parameters we used: The environments we used are as follows: ================================================ Hardware: We used a low-end machine for the EJB server. Database server: Compaq Proreliant ML370 Dual PIII 800 Windows 2000 Advanced Server Service Pack 2 1Gig Ram EJB Server: PII 300 NT 4.0 Server 128M Ram Servlet Engine/Web Server PIII Dual 550. Windows 2000 Workstation 256M Ram 5 client test machines...ranging in size Win 98 & Win NT machines. All hardware was isolated on the same 100M Switch/hub ================================================ Software Database Server == Oracle 8 EJB Server == Weblogic 6.1.0 Service Pack 2 Servlet Engine 1 == Tomcat 4.0.1 Servlet Engine 2 == Weblogic 6.1.0 Service Pack 2 (We did not have a separate web server...tomcat and weblogic acted as our web server!) ================================================ Load testing software: We used our own, home grown, load testing software. We first set up a proxy between a browser client and the web server. The proxy records all requests/ responses into a script. The script also records human pause times. Once the script has been recorded we have a master server program (written in Java) that can load the script. Finally, on each client test machine we install a script runner. Each runner connects to the master server and waits for the master to send it a job to run. The master can instruct all the connected script runners to do the following: The script runners record times for each request/response and report the metrics back to the master server once the run is complete. ALL of our test simulated 50 concurrent users with NO PAUSES. (5 clients X 10 threads a piece) ================================================================== Our application: We have implemented our business objects as interfaces and have two implementations that we can use: a) We can host the business objects in the servlet engine and allow them to directly talk to the oracle database. b) We can host the business objects in the ejb server and the servlet engine uses an business object implementation that proxies requests to the remote ejb server objects via RMI. All communicated data between the servlet engine and the EJB server are XML strings which are parsed on either end using xerces and SAX. Our presentation layer uses the struts framework (JSPs) and is very TAG heavy (We use jsp tags EVERYWHERE) The application is also very database intensive...the transaction we recorded had 20+ selects, and many inserts and we leverage a connection pool that is JTA aware. ================================================================= The results: -------------------------------------------------------- Tomcat & IBM JDK Business objects hosted in servlet engine -------------------------------------------------------- Concurrent Users : 50 Application server average CPU usage : n/a Servlet engine average CPU usage : 95% average job speed (seconds) : 38 worst job speed (seconds) : 46 -------------------------------------------------------- Tomcat & Sun Hotspot Server JVM 1.3.1 -server -verbosegc -XX:SurvivorRatio=2 -XX:NewSize=80m -XX:MaxNewSize=80m -Xms160m -Xmx160m Business objects hosted in EJB server -------------------------------------------------------- Concurrent Users : 50 Application server average CPU usage : 25% Servlet engine average CPU usage : 79% average job speed (seconds) : 83 worst job speed (seconds) : 130 * Numbers only recorded for first run (didn't give hotspot time to "warm-up" -------------------------------------------------------- Tomcat & Sun Hotspot Server JVM 1.3.1 -server -verbosegc -XX:SurvivorRatio=25 -XX:NewSize=80m -XX:MaxNewSize=80m -Xms160m -Xmx160m Business objects hosted in EJB server -------------------------------------------------------- Concurrent Users : 50 Application server average CPU usage : 30% Servlet engine average CPU usage : 74% average job speed (seconds) : 63 worst job speed (seconds) : 110 * Numbers only recorded for first run (didn't give hotspot time to "warm-up" -------------------------------------------------------- Tomcat & Sun Hotspot Server JVM 1.3.1 -server -verbosegc -XX:SurvivorRatio=25 -XX:NewSize=64m -XX:MaxNewSize=64m -Xms128m -Xmx128m Business objects hosted in EJB server -------------------------------------------------------- Concurrent Users : 50 1st 2nd ---------- Application server average CPU usage : 30% 50% Servlet engine average CPU usage : 81% 78% average job speed (seconds) : 73 31 worst job speed (seconds) : 111 42 * Ran twice to give hotspot a chance to optimize. -------------------------------------------------------- Tomcat & Sun Hotspot Server JVM 1.3.1 -server -verbosegc -XX:MaxLiveObjectEvacuationRatio=33 -XX:SurvivorRatio=25 -XX:NewSize=80m -XX:MaxNewSize=80m -Xms128m -Xmx128m Business objects hosted in EJB server -------------------------------------------------------- Concurrent Users : 50 1st 2nd ---------- Application server average CPU usage : 32% 57% Servlet engine average CPU usage : 78% 85% average job speed (seconds) : 65 31 worst job speed (seconds) : 93 39 * Ran twice to give hotspot a chance to optimize. -------------------------------------------------------- Weblogic (as servlet engine, talking to a remote weblogic EJB server) & Sun Hotspot Server JVM 1.3.1 -server -Xms128m -Xmx128m Business objects hosted in EJB server -------------------------------------------------------- Concurrent Users : 50 1st 2nd ---------- Application server average CPU usage : 70% 56% Servlet engine average CPU usage : 49% 50% average job speed (seconds) : 87 60 worst job speed (seconds) : 110 75 * Ran twice to give hotspot a chance to optimize. -------------------------------------------------------- Tomcat (plus buffer hack) & Sun Hotspot Server JVM 1.3.1 -server -verbosegc -XX:MaxLiveObjectEvacuationRatio=50 -XX:SurvivorRatio=25 -XX:NewSize=64m -XX:MaxNewSize=64m -Xms128m -Xmx128m Business objects hosted in EJB server -------------------------------------------------------- Concurrent Users : 50 1st 2nd ---------- Application server average CPU usage : 50% 70% Servlet engine average CPU usage : 77% 60% average job speed (seconds) : 34 21 worst job speed (seconds) : 47 27 * Ran twice to give hotspot a chance to optimize. -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
