Good analysis Chamil. Would it be better if we also use a Java profiler like JProfiler, and save the snapshots after each performance comparison? We should do this for both AS and Tomcat. Then, we can easily compare those which would be helpful later on to fix the hotspots in AS.
And, how about performance comparison of few webapps with popular frameworks like Spring MVC, Hibernate etc. We can also include CXF based webapps. Regards, KasunG On Mon, Dec 9, 2013 at 1:19 PM, Chamil Jeewantha <[email protected]> wrote: > > Hi All, > > I am currently woring on $subject. I have done the following work up to > now. Please provide your feedback to improve it further. > > *Target:* > > Quantify the compensation for the features in WS2 Application server. > Understand the improvable areas of WSO2 Application server. > It is obvious there is a performance compensation on AS due to features it > provides on top of Tomcat. This work is *NOT *targeted to say which > product is better. > > *Important Notes:* > > > - WSO2 Application server 5.2.0 (AS)[1] compared with pure Tomcat > 7.0.34 [2] because AS 5.2.0 is powered by Tomcat 7.0.34 > - All the tests done under default deployment configurations of both > servers > - Both test client tool (Apache Benchmark [3]) and servers are running > on the same computer > - The qualities like documentation, user friendliness where not > compared in numbers. > > *Computer Specification:* > > - Lenovo T530 laptop [4] > - Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz (Cores: 2, Threads 4) [5] > - 8GB DDR3 Memory > - Ubuntu 13.10 x64 > > *Methodology* > > The following items have been compared: > > 1. Performance on static web page > - A simple 80 bytes html file is tested. > 2. Performance on Servlet which accesses session info and generate > heavy response > - A Servlet which generates random page of a 500kb and do a read & > write on session is tested. > 3. Quality of documentation > - ***Based on my feeling. not a scientific analysis. :D > 4. TODO: Support availability, cost and quality comparison > - is that already done? > 5. TODO: Feature comparison > - is that already done? > 6. Your suggestions > - Please.... > > The 1, 2 items in the above list is tested using Apache benchmark tool by > sending 10000 concurrent requests in 100 concurrent threads. Three > deployment environments are considered. > > > - Deployment on pure Tomcat > - Deployment on AS as super admin > - Deployment on AS as a tenant > > > *Command :* > > ab -n 10000 -c 100 <url> > > The results of Apache benchmark is used to compare the performance. > > *Result* > > *Performance on static web page* > > Deployed a test application with a simple static html page (80 bytes in > size) on both AS (One as super admin and another as a Tenant) and Tomcat. > > > *Environment* *Tomcat* *AS No Tenant* *AS With Tenant* *Requests/Second > (mean)* 7289.58 7320.57 4080.14 *Time/Request [ms] (mean)* 13.718 13.66 > 24.509 > *Percentage of the requests served within a certain time (ms)* *50.00%* > 10 12 17 *66.00%* 13 15 20 *75.00%* 15 17 23 *80.00%* 17 18 25 > *90.00%* 24 23 34 *95.00%* 34 28 47 *98.00%* 53 38 75 *99.00%* 85 45 > 484 * 100% (longest request)* 160 102 518 > > *Performance on Servlet generating dynamic contents* > > Deployed a test application with a Servlet which generates random page of > a 500kb, do a read/write to/from the session on AS (One as super admin and > another as a Tenant) and Tomcat. > > > *Environment* *Tomcat* *AS No Tenant* *AS With Tenant* *Requests/Second > (mean)* 342.61 257.73 244.97 *Time/Request [ms] (mean)* 291.875 388.003 > 408.219 > *Percentage of the requests served within a certain time (ms)* *50.00%* > 263 320 327 *66.00%* 356 431 446 *75.00%* 412 513 535 *80.00%* 440 579 > 597 *90.00%* 520 765 807 *95.00%* 611 958 990 *98.00%* 749 1261 1369 > *99.00%* 857 1608 2313 * 100% (longest request)* 1489 2139 3321 > > *Result Analysis* > > Mean Time per request for simple static web page running on AS (*No > Tenant*) shows *-0.42%* gap in result compared to Tomcat. > Mean Time per request for simple static web page running on AS (*as a > Tenant*) shows *78.66%* gap in result compared to Tomcat. > > Mean Time per request for the dynamic servlet running on AS (*No Tenant*) > shows *32.93%* gap in result compared to Tomcat. > Mean Time per request for the dynamic servlet running on AS (*as a Tenant*) > shows *39.86%* gap in result compared to Tomcat. > > TODO: Compare an application that works with persistence. > > > *Documentation quality* > > Both AS and Tomcat documentations were easy to understand and completed. > However the "Getting started" section is just finished by starting up the > server. But in the Tomcat documentation, right after the "Setup" topic, > "First web application" link is there. Having an application deployed on > the server gives the feeling of "Completeness" to the documentation > follower. So I suggest to have a link to "Deploying Your Custom Web > Application"[6] at the bottom of the getting started page. > > There are many documents in the WSO2 Library regarding performance tuning > of various other products. but nothing for AS tuning or Application server > production setting recommendation. My suggestion is to have a library > article/documentation link for Application Server performance tuning. > > > References: > > [1] > http://dist.wso2.org/products/application-server/5.2.0/release-notes-as.html > [2] http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.34/RELEASE-NOTES > [3] http://httpd.apache.org/docs/2.2/programs/ab.html > [4] http://shop.lenovo.com/us/en/laptops/thinkpad/t-series/t530/#techspecs > [5] > http://ark.intel.com/products/64893/Intel-Core-i7-3520M-Processor-4M-Cache-up-to-3_60-GHz > [6] > http://docs.wso2.org/display/AS520/Deploying+Your+Custom+Web+Application > > -- > K.D. Chamil Jeewantha > Associate Technical Lead > WSO2, Inc.; http://wso2.com > Mobile: +94716813892 > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Kasun Gajasinghe* Software Engineer; WSO2 Inc.; http://wso2.com , *email: * *kasung AT spamfree wso2.com <http://wso2.com> ** cell: **+94 (77) 678-0813* *linked-in: *http://lk.linkedin.com/in/gajasinghe *blog: **http://kasunbg.org* <http://kasunbg.org> *twitter: **http://twitter.com/kasunbg* <http://twitter.com/kasunbg>
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
