Hi All,
I am currently woring on $subject. I have done the following work up to
now. Please provide your feedback to improve it further.
*Target:*
Quantify the compensation for the features in WS2 Application server.
Understand the improvable areas of WSO2 Application server.
It is obvious there is a performance compensation on AS due to features it
provides on top of Tomcat. This work is *NOT *targeted to say which product
is better.
*Important Notes:*
- WSO2 Application server 5.2.0 (AS)[1] compared with pure Tomcat 7.0.34
[2] because AS 5.2.0 is powered by Tomcat 7.0.34
- All the tests done under default deployment configurations of both
servers
- Both test client tool (Apache Benchmark [3]) and servers are running
on the same computer
- The qualities like documentation, user friendliness where not compared
in numbers.
*Computer Specification:*
- Lenovo T530 laptop [4]
- Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz (Cores: 2, Threads 4) [5]
- 8GB DDR3 Memory
- Ubuntu 13.10 x64
*Methodology*
The following items have been compared:
1. Performance on static web page
- A simple 80 bytes html file is tested.
2. Performance on Servlet which accesses session info and generate heavy
response
- A Servlet which generates random page of a 500kb and do a read &
write on session is tested.
3. Quality of documentation
- ***Based on my feeling. not a scientific analysis. :D
4. TODO: Support availability, cost and quality comparison
- is that already done?
5. TODO: Feature comparison
- is that already done?
6. Your suggestions
- Please....
The 1, 2 items in the above list is tested using Apache benchmark tool by
sending 10000 concurrent requests in 100 concurrent threads. Three
deployment environments are considered.
- Deployment on pure Tomcat
- Deployment on AS as super admin
- Deployment on AS as a tenant
*Command :*
ab -n 10000 -c 100 <url>
The results of Apache benchmark is used to compare the performance.
*Result*
*Performance on static web page*
Deployed a test application with a simple static html page (80 bytes in
size) on both AS (One as super admin and another as a Tenant) and Tomcat.
*Environment* *Tomcat* *AS No Tenant* *AS With Tenant* *Requests/Second
(mean)* 7289.58 7320.57 4080.14 *Time/Request [ms] (mean)* 13.718 13.66
24.509
*Percentage of the requests served within a certain time (ms)* *50.00%* 10
12 17 *66.00%* 13 15 20 *75.00%* 15 17 23 *80.00%* 17 18 25 *90.00%* 24
23 34 *95.00%* 34 28 47 *98.00%* 53 38 75 *99.00%* 85 45 484 * 100%
(longest request)* 160 102 518
*Performance on Servlet generating dynamic contents*
Deployed a test application with a Servlet which generates random page of a
500kb, do a read/write to/from the session on AS (One as super admin and
another as a Tenant) and Tomcat.
*Environment* *Tomcat* *AS No Tenant* *AS With Tenant* *Requests/Second
(mean)* 342.61 257.73 244.97 *Time/Request [ms] (mean)* 291.875 388.003
408.219
*Percentage of the requests served within a certain time (ms)* *50.00%*
263 320 327 *66.00%* 356 431 446 *75.00%* 412 513 535 *80.00%* 440 579
597 *90.00%* 520 765 807 *95.00%* 611 958 990 *98.00%* 749 1261 1369
*99.00%* 857 1608 2313 * 100% (longest request)* 1489 2139 3321
*Result Analysis*
Mean Time per request for simple static web page running on AS (*No Tenant*)
shows *-0.42%* gap in result compared to Tomcat.
Mean Time per request for simple static web page running on AS (*as a
Tenant*) shows *78.66%* gap in result compared to Tomcat.
Mean Time per request for the dynamic servlet running on AS (*No Tenant*)
shows *32.93%* gap in result compared to Tomcat.
Mean Time per request for the dynamic servlet running on AS (*as a Tenant*)
shows *39.86%* gap in result compared to Tomcat.
TODO: Compare an application that works with persistence.
*Documentation quality*
Both AS and Tomcat documentations were easy to understand and completed.
However the "Getting started" section is just finished by starting up the
server. But in the Tomcat documentation, right after the "Setup" topic,
"First web application" link is there. Having an application deployed on
the server gives the feeling of "Completeness" to the documentation
follower. So I suggest to have a link to "Deploying Your Custom Web
Application"[6] at the bottom of the getting started page.
There are many documents in the WSO2 Library regarding performance tuning
of various other products. but nothing for AS tuning or Application server
production setting recommendation. My suggestion is to have a library
article/documentation link for Application Server performance tuning.
References:
[1]
http://dist.wso2.org/products/application-server/5.2.0/release-notes-as.html
[2] http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.34/RELEASE-NOTES
[3] http://httpd.apache.org/docs/2.2/programs/ab.html
[4] http://shop.lenovo.com/us/en/laptops/thinkpad/t-series/t530/#techspecs
[5]
http://ark.intel.com/products/64893/Intel-Core-i7-3520M-Processor-4M-Cache-up-to-3_60-GHz
[6] http://docs.wso2.org/display/AS520/Deploying+Your+Custom+Web+Application
--
K.D. Chamil Jeewantha
Associate Technical Lead
WSO2, Inc.; http://wso2.com
Mobile: +94716813892
_______________________________________________
Architecture mailing list
[email protected]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
