The application used in http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been controversial as it pits all popular frameworks against one another. I've tried to replicate the results on my box with AB, and did some modifications to the code: - removed the image from the markup: browsers will hit the non-existent image causing long load times - did the categories list in a similar way that the Tapestry implementation did: using a StringBuilder (saves ~300ms for a single user with 5000 items) - randomized the results for each query to the products() method such that caching the results is not an option
I couldn't replicate the tapestry results with these settings (instead of being almost constant time, it now is linear but 1.5x faster than wicket). I'm not sure if the author actually looked at the results of the tapestry tests, but it seems to me that something went wrong. The code has been altered with the above changes, including changing the ListView into a RepeatingView, and rendering the categories in a StringBuilder. Here are the results of an ab test on my laptop for 200 products and 32 concurrent users: dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 1024 requests Completed 2048 requests Completed 3072 requests Completed 4096 requests Completed 5120 requests Completed 6144 requests Completed 7168 requests Completed 8192 requests Completed 9216 requests Completed 10240 requests Finished 10240 requests Server Software: Jetty(6.1.16) Server Hostname: localhost Server Port: 8080 Document Path: /wicketapp/products3/200 Document Length: 34689 bytes Concurrency Level: 32 Time taken for tests: 57.509 seconds Complete requests: 10240 Failed requests: 9670 (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0) Write errors: 0 Total transferred: 354991304 bytes HTML transferred: 352820424 bytes Requests per second: 178.06 [#/sec] (mean) Time per request: 179.714 [ms] (mean) Time per request: 5.616 [ms] (mean, across all concurrent requests) Transfer rate: 6028.16 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 1.6 0 17 Processing: 11 179 123.2 163 1000 Waiting: 11 171 123.1 154 999 Total: 12 179 123.3 164 1000 Percentage of the requests served within a certain time (ms) 50% 164 66% 218 75% 248 80% 268 90% 337 95% 418 98% 491 99% 552 100% 1000 (longest request) The same test but then using 16 users: dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 1024 requests Completed 2048 requests Completed 3072 requests Completed 4096 requests Completed 5120 requests Completed 6144 requests Completed 7168 requests Completed 8192 requests Completed 9216 requests Completed 10240 requests Finished 10240 requests Server Software: Jetty(6.1.16) Server Hostname: localhost Server Port: 8080 Document Path: /wicketapp/products3/200 Document Length: 34290 bytes Concurrency Level: 16 Time taken for tests: 54.005 seconds Complete requests: 10240 Failed requests: 6161 (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0) Write errors: 0 Total transferred: 355066608 bytes HTML transferred: 352895728 bytes Requests per second: 189.61 [#/sec] (mean) Time per request: 84.383 [ms] (mean) Time per request: 5.274 [ms] (mean, across all concurrent requests) Transfer rate: 6420.58 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.0 0 16 Processing: 11 84 51.2 74 397 Waiting: 8 77 49.4 66 397 Total: 11 84 51.2 75 398 Percentage of the requests served within a certain time (ms) 50% 75 66% 97 75% 113 80% 124 90% 154 95% 182 98% 217 99% 239 100% 398 (longest request) Now with 8 users: dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 1024 requests Completed 2048 requests Completed 3072 requests Completed 4096 requests Completed 5120 requests Completed 6144 requests Completed 7168 requests Completed 8192 requests Completed 9216 requests Completed 10240 requests Finished 10240 requests Server Software: Jetty(6.1.16) Server Hostname: localhost Server Port: 8080 Document Path: /wicketapp/products3/200 Document Length: 34290 bytes Concurrency Level: 8 Time taken for tests: 48.672 seconds Complete requests: 10240 Failed requests: 6308 (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0) Write errors: 0 Total transferred: 355046233 bytes HTML transferred: 352875353 bytes Requests per second: 210.39 [#/sec] (mean) Time per request: 38.025 [ms] (mean) Time per request: 4.753 [ms] (mean, across all concurrent requests) Transfer rate: 7123.72 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.9 0 10 Processing: 11 38 19.9 33 203 Waiting: 9 35 18.6 30 203 Total: 11 38 20.0 33 204 Percentage of the requests served within a certain time (ms) 50% 33 66% 41 75% 47 80% 53 90% 65 95% 77 98% 93 99% 105 100% 204 (longest request) And with 4 users: dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 1024 requests Completed 2048 requests Completed 3072 requests Completed 4096 requests Completed 5120 requests Completed 6144 requests Completed 7168 requests Completed 8192 requests Completed 9216 requests Completed 10240 requests Finished 10240 requests Server Software: Jetty(6.1.16) Server Hostname: localhost Server Port: 8080 Document Path: /wicketapp/products3/200 Document Length: 34290 bytes Concurrency Level: 4 Time taken for tests: 45.238 seconds Complete requests: 10240 Failed requests: 6134 (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0) Write errors: 0 Total transferred: 355063755 bytes HTML transferred: 352892875 bytes Requests per second: 226.36 [#/sec] (mean) Time per request: 17.671 [ms] (mean) Time per request: 4.418 [ms] (mean, across all concurrent requests) Transfer rate: 7664.91 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.2 0 7 Processing: 11 17 2.4 18 43 Waiting: 10 17 2.5 17 41 Total: 11 18 2.4 18 43 Percentage of the requests served within a certain time (ms) 50% 18 66% 18 75% 19 80% 19 90% 20 95% 21 98% 23 99% 25 100% 43 (longest request) And finally with 2 users: dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 1024 requests Completed 2048 requests Completed 3072 requests Completed 4096 requests Completed 5120 requests Completed 6144 requests Completed 7168 requests Completed 8192 requests Completed 9216 requests Completed 10240 requests Finished 10240 requests Server Software: Jetty(6.1.16) Server Hostname: localhost Server Port: 8080 Document Path: /wicketapp/products3/200 Document Length: 34691 bytes Concurrency Level: 2 Time taken for tests: 55.005 seconds Complete requests: 10240 Failed requests: 7532 (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0) Write errors: 0 Total transferred: 355060137 bytes HTML transferred: 352889257 bytes Requests per second: 186.16 [#/sec] (mean) Time per request: 10.743 [ms] (mean) Time per request: 5.372 [ms] (mean, across all concurrent requests) Transfer rate: 6303.74 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.0 0 1 Processing: 6 11 2.8 11 25 Waiting: 6 10 2.8 10 22 Total: 7 11 2.8 11 25 Percentage of the requests served within a certain time (ms) 50% 11 66% 12 75% 13 80% 14 90% 15 95% 15 98% 16 99% 16 100% 25 (longest request) Looking at the results I think it is safe to say that with 16 users we hit the limit of my core i7, and with 4 users we hit the sweet spot (which is probably correct since I have 2 cores with each 2 hyperthreads). I expected 4 users to perform similar to 2 users, but it appears to be slower. I'll run it using yourkit to see if there's a blocker somewhere. In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4. Notes from this test: - we report wrong length in several cases - exceptions occur when a request is broken off - performance seems on par with 1.4 Martijn
