his test is hand-tailored to fail any true component oriented framework. 5000 items per page? really? show me a single website that does that. even facebook doesnt show that many comments on the wall without making you page. but, after looking at other "i hate wicket" posts on his blog i am not surprised with his testing approach :)
if this situation came up in the real life any sane developer would wrap the entire product list into a single component that would stream html directly using string manipulation. and if you do that then wicket will be on par - if not faster - then the other ones. because at that point you are measuring the same thing - how fast the jvm loops. -igor On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst <[email protected]> wrote: > The application used in > http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been > controversial as it pits all popular frameworks against one another. > I've tried to replicate the results on my box with AB, and did some > modifications to the code: > - removed the image from the markup: browsers will hit the > non-existent image causing long load times > - did the categories list in a similar way that the Tapestry > implementation did: using a StringBuilder (saves ~300ms for a single > user with 5000 items) > - randomized the results for each query to the products() method such > that caching the results is not an option > > I couldn't replicate the tapestry results with these settings (instead > of being almost constant time, it now is linear but 1.5x faster than > wicket). I'm not sure if the author actually looked at the results of > the tapestry tests, but it seems to me that something went wrong. > > The code has been altered with the above changes, including changing > the ListView into a RepeatingView, and rendering the categories in a > StringBuilder. Here are the results of an ab test on my laptop for 200 > products and 32 concurrent users: > > dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200 > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 1024 requests > Completed 2048 requests > Completed 3072 requests > Completed 4096 requests > Completed 5120 requests > Completed 6144 requests > Completed 7168 requests > Completed 8192 requests > Completed 9216 requests > Completed 10240 requests > Finished 10240 requests > > > Server Software: Jetty(6.1.16) > Server Hostname: localhost > Server Port: 8080 > > Document Path: /wicketapp/products3/200 > Document Length: 34689 bytes > > Concurrency Level: 32 > Time taken for tests: 57.509 seconds > Complete requests: 10240 > Failed requests: 9670 > (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0) > Write errors: 0 > Total transferred: 354991304 bytes > HTML transferred: 352820424 bytes > Requests per second: 178.06 [#/sec] (mean) > Time per request: 179.714 [ms] (mean) > Time per request: 5.616 [ms] (mean, across all concurrent requests) > Transfer rate: 6028.16 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 1 1.6 0 17 > Processing: 11 179 123.2 163 1000 > Waiting: 11 171 123.1 154 999 > Total: 12 179 123.3 164 1000 > > Percentage of the requests served within a certain time (ms) > 50% 164 > 66% 218 > 75% 248 > 80% 268 > 90% 337 > 95% 418 > 98% 491 > 99% 552 > 100% 1000 (longest request) > > The same test but then using 16 users: > > dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200 > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 1024 requests > Completed 2048 requests > Completed 3072 requests > Completed 4096 requests > Completed 5120 requests > Completed 6144 requests > Completed 7168 requests > Completed 8192 requests > Completed 9216 requests > Completed 10240 requests > Finished 10240 requests > > > Server Software: Jetty(6.1.16) > Server Hostname: localhost > Server Port: 8080 > > Document Path: /wicketapp/products3/200 > Document Length: 34290 bytes > > Concurrency Level: 16 > Time taken for tests: 54.005 seconds > Complete requests: 10240 > Failed requests: 6161 > (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0) > Write errors: 0 > Total transferred: 355066608 bytes > HTML transferred: 352895728 bytes > Requests per second: 189.61 [#/sec] (mean) > Time per request: 84.383 [ms] (mean) > Time per request: 5.274 [ms] (mean, across all concurrent requests) > Transfer rate: 6420.58 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 1.0 0 16 > Processing: 11 84 51.2 74 397 > Waiting: 8 77 49.4 66 397 > Total: 11 84 51.2 75 398 > > Percentage of the requests served within a certain time (ms) > 50% 75 > 66% 97 > 75% 113 > 80% 124 > 90% 154 > 95% 182 > 98% 217 > 99% 239 > 100% 398 (longest request) > > Now with 8 users: > > dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200 > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 1024 requests > Completed 2048 requests > Completed 3072 requests > Completed 4096 requests > Completed 5120 requests > Completed 6144 requests > Completed 7168 requests > Completed 8192 requests > Completed 9216 requests > Completed 10240 requests > Finished 10240 requests > > > Server Software: Jetty(6.1.16) > Server Hostname: localhost > Server Port: 8080 > > Document Path: /wicketapp/products3/200 > Document Length: 34290 bytes > > Concurrency Level: 8 > Time taken for tests: 48.672 seconds > Complete requests: 10240 > Failed requests: 6308 > (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0) > Write errors: 0 > Total transferred: 355046233 bytes > HTML transferred: 352875353 bytes > Requests per second: 210.39 [#/sec] (mean) > Time per request: 38.025 [ms] (mean) > Time per request: 4.753 [ms] (mean, across all concurrent requests) > Transfer rate: 7123.72 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.9 0 10 > Processing: 11 38 19.9 33 203 > Waiting: 9 35 18.6 30 203 > Total: 11 38 20.0 33 204 > > Percentage of the requests served within a certain time (ms) > 50% 33 > 66% 41 > 75% 47 > 80% 53 > 90% 65 > 95% 77 > 98% 93 > 99% 105 > 100% 204 (longest request) > > And with 4 users: > > dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200 > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 1024 requests > Completed 2048 requests > Completed 3072 requests > Completed 4096 requests > Completed 5120 requests > Completed 6144 requests > Completed 7168 requests > Completed 8192 requests > Completed 9216 requests > Completed 10240 requests > Finished 10240 requests > > > Server Software: Jetty(6.1.16) > Server Hostname: localhost > Server Port: 8080 > > Document Path: /wicketapp/products3/200 > Document Length: 34290 bytes > > Concurrency Level: 4 > Time taken for tests: 45.238 seconds > Complete requests: 10240 > Failed requests: 6134 > (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0) > Write errors: 0 > Total transferred: 355063755 bytes > HTML transferred: 352892875 bytes > Requests per second: 226.36 [#/sec] (mean) > Time per request: 17.671 [ms] (mean) > Time per request: 4.418 [ms] (mean, across all concurrent requests) > Transfer rate: 7664.91 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.2 0 7 > Processing: 11 17 2.4 18 43 > Waiting: 10 17 2.5 17 41 > Total: 11 18 2.4 18 43 > > Percentage of the requests served within a certain time (ms) > 50% 18 > 66% 18 > 75% 19 > 80% 19 > 90% 20 > 95% 21 > 98% 23 > 99% 25 > 100% 43 (longest request) > > And finally with 2 users: > > dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200 > This is ApacheBench, Version 2.3 <$Revision: 655654 $> > Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ > Licensed to The Apache Software Foundation, http://www.apache.org/ > > Benchmarking localhost (be patient) > Completed 1024 requests > Completed 2048 requests > Completed 3072 requests > Completed 4096 requests > Completed 5120 requests > Completed 6144 requests > Completed 7168 requests > Completed 8192 requests > Completed 9216 requests > Completed 10240 requests > Finished 10240 requests > > > Server Software: Jetty(6.1.16) > Server Hostname: localhost > Server Port: 8080 > > Document Path: /wicketapp/products3/200 > Document Length: 34691 bytes > > Concurrency Level: 2 > Time taken for tests: 55.005 seconds > Complete requests: 10240 > Failed requests: 7532 > (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0) > Write errors: 0 > Total transferred: 355060137 bytes > HTML transferred: 352889257 bytes > Requests per second: 186.16 [#/sec] (mean) > Time per request: 10.743 [ms] (mean) > Time per request: 5.372 [ms] (mean, across all concurrent requests) > Transfer rate: 6303.74 [Kbytes/sec] received > > Connection Times (ms) > min mean[+/-sd] median max > Connect: 0 0 0.0 0 1 > Processing: 6 11 2.8 11 25 > Waiting: 6 10 2.8 10 22 > Total: 7 11 2.8 11 25 > > Percentage of the requests served within a certain time (ms) > 50% 11 > 66% 12 > 75% 13 > 80% 14 > 90% 15 > 95% 15 > 98% 16 > 99% 16 > 100% 25 (longest request) > > Looking at the results I think it is safe to say that with 16 users we > hit the limit of my core i7, and with 4 users we hit the sweet spot > (which is probably correct since I have 2 cores with each 2 > hyperthreads). > > I expected 4 users to perform similar to 2 users, but it appears to be > slower. I'll run it using yourkit to see if there's a blocker > somewhere. > > In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4. > > Notes from this test: > - we report wrong length in several cases > - exceptions occur when a request is broken off > - performance seems on par with 1.4 > > Martijn >
