Perhaps we should re-write it using wicket-velocity! ;)
On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <[email protected]> wrote: > his test is hand-tailored to fail any true component oriented > framework. 5000 items per page? really? show me a single website that > does that. even facebook doesnt show that many comments on the wall > without making you page. but, after looking at other "i hate wicket" > posts on his blog i am not surprised with his testing approach :) > > if this situation came up in the real life any sane developer would > wrap the entire product list into a single component that would stream > html directly using string manipulation. and if you do that then > wicket will be on par - if not faster - then the other ones. because > at that point you are measuring the same thing - how fast the jvm > loops. > > -igor > > > On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst > <[email protected]> wrote: >> The application used in >> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been >> controversial as it pits all popular frameworks against one another. >> I've tried to replicate the results on my box with AB, and did some >> modifications to the code: >> - removed the image from the markup: browsers will hit the >> non-existent image causing long load times >> - did the categories list in a similar way that the Tapestry >> implementation did: using a StringBuilder (saves ~300ms for a single >> user with 5000 items) >> - randomized the results for each query to the products() method such >> that caching the results is not an option >> >> I couldn't replicate the tapestry results with these settings (instead >> of being almost constant time, it now is linear but 1.5x faster than >> wicket). I'm not sure if the author actually looked at the results of >> the tapestry tests, but it seems to me that something went wrong. >> >> The code has been altered with the above changes, including changing >> the ListView into a RepeatingView, and rendering the categories in a >> StringBuilder. Here are the results of an ab test on my laptop for 200 >> products and 32 concurrent users: >> >> dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200 >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking localhost (be patient) >> Completed 1024 requests >> Completed 2048 requests >> Completed 3072 requests >> Completed 4096 requests >> Completed 5120 requests >> Completed 6144 requests >> Completed 7168 requests >> Completed 8192 requests >> Completed 9216 requests >> Completed 10240 requests >> Finished 10240 requests >> >> >> Server Software: Jetty(6.1.16) >> Server Hostname: localhost >> Server Port: 8080 >> >> Document Path: /wicketapp/products3/200 >> Document Length: 34689 bytes >> >> Concurrency Level: 32 >> Time taken for tests: 57.509 seconds >> Complete requests: 10240 >> Failed requests: 9670 >> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0) >> Write errors: 0 >> Total transferred: 354991304 bytes >> HTML transferred: 352820424 bytes >> Requests per second: 178.06 [#/sec] (mean) >> Time per request: 179.714 [ms] (mean) >> Time per request: 5.616 [ms] (mean, across all concurrent requests) >> Transfer rate: 6028.16 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 1 1.6 0 17 >> Processing: 11 179 123.2 163 1000 >> Waiting: 11 171 123.1 154 999 >> Total: 12 179 123.3 164 1000 >> >> Percentage of the requests served within a certain time (ms) >> 50% 164 >> 66% 218 >> 75% 248 >> 80% 268 >> 90% 337 >> 95% 418 >> 98% 491 >> 99% 552 >> 100% 1000 (longest request) >> >> The same test but then using 16 users: >> >> dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200 >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking localhost (be patient) >> Completed 1024 requests >> Completed 2048 requests >> Completed 3072 requests >> Completed 4096 requests >> Completed 5120 requests >> Completed 6144 requests >> Completed 7168 requests >> Completed 8192 requests >> Completed 9216 requests >> Completed 10240 requests >> Finished 10240 requests >> >> >> Server Software: Jetty(6.1.16) >> Server Hostname: localhost >> Server Port: 8080 >> >> Document Path: /wicketapp/products3/200 >> Document Length: 34290 bytes >> >> Concurrency Level: 16 >> Time taken for tests: 54.005 seconds >> Complete requests: 10240 >> Failed requests: 6161 >> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0) >> Write errors: 0 >> Total transferred: 355066608 bytes >> HTML transferred: 352895728 bytes >> Requests per second: 189.61 [#/sec] (mean) >> Time per request: 84.383 [ms] (mean) >> Time per request: 5.274 [ms] (mean, across all concurrent requests) >> Transfer rate: 6420.58 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 1.0 0 16 >> Processing: 11 84 51.2 74 397 >> Waiting: 8 77 49.4 66 397 >> Total: 11 84 51.2 75 398 >> >> Percentage of the requests served within a certain time (ms) >> 50% 75 >> 66% 97 >> 75% 113 >> 80% 124 >> 90% 154 >> 95% 182 >> 98% 217 >> 99% 239 >> 100% 398 (longest request) >> >> Now with 8 users: >> >> dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200 >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking localhost (be patient) >> Completed 1024 requests >> Completed 2048 requests >> Completed 3072 requests >> Completed 4096 requests >> Completed 5120 requests >> Completed 6144 requests >> Completed 7168 requests >> Completed 8192 requests >> Completed 9216 requests >> Completed 10240 requests >> Finished 10240 requests >> >> >> Server Software: Jetty(6.1.16) >> Server Hostname: localhost >> Server Port: 8080 >> >> Document Path: /wicketapp/products3/200 >> Document Length: 34290 bytes >> >> Concurrency Level: 8 >> Time taken for tests: 48.672 seconds >> Complete requests: 10240 >> Failed requests: 6308 >> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0) >> Write errors: 0 >> Total transferred: 355046233 bytes >> HTML transferred: 352875353 bytes >> Requests per second: 210.39 [#/sec] (mean) >> Time per request: 38.025 [ms] (mean) >> Time per request: 4.753 [ms] (mean, across all concurrent requests) >> Transfer rate: 7123.72 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.9 0 10 >> Processing: 11 38 19.9 33 203 >> Waiting: 9 35 18.6 30 203 >> Total: 11 38 20.0 33 204 >> >> Percentage of the requests served within a certain time (ms) >> 50% 33 >> 66% 41 >> 75% 47 >> 80% 53 >> 90% 65 >> 95% 77 >> 98% 93 >> 99% 105 >> 100% 204 (longest request) >> >> And with 4 users: >> >> dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200 >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking localhost (be patient) >> Completed 1024 requests >> Completed 2048 requests >> Completed 3072 requests >> Completed 4096 requests >> Completed 5120 requests >> Completed 6144 requests >> Completed 7168 requests >> Completed 8192 requests >> Completed 9216 requests >> Completed 10240 requests >> Finished 10240 requests >> >> >> Server Software: Jetty(6.1.16) >> Server Hostname: localhost >> Server Port: 8080 >> >> Document Path: /wicketapp/products3/200 >> Document Length: 34290 bytes >> >> Concurrency Level: 4 >> Time taken for tests: 45.238 seconds >> Complete requests: 10240 >> Failed requests: 6134 >> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0) >> Write errors: 0 >> Total transferred: 355063755 bytes >> HTML transferred: 352892875 bytes >> Requests per second: 226.36 [#/sec] (mean) >> Time per request: 17.671 [ms] (mean) >> Time per request: 4.418 [ms] (mean, across all concurrent requests) >> Transfer rate: 7664.91 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.2 0 7 >> Processing: 11 17 2.4 18 43 >> Waiting: 10 17 2.5 17 41 >> Total: 11 18 2.4 18 43 >> >> Percentage of the requests served within a certain time (ms) >> 50% 18 >> 66% 18 >> 75% 19 >> 80% 19 >> 90% 20 >> 95% 21 >> 98% 23 >> 99% 25 >> 100% 43 (longest request) >> >> And finally with 2 users: >> >> dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200 >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking localhost (be patient) >> Completed 1024 requests >> Completed 2048 requests >> Completed 3072 requests >> Completed 4096 requests >> Completed 5120 requests >> Completed 6144 requests >> Completed 7168 requests >> Completed 8192 requests >> Completed 9216 requests >> Completed 10240 requests >> Finished 10240 requests >> >> >> Server Software: Jetty(6.1.16) >> Server Hostname: localhost >> Server Port: 8080 >> >> Document Path: /wicketapp/products3/200 >> Document Length: 34691 bytes >> >> Concurrency Level: 2 >> Time taken for tests: 55.005 seconds >> Complete requests: 10240 >> Failed requests: 7532 >> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0) >> Write errors: 0 >> Total transferred: 355060137 bytes >> HTML transferred: 352889257 bytes >> Requests per second: 186.16 [#/sec] (mean) >> Time per request: 10.743 [ms] (mean) >> Time per request: 5.372 [ms] (mean, across all concurrent requests) >> Transfer rate: 6303.74 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.0 0 1 >> Processing: 6 11 2.8 11 25 >> Waiting: 6 10 2.8 10 22 >> Total: 7 11 2.8 11 25 >> >> Percentage of the requests served within a certain time (ms) >> 50% 11 >> 66% 12 >> 75% 13 >> 80% 14 >> 90% 15 >> 95% 15 >> 98% 16 >> 99% 16 >> 100% 25 (longest request) >> >> Looking at the results I think it is safe to say that with 16 users we >> hit the limit of my core i7, and with 4 users we hit the sweet spot >> (which is probably correct since I have 2 cores with each 2 >> hyperthreads). >> >> I expected 4 users to perform similar to 2 users, but it appears to be >> slower. I'll run it using yourkit to see if there's a blocker >> somewhere. >> >> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4. >> >> Notes from this test: >> - we report wrong length in several cases >> - exceptions occur when a request is broken off >> - performance seems on par with 1.4 >> >> Martijn >> >
