lol. i was waiting for someone to say that! -igor
On Thu, Jun 16, 2011 at 8:56 AM, James Carman <[email protected]> wrote: > Perhaps we should re-write it using wicket-velocity! ;) > > On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <[email protected]> > wrote: >> his test is hand-tailored to fail any true component oriented >> framework. 5000 items per page? really? show me a single website that >> does that. even facebook doesnt show that many comments on the wall >> without making you page. but, after looking at other "i hate wicket" >> posts on his blog i am not surprised with his testing approach :) >> >> if this situation came up in the real life any sane developer would >> wrap the entire product list into a single component that would stream >> html directly using string manipulation. and if you do that then >> wicket will be on par - if not faster - then the other ones. because >> at that point you are measuring the same thing - how fast the jvm >> loops. >> >> -igor >> >> >> On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst >> <[email protected]> wrote: >>> The application used in >>> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been >>> controversial as it pits all popular frameworks against one another. >>> I've tried to replicate the results on my box with AB, and did some >>> modifications to the code: >>> - removed the image from the markup: browsers will hit the >>> non-existent image causing long load times >>> - did the categories list in a similar way that the Tapestry >>> implementation did: using a StringBuilder (saves ~300ms for a single >>> user with 5000 items) >>> - randomized the results for each query to the products() method such >>> that caching the results is not an option >>> >>> I couldn't replicate the tapestry results with these settings (instead >>> of being almost constant time, it now is linear but 1.5x faster than >>> wicket). I'm not sure if the author actually looked at the results of >>> the tapestry tests, but it seems to me that something went wrong. >>> >>> The code has been altered with the above changes, including changing >>> the ListView into a RepeatingView, and rendering the categories in a >>> StringBuilder. Here are the results of an ab test on my laptop for 200 >>> products and 32 concurrent users: >>> >>> dashorst$ ab -c 32 -n 10240 http://localhost:8080/wicketapp/products3/200 >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking localhost (be patient) >>> Completed 1024 requests >>> Completed 2048 requests >>> Completed 3072 requests >>> Completed 4096 requests >>> Completed 5120 requests >>> Completed 6144 requests >>> Completed 7168 requests >>> Completed 8192 requests >>> Completed 9216 requests >>> Completed 10240 requests >>> Finished 10240 requests >>> >>> >>> Server Software: Jetty(6.1.16) >>> Server Hostname: localhost >>> Server Port: 8080 >>> >>> Document Path: /wicketapp/products3/200 >>> Document Length: 34689 bytes >>> >>> Concurrency Level: 32 >>> Time taken for tests: 57.509 seconds >>> Complete requests: 10240 >>> Failed requests: 9670 >>> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0) >>> Write errors: 0 >>> Total transferred: 354991304 bytes >>> HTML transferred: 352820424 bytes >>> Requests per second: 178.06 [#/sec] (mean) >>> Time per request: 179.714 [ms] (mean) >>> Time per request: 5.616 [ms] (mean, across all concurrent requests) >>> Transfer rate: 6028.16 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 1 1.6 0 17 >>> Processing: 11 179 123.2 163 1000 >>> Waiting: 11 171 123.1 154 999 >>> Total: 12 179 123.3 164 1000 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 164 >>> 66% 218 >>> 75% 248 >>> 80% 268 >>> 90% 337 >>> 95% 418 >>> 98% 491 >>> 99% 552 >>> 100% 1000 (longest request) >>> >>> The same test but then using 16 users: >>> >>> dashorst$ ab -c 16 -n 10240 http://localhost:8080/wicketapp/products3/200 >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking localhost (be patient) >>> Completed 1024 requests >>> Completed 2048 requests >>> Completed 3072 requests >>> Completed 4096 requests >>> Completed 5120 requests >>> Completed 6144 requests >>> Completed 7168 requests >>> Completed 8192 requests >>> Completed 9216 requests >>> Completed 10240 requests >>> Finished 10240 requests >>> >>> >>> Server Software: Jetty(6.1.16) >>> Server Hostname: localhost >>> Server Port: 8080 >>> >>> Document Path: /wicketapp/products3/200 >>> Document Length: 34290 bytes >>> >>> Concurrency Level: 16 >>> Time taken for tests: 54.005 seconds >>> Complete requests: 10240 >>> Failed requests: 6161 >>> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0) >>> Write errors: 0 >>> Total transferred: 355066608 bytes >>> HTML transferred: 352895728 bytes >>> Requests per second: 189.61 [#/sec] (mean) >>> Time per request: 84.383 [ms] (mean) >>> Time per request: 5.274 [ms] (mean, across all concurrent requests) >>> Transfer rate: 6420.58 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 1.0 0 16 >>> Processing: 11 84 51.2 74 397 >>> Waiting: 8 77 49.4 66 397 >>> Total: 11 84 51.2 75 398 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 75 >>> 66% 97 >>> 75% 113 >>> 80% 124 >>> 90% 154 >>> 95% 182 >>> 98% 217 >>> 99% 239 >>> 100% 398 (longest request) >>> >>> Now with 8 users: >>> >>> dashorst$ ab -c 8 -n 10240 http://localhost:8080/wicketapp/products3/200 >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking localhost (be patient) >>> Completed 1024 requests >>> Completed 2048 requests >>> Completed 3072 requests >>> Completed 4096 requests >>> Completed 5120 requests >>> Completed 6144 requests >>> Completed 7168 requests >>> Completed 8192 requests >>> Completed 9216 requests >>> Completed 10240 requests >>> Finished 10240 requests >>> >>> >>> Server Software: Jetty(6.1.16) >>> Server Hostname: localhost >>> Server Port: 8080 >>> >>> Document Path: /wicketapp/products3/200 >>> Document Length: 34290 bytes >>> >>> Concurrency Level: 8 >>> Time taken for tests: 48.672 seconds >>> Complete requests: 10240 >>> Failed requests: 6308 >>> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0) >>> Write errors: 0 >>> Total transferred: 355046233 bytes >>> HTML transferred: 352875353 bytes >>> Requests per second: 210.39 [#/sec] (mean) >>> Time per request: 38.025 [ms] (mean) >>> Time per request: 4.753 [ms] (mean, across all concurrent requests) >>> Transfer rate: 7123.72 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.9 0 10 >>> Processing: 11 38 19.9 33 203 >>> Waiting: 9 35 18.6 30 203 >>> Total: 11 38 20.0 33 204 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 33 >>> 66% 41 >>> 75% 47 >>> 80% 53 >>> 90% 65 >>> 95% 77 >>> 98% 93 >>> 99% 105 >>> 100% 204 (longest request) >>> >>> And with 4 users: >>> >>> dashorst$ ab -c 4 -n 10240 http://localhost:8080/wicketapp/products3/200 >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking localhost (be patient) >>> Completed 1024 requests >>> Completed 2048 requests >>> Completed 3072 requests >>> Completed 4096 requests >>> Completed 5120 requests >>> Completed 6144 requests >>> Completed 7168 requests >>> Completed 8192 requests >>> Completed 9216 requests >>> Completed 10240 requests >>> Finished 10240 requests >>> >>> >>> Server Software: Jetty(6.1.16) >>> Server Hostname: localhost >>> Server Port: 8080 >>> >>> Document Path: /wicketapp/products3/200 >>> Document Length: 34290 bytes >>> >>> Concurrency Level: 4 >>> Time taken for tests: 45.238 seconds >>> Complete requests: 10240 >>> Failed requests: 6134 >>> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0) >>> Write errors: 0 >>> Total transferred: 355063755 bytes >>> HTML transferred: 352892875 bytes >>> Requests per second: 226.36 [#/sec] (mean) >>> Time per request: 17.671 [ms] (mean) >>> Time per request: 4.418 [ms] (mean, across all concurrent requests) >>> Transfer rate: 7664.91 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.2 0 7 >>> Processing: 11 17 2.4 18 43 >>> Waiting: 10 17 2.5 17 41 >>> Total: 11 18 2.4 18 43 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 18 >>> 66% 18 >>> 75% 19 >>> 80% 19 >>> 90% 20 >>> 95% 21 >>> 98% 23 >>> 99% 25 >>> 100% 43 (longest request) >>> >>> And finally with 2 users: >>> >>> dashorst$ ab -c 2 -n 10240 http://localhost:8080/wicketapp/products3/200 >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking localhost (be patient) >>> Completed 1024 requests >>> Completed 2048 requests >>> Completed 3072 requests >>> Completed 4096 requests >>> Completed 5120 requests >>> Completed 6144 requests >>> Completed 7168 requests >>> Completed 8192 requests >>> Completed 9216 requests >>> Completed 10240 requests >>> Finished 10240 requests >>> >>> >>> Server Software: Jetty(6.1.16) >>> Server Hostname: localhost >>> Server Port: 8080 >>> >>> Document Path: /wicketapp/products3/200 >>> Document Length: 34691 bytes >>> >>> Concurrency Level: 2 >>> Time taken for tests: 55.005 seconds >>> Complete requests: 10240 >>> Failed requests: 7532 >>> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0) >>> Write errors: 0 >>> Total transferred: 355060137 bytes >>> HTML transferred: 352889257 bytes >>> Requests per second: 186.16 [#/sec] (mean) >>> Time per request: 10.743 [ms] (mean) >>> Time per request: 5.372 [ms] (mean, across all concurrent requests) >>> Transfer rate: 6303.74 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.0 0 1 >>> Processing: 6 11 2.8 11 25 >>> Waiting: 6 10 2.8 10 22 >>> Total: 7 11 2.8 11 25 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 11 >>> 66% 12 >>> 75% 13 >>> 80% 14 >>> 90% 15 >>> 95% 15 >>> 98% 16 >>> 99% 16 >>> 100% 25 (longest request) >>> >>> Looking at the results I think it is safe to say that with 16 users we >>> hit the limit of my core i7, and with 4 users we hit the sweet spot >>> (which is probably correct since I have 2 cores with each 2 >>> hyperthreads). >>> >>> I expected 4 users to perform similar to 2 users, but it appears to be >>> slower. I'll run it using yourkit to see if there's a blocker >>> somewhere. >>> >>> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4. >>> >>> Notes from this test: >>> - we report wrong length in several cases >>> - exceptions occur when a request is broken off >>> - performance seems on par with 1.4 >>> >>> Martijn >>> >> >
