You can use wicket-threadtest On Thu, Jun 16, 2011 at 9:42 PM, Bruno Borges <[email protected]> wrote: > Martijn, would you mind to share the benchmark suite you have? > > I'd like to test a few changes. > > > *Bruno Borges* > www.brunoborges.com.br > +55 21 76727099 > > > > On Thu, Jun 16, 2011 at 1:00 PM, Igor Vaynberg <[email protected]>wrote: > >> lol. i was waiting for someone to say that! >> >> -igor >> >> On Thu, Jun 16, 2011 at 8:56 AM, James Carman >> <[email protected]> wrote: >> > Perhaps we should re-write it using wicket-velocity! ;) >> > >> > On Thu, Jun 16, 2011 at 11:53 AM, Igor Vaynberg <[email protected]> >> wrote: >> >> his test is hand-tailored to fail any true component oriented >> >> framework. 5000 items per page? really? show me a single website that >> >> does that. even facebook doesnt show that many comments on the wall >> >> without making you page. but, after looking at other "i hate wicket" >> >> posts on his blog i am not surprised with his testing approach :) >> >> >> >> if this situation came up in the real life any sane developer would >> >> wrap the entire product list into a single component that would stream >> >> html directly using string manipulation. and if you do that then >> >> wicket will be on par - if not faster - then the other ones. because >> >> at that point you are measuring the same thing - how fast the jvm >> >> loops. >> >> >> >> -igor >> >> >> >> >> >> On Thu, Jun 16, 2011 at 6:56 AM, Martijn Dashorst >> >> <[email protected]> wrote: >> >>> The application used in >> >>> http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/ has been >> >>> controversial as it pits all popular frameworks against one another. >> >>> I've tried to replicate the results on my box with AB, and did some >> >>> modifications to the code: >> >>> - removed the image from the markup: browsers will hit the >> >>> non-existent image causing long load times >> >>> - did the categories list in a similar way that the Tapestry >> >>> implementation did: using a StringBuilder (saves ~300ms for a single >> >>> user with 5000 items) >> >>> - randomized the results for each query to the products() method such >> >>> that caching the results is not an option >> >>> >> >>> I couldn't replicate the tapestry results with these settings (instead >> >>> of being almost constant time, it now is linear but 1.5x faster than >> >>> wicket). I'm not sure if the author actually looked at the results of >> >>> the tapestry tests, but it seems to me that something went wrong. >> >>> >> >>> The code has been altered with the above changes, including changing >> >>> the ListView into a RepeatingView, and rendering the categories in a >> >>> StringBuilder. Here are the results of an ab test on my laptop for 200 >> >>> products and 32 concurrent users: >> >>> >> >>> dashorst$ ab -c 32 -n 10240 >> http://localhost:8080/wicketapp/products3/200 >> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >> http://www.zeustech.net/ >> >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >>> >> >>> Benchmarking localhost (be patient) >> >>> Completed 1024 requests >> >>> Completed 2048 requests >> >>> Completed 3072 requests >> >>> Completed 4096 requests >> >>> Completed 5120 requests >> >>> Completed 6144 requests >> >>> Completed 7168 requests >> >>> Completed 8192 requests >> >>> Completed 9216 requests >> >>> Completed 10240 requests >> >>> Finished 10240 requests >> >>> >> >>> >> >>> Server Software: Jetty(6.1.16) >> >>> Server Hostname: localhost >> >>> Server Port: 8080 >> >>> >> >>> Document Path: /wicketapp/products3/200 >> >>> Document Length: 34689 bytes >> >>> >> >>> Concurrency Level: 32 >> >>> Time taken for tests: 57.509 seconds >> >>> Complete requests: 10240 >> >>> Failed requests: 9670 >> >>> (Connect: 0, Receive: 0, Length: 9670, Exceptions: 0) >> >>> Write errors: 0 >> >>> Total transferred: 354991304 bytes >> >>> HTML transferred: 352820424 bytes >> >>> Requests per second: 178.06 [#/sec] (mean) >> >>> Time per request: 179.714 [ms] (mean) >> >>> Time per request: 5.616 [ms] (mean, across all concurrent >> requests) >> >>> Transfer rate: 6028.16 [Kbytes/sec] received >> >>> >> >>> Connection Times (ms) >> >>> min mean[+/-sd] median max >> >>> Connect: 0 1 1.6 0 17 >> >>> Processing: 11 179 123.2 163 1000 >> >>> Waiting: 11 171 123.1 154 999 >> >>> Total: 12 179 123.3 164 1000 >> >>> >> >>> Percentage of the requests served within a certain time (ms) >> >>> 50% 164 >> >>> 66% 218 >> >>> 75% 248 >> >>> 80% 268 >> >>> 90% 337 >> >>> 95% 418 >> >>> 98% 491 >> >>> 99% 552 >> >>> 100% 1000 (longest request) >> >>> >> >>> The same test but then using 16 users: >> >>> >> >>> dashorst$ ab -c 16 -n 10240 >> http://localhost:8080/wicketapp/products3/200 >> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >> http://www.zeustech.net/ >> >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >>> >> >>> Benchmarking localhost (be patient) >> >>> Completed 1024 requests >> >>> Completed 2048 requests >> >>> Completed 3072 requests >> >>> Completed 4096 requests >> >>> Completed 5120 requests >> >>> Completed 6144 requests >> >>> Completed 7168 requests >> >>> Completed 8192 requests >> >>> Completed 9216 requests >> >>> Completed 10240 requests >> >>> Finished 10240 requests >> >>> >> >>> >> >>> Server Software: Jetty(6.1.16) >> >>> Server Hostname: localhost >> >>> Server Port: 8080 >> >>> >> >>> Document Path: /wicketapp/products3/200 >> >>> Document Length: 34290 bytes >> >>> >> >>> Concurrency Level: 16 >> >>> Time taken for tests: 54.005 seconds >> >>> Complete requests: 10240 >> >>> Failed requests: 6161 >> >>> (Connect: 0, Receive: 0, Length: 6161, Exceptions: 0) >> >>> Write errors: 0 >> >>> Total transferred: 355066608 bytes >> >>> HTML transferred: 352895728 bytes >> >>> Requests per second: 189.61 [#/sec] (mean) >> >>> Time per request: 84.383 [ms] (mean) >> >>> Time per request: 5.274 [ms] (mean, across all concurrent >> requests) >> >>> Transfer rate: 6420.58 [Kbytes/sec] received >> >>> >> >>> Connection Times (ms) >> >>> min mean[+/-sd] median max >> >>> Connect: 0 0 1.0 0 16 >> >>> Processing: 11 84 51.2 74 397 >> >>> Waiting: 8 77 49.4 66 397 >> >>> Total: 11 84 51.2 75 398 >> >>> >> >>> Percentage of the requests served within a certain time (ms) >> >>> 50% 75 >> >>> 66% 97 >> >>> 75% 113 >> >>> 80% 124 >> >>> 90% 154 >> >>> 95% 182 >> >>> 98% 217 >> >>> 99% 239 >> >>> 100% 398 (longest request) >> >>> >> >>> Now with 8 users: >> >>> >> >>> dashorst$ ab -c 8 -n 10240 >> http://localhost:8080/wicketapp/products3/200 >> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >> http://www.zeustech.net/ >> >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >>> >> >>> Benchmarking localhost (be patient) >> >>> Completed 1024 requests >> >>> Completed 2048 requests >> >>> Completed 3072 requests >> >>> Completed 4096 requests >> >>> Completed 5120 requests >> >>> Completed 6144 requests >> >>> Completed 7168 requests >> >>> Completed 8192 requests >> >>> Completed 9216 requests >> >>> Completed 10240 requests >> >>> Finished 10240 requests >> >>> >> >>> >> >>> Server Software: Jetty(6.1.16) >> >>> Server Hostname: localhost >> >>> Server Port: 8080 >> >>> >> >>> Document Path: /wicketapp/products3/200 >> >>> Document Length: 34290 bytes >> >>> >> >>> Concurrency Level: 8 >> >>> Time taken for tests: 48.672 seconds >> >>> Complete requests: 10240 >> >>> Failed requests: 6308 >> >>> (Connect: 0, Receive: 0, Length: 6308, Exceptions: 0) >> >>> Write errors: 0 >> >>> Total transferred: 355046233 bytes >> >>> HTML transferred: 352875353 bytes >> >>> Requests per second: 210.39 [#/sec] (mean) >> >>> Time per request: 38.025 [ms] (mean) >> >>> Time per request: 4.753 [ms] (mean, across all concurrent >> requests) >> >>> Transfer rate: 7123.72 [Kbytes/sec] received >> >>> >> >>> Connection Times (ms) >> >>> min mean[+/-sd] median max >> >>> Connect: 0 0 0.9 0 10 >> >>> Processing: 11 38 19.9 33 203 >> >>> Waiting: 9 35 18.6 30 203 >> >>> Total: 11 38 20.0 33 204 >> >>> >> >>> Percentage of the requests served within a certain time (ms) >> >>> 50% 33 >> >>> 66% 41 >> >>> 75% 47 >> >>> 80% 53 >> >>> 90% 65 >> >>> 95% 77 >> >>> 98% 93 >> >>> 99% 105 >> >>> 100% 204 (longest request) >> >>> >> >>> And with 4 users: >> >>> >> >>> dashorst$ ab -c 4 -n 10240 >> http://localhost:8080/wicketapp/products3/200 >> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >> http://www.zeustech.net/ >> >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >>> >> >>> Benchmarking localhost (be patient) >> >>> Completed 1024 requests >> >>> Completed 2048 requests >> >>> Completed 3072 requests >> >>> Completed 4096 requests >> >>> Completed 5120 requests >> >>> Completed 6144 requests >> >>> Completed 7168 requests >> >>> Completed 8192 requests >> >>> Completed 9216 requests >> >>> Completed 10240 requests >> >>> Finished 10240 requests >> >>> >> >>> >> >>> Server Software: Jetty(6.1.16) >> >>> Server Hostname: localhost >> >>> Server Port: 8080 >> >>> >> >>> Document Path: /wicketapp/products3/200 >> >>> Document Length: 34290 bytes >> >>> >> >>> Concurrency Level: 4 >> >>> Time taken for tests: 45.238 seconds >> >>> Complete requests: 10240 >> >>> Failed requests: 6134 >> >>> (Connect: 0, Receive: 0, Length: 6134, Exceptions: 0) >> >>> Write errors: 0 >> >>> Total transferred: 355063755 bytes >> >>> HTML transferred: 352892875 bytes >> >>> Requests per second: 226.36 [#/sec] (mean) >> >>> Time per request: 17.671 [ms] (mean) >> >>> Time per request: 4.418 [ms] (mean, across all concurrent >> requests) >> >>> Transfer rate: 7664.91 [Kbytes/sec] received >> >>> >> >>> Connection Times (ms) >> >>> min mean[+/-sd] median max >> >>> Connect: 0 0 0.2 0 7 >> >>> Processing: 11 17 2.4 18 43 >> >>> Waiting: 10 17 2.5 17 41 >> >>> Total: 11 18 2.4 18 43 >> >>> >> >>> Percentage of the requests served within a certain time (ms) >> >>> 50% 18 >> >>> 66% 18 >> >>> 75% 19 >> >>> 80% 19 >> >>> 90% 20 >> >>> 95% 21 >> >>> 98% 23 >> >>> 99% 25 >> >>> 100% 43 (longest request) >> >>> >> >>> And finally with 2 users: >> >>> >> >>> dashorst$ ab -c 2 -n 10240 >> http://localhost:8080/wicketapp/products3/200 >> >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, >> http://www.zeustech.net/ >> >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >>> >> >>> Benchmarking localhost (be patient) >> >>> Completed 1024 requests >> >>> Completed 2048 requests >> >>> Completed 3072 requests >> >>> Completed 4096 requests >> >>> Completed 5120 requests >> >>> Completed 6144 requests >> >>> Completed 7168 requests >> >>> Completed 8192 requests >> >>> Completed 9216 requests >> >>> Completed 10240 requests >> >>> Finished 10240 requests >> >>> >> >>> >> >>> Server Software: Jetty(6.1.16) >> >>> Server Hostname: localhost >> >>> Server Port: 8080 >> >>> >> >>> Document Path: /wicketapp/products3/200 >> >>> Document Length: 34691 bytes >> >>> >> >>> Concurrency Level: 2 >> >>> Time taken for tests: 55.005 seconds >> >>> Complete requests: 10240 >> >>> Failed requests: 7532 >> >>> (Connect: 0, Receive: 0, Length: 7532, Exceptions: 0) >> >>> Write errors: 0 >> >>> Total transferred: 355060137 bytes >> >>> HTML transferred: 352889257 bytes >> >>> Requests per second: 186.16 [#/sec] (mean) >> >>> Time per request: 10.743 [ms] (mean) >> >>> Time per request: 5.372 [ms] (mean, across all concurrent >> requests) >> >>> Transfer rate: 6303.74 [Kbytes/sec] received >> >>> >> >>> Connection Times (ms) >> >>> min mean[+/-sd] median max >> >>> Connect: 0 0 0.0 0 1 >> >>> Processing: 6 11 2.8 11 25 >> >>> Waiting: 6 10 2.8 10 22 >> >>> Total: 7 11 2.8 11 25 >> >>> >> >>> Percentage of the requests served within a certain time (ms) >> >>> 50% 11 >> >>> 66% 12 >> >>> 75% 13 >> >>> 80% 14 >> >>> 90% 15 >> >>> 95% 15 >> >>> 98% 16 >> >>> 99% 16 >> >>> 100% 25 (longest request) >> >>> >> >>> Looking at the results I think it is safe to say that with 16 users we >> >>> hit the limit of my core i7, and with 4 users we hit the sweet spot >> >>> (which is probably correct since I have 2 cores with each 2 >> >>> hyperthreads). >> >>> >> >>> I expected 4 users to perform similar to 2 users, but it appears to be >> >>> slower. I'll run it using yourkit to see if there's a blocker >> >>> somewhere. >> >>> >> >>> In my overall experience, 1.5-SNAPSHOT performs comparable to 1.4. >> >>> >> >>> Notes from this test: >> >>> - we report wrong length in several cases >> >>> - exceptions occur when a request is broken off >> >>> - performance seems on par with 1.4 >> >>> >> >>> Martijn >> >>> >> >> >> > >> >
-- Martin Grigorov jWeekend Training, Consulting, Development http://jWeekend.com
