Hi Isuru, Interesting results and findings and it would be interesting to see how these values vary under different conditions (e.g. number of threads). I notice that the error is quite high for Stormpot Blaze Pool (which has the highest throughput) . Any thoughts on how to interpret the "error" here ? (for example, does it reflect the variability of the throughput). When we bench mark the latency we want to see the min, max and latency percentiles etc. Is there a way to get this using JMH?
On Fri, Nov 11, 2016 at 7:43 PM, Sanjeewa Malalgoda <[email protected]> wrote: > Thanks for bring this up isuru. Stack object pool is critical component > for API Manager. > We are using it to do two most critical operations (key validation and > throttle data process and publish) in our gateway. Also we are using it for > both blocking and non blocking operations. So improving this component will > surely help us. Please keep us update with your findings. When we implement > key validation for next version of API Manager we can add this improvement. > > Thanks, > sanjeewa. > > On Fri, Nov 11, 2016 at 3:55 PM, Isuru Perera <[email protected]> wrote: > >> Hi >> >> I wrote $subject using JMH and I committed the code to my GitHub repo [1]. >> >> Currently API Manager uses a Stack Object Pool for key validation clients >> and we noticed that there are lot contentions when trying to use the pool. >> Therefore we thought of writing a benchmark to compare the performance with >> other Object Pool implementations. >> >> Following are the results after running the benchmark with 100 threads. >> The command I used is "java -jar target/objectpools-0.0.1-SNAPSHOT.jar >> -t 100 -f 3 -wi 5 -i 10" >> >> # Run complete. Total time: 00:12:28 >> >> Benchmark (poolSize) >> Mode Cnt Score Error Units >> CommonsPool2GenericObjectPoolBenchmark.useObject 100 >> thrpt 30 2470759.779 ± 204309.885 ops/s >> CommonsPool2SoftReferenceObjectPoolBenchmark.useObject 100 >> thrpt 30 1305244.745 ± 57823.313 ops/s >> CommonsPoolGenericObjectPoolBenchmark.useObject 100 >> thrpt 30 2277105.922 ± 40957.282 ops/s >> CommonsPoolSoftReferenceObjectPoolBenchmark.useObject 100 >> thrpt 30 11163239.940 ± 449147.426 ops/s >> FastObjectPoolBenchmark.useObject 100 >> thrpt 30 30164548.113 ± 984247.958 ops/s >> FuriousObjectPoolBenchmark.useObject 100 >> thrpt 30 14279182.642 ± 115644.976 ops/s >> StackObjectPoolBenchmark.useObject 100 >> thrpt 30 5464597.797 ± 135726.389 ops/s >> StormpotBlazePoolBenchmark.useObject 100 >> thrpt 30 77752385.827 ± 3015457.000 ops/s >> StormpotQueuePoolBenchmark.useObject 100 >> thrpt 30 7670529.272 ± 283981.943 ops/s >> TestObjectBenchmark.expensiveObjectCreate N/A >> thrpt 30 101870.001 ± 2872.885 ops/s >> TestObjectBenchmark.simpleObjectCreate N/A >> thrpt 30 917400188.941 ± 20593977.714 ops/s >> ViburObjectPoolBenchmark.useObject 100 >> thrpt 30 5924114.964 ± 281625.445 ops/s >> >> >> For this benchmark, Stormpot Blaze Pool [2] has the highest throughput >> and it performs 14 times better than the Stack Object Pool, which is used >> in API Manager. >> >> Stormpot is also Apache licensed. >> >> When I was writing the benchmarks, I found out that Stormpot author also >> has written similar benchmarks [3]. But I continued with my project and >> benchmark code has more implementations. I also use latest versions of all >> object pool implementations. >> >> I'm writing a blog about this and planning to get results for different >> scenarios. I need to analyze latency results as well. I'll share those >> results in this mail thread. >> >> Thanks! >> >> Best Regards, >> >> [1] https://github.com/chrishantha/microbenchmarks/tree/master/o >> bjectpools >> [2] https://github.com/chrisvest/stormpot >> [3] https://github.com/chrisvest/object-pool-benchmarks >> >> -- >> Isuru Perera >> Associate Technical Lead | WSO2, Inc. | http://wso2.com/ >> Lean . Enterprise . Middleware >> >> about.me/chrishantha >> Contact: +IsuruPereraWSO2 <https://www.google.com/+IsuruPereraWSO2/about> >> > > > > -- > > *Sanjeewa Malalgoda* > WSO2 Inc. > Mobile : +94713068779 > > <http://sanjeewamalalgoda.blogspot.com/>blog :http://sanjeewamalalgoda. > blogspot.com/ <http://sanjeewamalalgoda.blogspot.com/> > > > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Malith Jayasinghe WSO2, Inc. (http://wso2.com) Email : [email protected] Mobile : 0770704040 Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
