Hi Phil, Thanks for your reply
1) CP Version: I tried both 1.5.5 and 1.6.0, same results. 2) My machine is a 2010-mid iMac, 3.2GHz i3 with 4 cores, 16G RAM, jdk 1.7u40 3) I don't think StringBuilder is a problem, my profiler tool shows that GC is not frequent, there is no memory pressure. But the tool shows that most of the time are consumed at pool locking. -- Daniel Wu Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, September 30, 2013 at 12:25 AM, Phil Steitz wrote: > On 9/28/13 11:50 PM, Daniel Wu wrote: > > > Hi guys, > > > > > > I wrote a simple test for commons-pool. The result shows that the > > > performance drops dramatically when concurrent threads increase. > > > e.g, With 50 threads, the throughput is 603326 transactions per > > > second, when threads increase to 600, the throughput drops to > > > 32797. The diagram below is the response time in milliseconds, > > > you can see as the threads increase, the response time grows quickly. > > > > > > > > > > > > I guess my benchmark test code must have something wrong, could > > > you guys help me review the test code below and figure out what's > > > wrong with my test code? > > > > > > > > > public class BenchmarkCommons { > > > > > > public BenchmarkCommons(int workerCount, int loop) throws > > > Exception { > > > double[] statsAvgRespTime = new double[workerCount]; > > > CountDownLatch latch = new CountDownLatch(workerCount); > > > > > > GenericObjectPool pool = new GenericObjectPool(new > > > PoolableObjectFactory() { > > > @Override > > > public Object makeObject() throws Exception { > > > return new StringBuilder(); > > > } > > > @Override > > > public void destroyObject(Object o) throws Exception { > > > } > > > @Override > > > public boolean validateObject(Object o) { > > > return true; > > > } > > > @Override > > > public void activateObject(Object o) throws Exception { > > > } > > > @Override > > > public void passivateObject(Object o) throws Exception { > > > } > > > }); > > > pool.setMinIdle(25); > > > pool.setMaxIdle(50); > > > pool.setMaxActive(50); > > > > > > Worker[] workers = new Worker[workerCount]; > > > for (int i = 0; i < workerCount; i++) { > > > workers[i] = new Worker(i, pool, latch, loop, > > > statsAvgRespTime); > > > } > > > long t1 = System.currentTimeMillis(); > > > for (int i = 0; i < workerCount; i++) { > > > workers[i].start(); > > > } > > > latch.await(); > > > long t2 = System.currentTimeMillis(); > > > double stats = 0; > > > for (int i = 0; i < workerCount; i++) { > > > stats += statsAvgRespTime[i]; > > > } > > > System.out.println("Average Response Time:" + new > > > DecimalFormat("0").format(stats / workerCount)); > > > System.out.println("Average Througput Per Second:" + new > > > DecimalFormat("0").format(( (double) loop * workerCount * 1000 ) > > > / (t2 - t1) )); > > > } > > > > > > private static class Worker extends Thread { > > > > > > private final int id; > > > private final GenericObjectPool pool; > > > private final CountDownLatch latch; > > > private final int loop; > > > private final double[] statsAvgRespTime; > > > > > > public Worker(int id, GenericObjectPool pool, > > > CountDownLatch latch, int loop, double[] statsAvgRespTime) { > > > this.id = id; > > > this.pool = pool; > > > this.latch = latch; > > > this.loop = loop; > > > this.statsAvgRespTime = statsAvgRespTime; > > > } > > > > > > @Override public void run() { > > > long t1 = System.currentTimeMillis(); > > > for (int i = 0; i < loop; i++) { > > > StringBuilder obj = null; > > > try { > > > obj = (StringBuilder) pool.borrowObject(); > > > obj.append("x"); > > > } catch (Exception e) { > > > e.printStackTrace(); > > > } finally { > > > if (obj != null) { > > > try { > > > pool.returnObject(obj); > > > } catch (Exception e) { > > > e.printStackTrace(); > > > } > > > } > > > } > > > } > > > long t2 = System.currentTimeMillis(); > > > statsAvgRespTime[id] = ((double) (t2 - t1)) / loop; > > > latch.countDown(); > > > } > > > } > > > > > > public static void main(String[] args) throws Exception { > > > System.out.println("-----------warm up------------"); > > > new BenchmarkCommons(50, 100); > > > System.out.println("------------Apache commons pool > > > test-----------"); > > > new BenchmarkCommons(50, 500000); > > > new BenchmarkCommons(100, 500000); > > > new BenchmarkCommons(150, 300000); > > > new BenchmarkCommons(200, 300000); > > > new BenchmarkCommons(250, 100000); > > > new BenchmarkCommons(300, 100000); > > > new BenchmarkCommons(350, 50000); > > > new BenchmarkCommons(400, 50000); > > > new BenchmarkCommons(450, 20000); > > > new BenchmarkCommons(500, 20000); > > > new BenchmarkCommons(550, 10000); > > > new BenchmarkCommons(600, 10000); > > > } > > > > > > } > > First some questions, then some observations. > > 0) What version of Commons Pool are you running? > 1) How many physical CPU cores does the machine you are using to run > the test have? > > Now some observations: > > 2) To measure the performance of the pool itself, you should time > the borrows and returns, not the full loop including the client action > 3) With a large number of threads, the StringBuilders are going to > start to get big. You could be running into delays managing these > buffers. > > If the answer to 1) above is only a few, you could be running into > thread scheduling delays. That together with churn caused by 3) > could be the main effect. To rule out 1), you could do a test just > assigning singleton buffers to each thread, effectively eliminating > the pool altogether. I would wager that with a small number of > cores you will see degraded performance fairly quickly. To rule out > 3), either replace the buffer append with a no-op or use a sleep > instead. To more accurately measure pool performance, per 2), time > just the pool borrow / returns. > > Phil > > > > > > Regards, > > > > > > Daniel Wu > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > (mailto:[email protected]) > For additional commands, e-mail: [email protected] > (mailto:[email protected]) > >
