Re: commons pool benchmark issue
On 30/09/2013 03:35, Daniel Wu wrote: Without the pool, it's lightning fast. JProfiler shows most of the time is consumed on locking, thread dump also shows the pool always waits. I tried to timing the pool operations, the results proved the JProfiler reports e,g. with 50 threads, the log shows Average Response Time:0.06 Average Borrow Time:0.04 Average Return Time:0.02 So, the time are consumed with StringBuilder and GC is negligible, most of the time are wasted in pool locking. With 100 threads, the log shows Average Response Time:1.24 Average Borrow Time:0.90 Average Return Time:0.34 The more threads try to content with the pool, the slower the pool is. I kind of believe CP is just slow. In environments where there is more than one core/processor available and there is a high rate of borrow/return events then pool 1.x is slow and it will get worse the more cores/processors that there are available. Pool 2 was designed to address those performance issues. 1.5.x and 1.6.x are better than earlier 1.x versions but 2.x should be a significant improvement. Mark - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: commons pool benchmark issue
Hi Thomas, Thanks a lot. For now, I have to use my own implementation which is 50 times faster than CP but lack of many features from CP. Looking forward to CP 2.x release, and I don't like invent my own wheels. -- Daniel Wu Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, September 30, 2013 at 3:27 PM, Mark Thomas wrote: On 30/09/2013 03:35, Daniel Wu wrote: Without the pool, it's lightning fast. JProfiler shows most of the time is consumed on locking, thread dump also shows the pool always waits. I tried to timing the pool operations, the results proved the JProfiler reports e,g. with 50 threads, the log shows Average Response Time:0.06 Average Borrow Time:0.04 Average Return Time:0.02 So, the time are consumed with StringBuilder and GC is negligible, most of the time are wasted in pool locking. With 100 threads, the log shows Average Response Time:1.24 Average Borrow Time:0.90 Average Return Time:0.34 The more threads try to content with the pool, the slower the pool is. I kind of believe CP is just slow. In environments where there is more than one core/processor available and there is a high rate of borrow/return events then pool 1.x is slow and it will get worse the more cores/processors that there are available. Pool 2 was designed to address those performance issues. 1.5.x and 1.6.x are better than earlier 1.x versions but 2.x should be a significant improvement. Mark - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org (mailto:user-unsubscr...@commons.apache.org) For additional commands, e-mail: user-h...@commons.apache.org (mailto:user-h...@commons.apache.org)
Re: commons pool benchmark issue
Perhaps you would care to share your implementation with Commons Pool and help make the library better? On Mon, Sep 30, 2013 at 3:58 AM, Daniel Wu daniel.y@gmail.com wrote: Hi Thomas, Thanks a lot. For now, I have to use my own implementation which is 50 times faster than CP but lack of many features from CP. Looking forward to CP 2.x release, and I don't like invent my own wheels. -- Daniel Wu Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, September 30, 2013 at 3:27 PM, Mark Thomas wrote: On 30/09/2013 03:35, Daniel Wu wrote: Without the pool, it's lightning fast. JProfiler shows most of the time is consumed on locking, thread dump also shows the pool always waits. I tried to timing the pool operations, the results proved the JProfiler reports e,g. with 50 threads, the log shows Average Response Time:0.06 Average Borrow Time:0.04 Average Return Time:0.02 So, the time are consumed with StringBuilder and GC is negligible, most of the time are wasted in pool locking. With 100 threads, the log shows Average Response Time:1.24 Average Borrow Time:0.90 Average Return Time:0.34 The more threads try to content with the pool, the slower the pool is. I kind of believe CP is just slow. In environments where there is more than one core/processor available and there is a high rate of borrow/return events then pool 1.x is slow and it will get worse the more cores/processors that there are available. Pool 2 was designed to address those performance issues. 1.5.x and 1.6.x are better than earlier 1.x versions but 2.x should be a significant improvement. Mark - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org (mailto:user-unsubscr...@commons.apache.org) For additional commands, e-mail: user-h...@commons.apache.org (mailto:user-h...@commons.apache.org) - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: commons pool benchmark issue
On 30/09/2013 08:58, Daniel Wu wrote: Hi Thomas, Thanks a lot. For now, I have to use my own implementation which is 50 times faster than CP but lack of many features from CP. Looking forward to CP 2.x release, and I don't like invent my own wheels. You can get a Pool 2.x snapshot release from here to test with. https://repository.apache.org/content/groups/snapshots/org/apache/commons/commons-pool2/2.0-SNAPSHOT/ Mark - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
commons pool benchmark issue
Hi guys,I wrote a simple test for commons-pool. The result shows that the performance drops dramatically when concurrent threads increase. e.g, With 50 threads, the throughput is 603326 transactions per second, when threads increase to 600, the throughput drops to 32797. The diagram below is the response time in milliseconds, you can see as the threads increase, the response time grows quickly.I guess my benchmark test code must have something wrong, could you guys help me review the test code below and figure out what's wrong with my test code?public class BenchmarkCommons { public BenchmarkCommons(int workerCount, int loop) throws Exception {double[] statsAvgRespTime = new double[workerCount];CountDownLatch latch = new CountDownLatch(workerCount);GenericObjectPool pool = new GenericObjectPool(new PoolableObjectFactory() { @Override public Object makeObject() throws Exception {return new StringBuilder(); } @Override public void destroyObject(Object o) throws Exception { } @Override public boolean validateObject(Object o) {return true; } @Override public void activateObject(Object o) throws Exception { } @Override public void passivateObject(Object o) throws Exception { }});pool.setMinIdle(25);pool.setMaxIdle(50);pool.setMaxActive(50);Worker[] workers = new Worker[workerCount];for (int i = 0; i workerCount; i++) { workers[i] = new Worker(i, pool, latch, loop, statsAvgRespTime);}long t1 = System.currentTimeMillis();for (int i = 0; i workerCount; i++) { workers[i].start();}latch.await();long t2 = System.currentTimeMillis();double stats = 0;for (int i = 0; i workerCount; i++) { stats += statsAvgRespTime[i];}System.out.println("Average Response Time:" + new DecimalFormat("0").format(stats / workerCount));System.out.println("Average Througput Per Second:" + new DecimalFormat("0").format(( (double) loop * workerCount * 1000 ) / (t2 - t1) )); } private static class Worker extends Thread {private final int id;private final GenericObjectPool pool;private final CountDownLatch latch;private final int loop;private final double[] statsAvgRespTime;public Worker(int id, GenericObjectPool pool, CountDownLatch latch, int loop, double[] statsAvgRespTime) { this.id = id; this.pool = pool; this.latch = latch; this.loop = loop; this.statsAvgRespTime = statsAvgRespTime;}@Override public void run() { long t1 = System.currentTimeMillis(); for (int i = 0; i loop; i++) {StringBuilder obj = null;try { obj = (StringBuilder) pool.borrowObject(); obj.append("x");} catch (Exception e) { e.printStackTrace();} finally { if (obj != null) {try { pool.returnObject(obj);} catch (Exception e) { e.printStackTrace();} }} } long t2 = System.currentTimeMillis(); statsAvgRespTime[id] = ((double) (t2 - t1)) / loop; latch.countDown();} } public static void main(String[] args) throws Exception {System.out.println("---warm up");new BenchmarkCommons(50, 100);System.out.println("Apache commons pool test---");new BenchmarkCommons(50, 50);new BenchmarkCommons(100, 50);new BenchmarkCommons(150, 30);new BenchmarkCommons(200, 30);new BenchmarkCommons(250, 10);new BenchmarkCommons(300, 10);new BenchmarkCommons(350, 5);new BenchmarkCommons(400, 5);new BenchmarkCommons(450, 2);new BenchmarkCommons(500, 2);new BenchmarkCommons(550, 1);new BenchmarkCommons(600, 1); }}Regards,Daniel Wu
Re: commons pool benchmark issue
On 9/28/13 11:50 PM, Daniel Wu wrote:
Hi guys,
I wrote a simple test for commons-pool. The result shows that the
performance drops dramatically when concurrent threads increase.
e.g, With 50 threads, the throughput is 603326 transactions per
second, when threads increase to 600, the throughput drops to
32797. The diagram below is the response time in milliseconds,
you can see as the threads increase, the response time grows quickly.
I guess my benchmark test code must have something wrong, could
you guys help me review the test code below and figure out what's
wrong with my test code?
public class BenchmarkCommons {
public BenchmarkCommons(int workerCount, int loop) throws
Exception {
double[] statsAvgRespTime = new double[workerCount];
CountDownLatch latch = new CountDownLatch(workerCount);
GenericObjectPool pool = new GenericObjectPool(new
PoolableObjectFactory() {
@Override
public Object makeObject() throws Exception {
return new StringBuilder();
}
@Override
public void destroyObject(Object o) throws Exception {
}
@Override
public boolean validateObject(Object o) {
return true;
}
@Override
public void activateObject(Object o) throws Exception {
}
@Override
public void passivateObject(Object o) throws Exception {
}
});
pool.setMinIdle(25);
pool.setMaxIdle(50);
pool.setMaxActive(50);
Worker[] workers = new Worker[workerCount];
for (int i = 0; i workerCount; i++) {
workers[i] = new Worker(i, pool, latch, loop,
statsAvgRespTime);
}
long t1 = System.currentTimeMillis();
for (int i = 0; i workerCount; i++) {
workers[i].start();
}
latch.await();
long t2 = System.currentTimeMillis();
double stats = 0;
for (int i = 0; i workerCount; i++) {
stats += statsAvgRespTime[i];
}
System.out.println(Average Response Time: + new
DecimalFormat(0).format(stats / workerCount));
System.out.println(Average Througput Per Second: + new
DecimalFormat(0).format(( (double) loop * workerCount * 1000 )
/ (t2 - t1) ));
}
private static class Worker extends Thread {
private final int id;
private final GenericObjectPool pool;
private final CountDownLatch latch;
private final int loop;
private final double[] statsAvgRespTime;
public Worker(int id, GenericObjectPool pool,
CountDownLatch latch, int loop, double[] statsAvgRespTime) {
this.id = id;
this.pool = pool;
this.latch = latch;
this.loop = loop;
this.statsAvgRespTime = statsAvgRespTime;
}
@Override public void run() {
long t1 = System.currentTimeMillis();
for (int i = 0; i loop; i++) {
StringBuilder obj = null;
try {
obj = (StringBuilder) pool.borrowObject();
obj.append(x);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (obj != null) {
try {
pool.returnObject(obj);
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
long t2 = System.currentTimeMillis();
statsAvgRespTime[id] = ((double) (t2 - t1)) / loop;
latch.countDown();
}
}
public static void main(String[] args) throws Exception {
System.out.println(---warm up);
new BenchmarkCommons(50, 100);
System.out.println(Apache commons pool
test---);
new BenchmarkCommons(50, 50);
new BenchmarkCommons(100, 50);
new BenchmarkCommons(150, 30);
new BenchmarkCommons(200, 30);
new BenchmarkCommons(250, 10);
new BenchmarkCommons(300, 10);
new BenchmarkCommons(350, 5);
new BenchmarkCommons(400, 5);
new BenchmarkCommons(450, 2);
new BenchmarkCommons(500, 2);
new BenchmarkCommons(550, 1);
new BenchmarkCommons(600, 1);
}
}
First some questions, then some observations.
0) What version of Commons Pool are you running?
1) How many physical CPU cores does the machine you are using to run
the test have?
Now some observations:
2) To measure the performance of the pool itself, you should time
the borrows and returns, not the full loop including the client action
3) With a
Re: commons pool benchmark issue
Hi Phil,
Thanks for your reply
1) CP Version: I tried both 1.5.5 and 1.6.0, same results.
2) My machine is a 2010-mid iMac, 3.2GHz i3 with 4 cores, 16G RAM, jdk 1.7u40
3) I don't think StringBuilder is a problem, my profiler tool shows that GC is
not frequent, there is no memory pressure. But the tool shows that most of the
time are consumed at pool locking.
--
Daniel Wu
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Monday, September 30, 2013 at 12:25 AM, Phil Steitz wrote:
On 9/28/13 11:50 PM, Daniel Wu wrote:
Hi guys,
I wrote a simple test for commons-pool. The result shows that the
performance drops dramatically when concurrent threads increase.
e.g, With 50 threads, the throughput is 603326 transactions per
second, when threads increase to 600, the throughput drops to
32797. The diagram below is the response time in milliseconds,
you can see as the threads increase, the response time grows quickly.
I guess my benchmark test code must have something wrong, could
you guys help me review the test code below and figure out what's
wrong with my test code?
public class BenchmarkCommons {
public BenchmarkCommons(int workerCount, int loop) throws
Exception {
double[] statsAvgRespTime = new double[workerCount];
CountDownLatch latch = new CountDownLatch(workerCount);
GenericObjectPool pool = new GenericObjectPool(new
PoolableObjectFactory() {
@Override
public Object makeObject() throws Exception {
return new StringBuilder();
}
@Override
public void destroyObject(Object o) throws Exception {
}
@Override
public boolean validateObject(Object o) {
return true;
}
@Override
public void activateObject(Object o) throws Exception {
}
@Override
public void passivateObject(Object o) throws Exception {
}
});
pool.setMinIdle(25);
pool.setMaxIdle(50);
pool.setMaxActive(50);
Worker[] workers = new Worker[workerCount];
for (int i = 0; i workerCount; i++) {
workers[i] = new Worker(i, pool, latch, loop,
statsAvgRespTime);
}
long t1 = System.currentTimeMillis();
for (int i = 0; i workerCount; i++) {
workers[i].start();
}
latch.await();
long t2 = System.currentTimeMillis();
double stats = 0;
for (int i = 0; i workerCount; i++) {
stats += statsAvgRespTime[i];
}
System.out.println(Average Response Time: + new
DecimalFormat(0).format(stats / workerCount));
System.out.println(Average Througput Per Second: + new
DecimalFormat(0).format(( (double) loop * workerCount * 1000 )
/ (t2 - t1) ));
}
private static class Worker extends Thread {
private final int id;
private final GenericObjectPool pool;
private final CountDownLatch latch;
private final int loop;
private final double[] statsAvgRespTime;
public Worker(int id, GenericObjectPool pool,
CountDownLatch latch, int loop, double[] statsAvgRespTime) {
this.id = id;
this.pool = pool;
this.latch = latch;
this.loop = loop;
this.statsAvgRespTime = statsAvgRespTime;
}
@Override public void run() {
long t1 = System.currentTimeMillis();
for (int i = 0; i loop; i++) {
StringBuilder obj = null;
try {
obj = (StringBuilder) pool.borrowObject();
obj.append(x);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (obj != null) {
try {
pool.returnObject(obj);
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
long t2 = System.currentTimeMillis();
statsAvgRespTime[id] = ((double) (t2 - t1)) / loop;
latch.countDown();
}
}
public static void main(String[] args) throws Exception {
System.out.println(---warm up);
new BenchmarkCommons(50, 100);
System.out.println(Apache commons pool
test---);
new BenchmarkCommons(50, 50);
new BenchmarkCommons(100, 50);
new BenchmarkCommons(150, 30);
new BenchmarkCommons(200, 30);
new BenchmarkCommons(250, 10);
new BenchmarkCommons(300, 10);
new BenchmarkCommons(350, 5);
new BenchmarkCommons(400, 5);
new BenchmarkCommons(450, 2);
new BenchmarkCommons(500, 2);
new BenchmarkCommons(550, 1);
new BenchmarkCommons(600, 1);
}
}
First some questions, then some observations.
0) What version of Commons Pool are you running?
1) How many physical CPU cores does the machine you are using to run
the test have?
Now some observations:
2) To measure the performance of the pool itself, you should time
the borrows and returns, not the full loop including the client action
3) With a large number of threads, the StringBuilders are going to
start to get big. You could be running into delays managing these
buffers.
If the answer to 1) above is only a few, you could be running into
thread scheduling delays. That together with churn caused by 3)
could be
Re: commons pool benchmark issue
On 9/29/13 6:23 PM, Daniel Wu wrote: Hi Phil, Thanks for your reply 1) CP Version: I tried both 1.5.5 and 1.6.0, same results. 2) My machine is a 2010-mid iMac, 3.2GHz i3 with 4 cores, 16G RAM, jdk 1.7u40 3) I don't think StringBuilder is a problem, my profiler tool shows that GC is not frequent, there is no memory pressure. But the tool shows that most of the time are consumed at pool locking. Did you try to run it without the pool? Did you try timing just the pool operations? - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: commons pool benchmark issue
Without the pool, it's lightning fast. JProfiler shows most of the time is consumed on locking, thread dump also shows the pool always waits. I tried to timing the pool operations, the results proved the JProfiler reports e,g. with 50 threads, the log shows Average Response Time:0.06 Average Borrow Time:0.04 Average Return Time:0.02 So, the time are consumed with StringBuilder and GC is negligible, most of the time are wasted in pool locking. With 100 threads, the log shows Average Response Time:1.24 Average Borrow Time:0.90 Average Return Time:0.34 The more threads try to content with the pool, the slower the pool is. I kind of believe CP is just slow. -- Daniel Wu Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, September 30, 2013 at 10:02 AM, Phil Steitz wrote: On 9/29/13 6:23 PM, Daniel Wu wrote: Hi Phil, Thanks for your reply 1) CP Version: I tried both 1.5.5 and 1.6.0, same results. 2) My machine is a 2010-mid iMac, 3.2GHz i3 with 4 cores, 16G RAM, jdk 1.7u40 3) I don't think StringBuilder is a problem, my profiler tool shows that GC is not frequent, there is no memory pressure. But the tool shows that most of the time are consumed at pool locking. Did you try to run it without the pool? Did you try timing just the pool operations? - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org (mailto:user-unsubscr...@commons.apache.org) For additional commands, e-mail: user-h...@commons.apache.org (mailto:user-h...@commons.apache.org)
Re: commons pool benchmark issue
On 9/29/13 7:35 PM, Daniel Wu wrote: Without the pool, it's lightning fast. JProfiler shows most of the time is consumed on locking, thread dump also shows the pool always waits. I tried to timing the pool operations, the results proved the JProfiler reports e,g. with 50 threads, the log shows Average Response Time:0.06 Average Borrow Time:0.04 Average Return Time:0.02 So, the time are consumed with StringBuilder and GC is negligible, most of the time are wasted in pool locking. With 100 threads, the log shows Average Response Time:1.24 Average Borrow Time:0.90 Average Return Time:0.34 The more threads try to content with the pool, the slower the pool is. I kind of believe CP is just slow. What happens when you replace the append with a no-op? Phil - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: commons pool benchmark issue
I removed the line obj.append(x), so the worker threads simple borrow and return. Same result, StringBuilder and GC are negligible. -- Daniel Wu Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, September 30, 2013 at 12:04 PM, Phil Steitz wrote: On 9/29/13 7:35 PM, Daniel Wu wrote: Without the pool, it's lightning fast. JProfiler shows most of the time is consumed on locking, thread dump also shows the pool always waits. I tried to timing the pool operations, the results proved the JProfiler reports e,g. with 50 threads, the log shows Average Response Time:0.06 Average Borrow Time:0.04 Average Return Time:0.02 So, the time are consumed with StringBuilder and GC is negligible, most of the time are wasted in pool locking. With 100 threads, the log shows Average Response Time:1.24 Average Borrow Time:0.90 Average Return Time:0.34 The more threads try to content with the pool, the slower the pool is. I kind of believe CP is just slow. What happens when you replace the append with a no-op? Phil - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org (mailto:user-unsubscr...@commons.apache.org) For additional commands, e-mail: user-h...@commons.apache.org (mailto:user-h...@commons.apache.org)
