[
https://issues.apache.org/jira/browse/HBASE-11590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958560#comment-14958560
]
Nicolas Liochon commented on HBASE-11590:
-----------------------------------------
The issue is that the ThreadPoolExecutor leaked all over the place, often for
monitoring reasons.
All lot of code depends on ThreadPoolExecutor rather than the
ExecutorService...
For example, see
{code}
/**
* This class will coalesce increments from a thift server if
* hbase.regionserver.thrift.coalesceIncrement is set to true. Turning this
* config to true will cause the thrift server to queue increments into an
* instance of this class. The thread pool associated with this class will drain
* the coalesced increments as the thread is able. This can cause data loss if
the
* thrift server dies or is shut down before everything in the queue is drained.
*
*/
public class IncrementCoalescer implements IncrementCoalescerMBean {
// snip
// MBean get/set methods
public int getQueueSize() {
return pool.getQueue().size();
}
public int getMaxQueueSize() {
return this.maxQueueSize;
}
public void setMaxQueueSize(int newSize) {
this.maxQueueSize = newSize;
}
public long getPoolCompletedTaskCount() {
return pool.getCompletedTaskCount();
}
public long getPoolTaskCount() {
return pool.getTaskCount();
}
public int getPoolLargestPoolSize() {
return pool.getLargestPoolSize();
}
public int getCorePoolSize() {
return pool.getCorePoolSize();
}
public void setCorePoolSize(int newCoreSize) {
pool.setCorePoolSize(newCoreSize);
}
public int getMaxPoolSize() {
return pool.getMaximumPoolSize();
}
public void setMaxPoolSize(int newMaxSize) {
pool.setMaximumPoolSize(newMaxSize);
}
{code}
I'm going to limit this patch to the easy/client stuff...
> use a specific ThreadPoolExecutor
> ---------------------------------
>
> Key: HBASE-11590
> URL: https://issues.apache.org/jira/browse/HBASE-11590
> Project: HBase
> Issue Type: Bug
> Components: Client, Performance
> Affects Versions: 1.0.0, 2.0.0
> Reporter: Nicolas Liochon
> Assignee: Nicolas Liochon
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: ExecutorServiceTest.java,
> LifoThreadPoolExecutorSQP.java, UnitQueueP.java, UnitQueuePU.java, tp.patch
>
>
> The JDK TPE creates all the threads in the pool. As a consequence, we create
> (by default) 256 threads even if we just need a few.
> The attached TPE create threads only if we have something in the queue.
> On a PE test with replica on, it improved the 99 latency percentile by 5%.
> Warning: there are likely some race conditions, but I'm posting it here
> because there is may be an implementation available somewhere we can use, or
> a good reason not to do that. So feedback welcome as usual.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
