[
https://issues.apache.org/jira/browse/HBASE-16440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432483#comment-15432483
]
Anoop Sam John commented on HBASE-16440:
----------------------------------------
Ok I was just trying to clean here and there and change for the reporting
thread start went in the patch. Will remove.
Its not about multi threaded single memstore flush. It is about concurrent
MSLAB close because of diff memstore flushes. All use same MSLAB pool. So more
than one thread can call putBack API with a list of chunks concurrently. Also
before the patch, u can see that the getChunk will return a new chunk even if
the pool reached its max size and no free chunks as of now. The max count
enforcement we tried to do during putBack APIs. But as that is not thread safe
we may cross.
Now the major diff is the getChunk API enforces the max capacity. It will
return chunks which can get pooled. Else it wl return null. Means pool says to
callers that there are no pooled/poolable chunks from this pool. The atomic
int was already there. That was used for reporting alone but now that is key
in deciding whether we can create new PooleChunk or not.
bq.The reclaimedChunks blockingqueue can be initialized with max Size ==
maxcount. Currently it is with no max Size so it will go upto INTEGER.MAX_VALUE.
As per current status ya we can change. But see HBASE-16407. When we do this,
the max capacity itself is changeable. So I will keep this as is.
> MemstoreChunkPool might cross its maxCount of chunks to pool
> ------------------------------------------------------------
>
> Key: HBASE-16440
> URL: https://issues.apache.org/jira/browse/HBASE-16440
> Project: HBase
> Issue Type: Sub-task
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 2.0.0
>
> Attachments: HBASE-16440.patch, HBASE-16440_V2.patch,
> HBASE-16440_V3.patch
>
>
> {code}
> void putbackChunks(BlockingQueue<Chunk> chunks) {
> int maxNumToPutback = this.maxCount - reclaimedChunks.size();
> if (maxNumToPutback <= 0) {
> return;
> }
> chunks.drainTo(reclaimedChunks, maxNumToPutback);
> // clear reference of any non-reclaimable chunks
> if (chunks.size() > 0) {
> if (LOG.isTraceEnabled()) {
> LOG.trace("Left " + chunks.size() + " unreclaimable chunks, removing
> them from queue");
> }
> chunks.clear();
> }
> }
> {code}
> There is no synchroization. 2 threads might be calling this API as part of a
> MSLAB close. (Once the memstore is flushed). It pass all the chunks used by
> it. (Those might not have been come from pool also). We try to put back
> chunks such that it is not crossing maxCount. Suppose maxCount is 10 and
> currently no chunks in 'reclaimedChunks'. Say both threads at line one. Both
> see 'maxNumToPutback ' as 10 and that will make 20 chunks being pooled.
> Similar issue is in putbackChunk(Chunk chunk) API also.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
