Anastasia Braginsky commented on HBASE-18375:

Hey [~anoop.hbase],

Great minds thinks alike!
We have just realized what the problem is here (in details) and I came to write 
the comment and saw your comment :)
As [~ram_krish] has written we didn't know the true reason why this patch 
resolves the problem, so I continued to look for the "truth".

So now we have the full understanding what is going on. Here it is:
1. The Maps in ChunkCreator are referencing chunks, while Cells in CSLM and 
CellArayMap are referencing ByteBuffers. 
2. Therefore chunks that are referenced from WeakMap or SoftMap are getting 
released, however ByteBuffers are still there.
3. There is no meaning in WeakMap or SoftMap. Only StrongMap can truly save the 
chunks from GC while they are still in active CSLM. So only StrongMap really 
helps for CellChunkMap.

>From here the patch presented here resolves a true problem of pool chunks 
>being not returned from pool.
The problem exists also in Release 2.0 (!), but since the references in map 
there are soft references, which are released last, the frequency of problem to 
appear is less.

As solution I suggest to use only StrongMap for all chunks. So let me present 
another patch for this JIRA.
As solution to Release 2.0 I would suggest to bring back the chunks (from pool) 
linked list into the MSLAB instead of chunk IDs linked list.
What do you think?
bq. Are we always sure when the above is called, the weak map will have refs to 
Chunks still ?
Clearly not, the chunks are released very fast, just after they were added to 

> The pool chunks from ChunkCreator are deallocated while in pool because there 
> is no reference to them
> -----------------------------------------------------------------------------------------------------
>                 Key: HBASE-18375
>                 URL: https://issues.apache.org/jira/browse/HBASE-18375
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0-alpha-1
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>            Priority: Critical
>             Fix For: 2.0.0, 3.0.0, 2.0.0-alpha-2
>         Attachments: HBASE-18375-V01.patch, HBASE-18375-V02.patch, 
> HBASE-18375-V03.patch, HBASE-18375-V04.patch
> Because MSLAB list of chunks was changed to list of chunk IDs, the chunks 
> returned back to pool can be deallocated by JVM because there is no reference 
> to them. The solution is to protect pool chunks from GC by the strong map of 
> ChunkCreator introduced by HBASE-18010. Will prepare the patch today.

This message was sent by Atlassian JIRA

Reply via email to