Misha Dmitriev commented on HBASE-10656:

To be more specific, our RegionServers end up with millions of Counter$Cell 
objects in memory:

 #instances     Shallow size     #instances    Shallow size       Class name
  garbage       garbage           live         live                         
 2,985,951   396,571K (32.8%)      766,919    101,856K (8.4%)     
 2,985,949     69,983K (5.8%)      766,918     17,974K (1.5%)     

I think there is no point in talking about optimizations where we forcefully 
prevent two Cell objects from sharing a single cache line where there are so 
many of them that they just cause memory blowup.

A simple way of solving this problem would be to just remove the extra padding 
long fields. However, I am totally new to HBase and don't know whether a large 
number of these objects is always the case. Maybe in some setups there are very 
few of them. In that case, maybe it would make sense to have two alternative 
implementations of Cell:
 - one that assumes a small number of objects and optimized for cache speed, as 
 - another that's just as compact as possible.

>  high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
> ----------------------------------------------------------------------------
>                 Key: HBASE-10656
>                 URL: https://issues.apache.org/jira/browse/HBASE-10656
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Hiroshi Ikeda
>            Assignee: Hiroshi Ikeda
>            Priority: Minor
>             Fix For: 0.96.2, 0.98.1, 0.99.0
>         Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, 
> HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, 
> MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, 
> MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, 
> output.pdf, output.txt, output2.pdf, output2.txt
> Cliff's high-scale-lib's Counter is used in important classes (for example, 
> HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation 
> detail of the Java standard library and belongs to Oracle (Sun). That 
> consequently makes HBase depend on the specific JRE Implementation.
> To make matters worse, Counter has a bug and you may get wrong result if you 
> mix a reading method into your logic calling writing methods.
> In more detail, I think the bug is caused by reading an internal array field 
> without resolving memory caching, which is intentional the comment says, but 
> storing the read result into a volatile field. That field may be not changed 
> after you can see the true values of the array field, and also may be not 
> changed after updating the "next" CAT instance's values in some race 
> condition when extending CAT instance chain.
> Anyway, it is possible that you create a new alternative class which only 
> depends on the standard library. I know Java8 provides its alternative, but 
> HBase should support Java6 and Java7 for some time.

This message was sent by Atlassian JIRA

Reply via email to