[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15634357#comment-15634357 ] Gary Helmling commented on HBASE-10656: --- Yes, it does call reset() after each iteration, because that simulates the actual usage in our metrics code. Benchmarking against a long lived histogram does not represent what actually happens on a live regionserver. If the argument is that we should not be using Counter in histograms, that something we can look at. Again, benchmarks on HBASE-16146 show HdrHistogram implementations outperforming all others. The benchmark there also shows that Counter remains much better than AtomicLong. Where do you see it now performing worse? > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621076#comment-15621076 ] Hiroshi Ikeda commented on HBASE-10656: --- bq. No. AtomicLong is pretty bad for metrics usage. I meant the patch of HBASE-16146 makes Counter worse than AtomicLonger. Moreover, if the usage frequently creates and drops instances, AtomicLonger might give a good result, especially against Counter with ThreadLocal per instance. I suspect the benchmark MultiHistogramBenchmark in HBASE-16146 calls FastLongHistogram.reset in its measurement, but I failed to find its source code. > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587767#comment-15587767 ] Hiroshi Ikeda commented on HBASE-10656: --- - The Counter is not intended to be frequently created and discarded into garbage. There seems many abuses :P - The javadoc of sun.misc.Contended in Java 8 says: "The effects of this annotation will nearly always add significant space overhead to objects." It seems VM implementations will automatically add pads for you and abusing LongAdder will still cause the same memory issue. - As I wrote in HBASE-16616, it would be good to change indexHolderThreadLocal to be static, not only to avoid frequently calling ThreadLocalMap.expungeStaleEntry, also to avoid frequently calling ThreadLocalMap.getEntryAfterMiss, which seems the cause of HBASE-16146 according to comments, but still counters require memory overhead and should not be abused. - Revert HBASE-16146 or it would be better to replace all counters with AtomicLong. Cache line contention is difficult to be detected on the language level and we should stretch a net to catch the sign by chance through many cache line contentions occur, even though the degradation is visible, and it wastes to use the sign just once, throw away, and continue to suffer further contentions. - I don't like to remove pads and pray :P > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587208#comment-15587208 ] Misha Dmitriev commented on HBASE-10656: Correct - no user activity. Though you may be right that the monitoring tool does something inside HBase. > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587191#comment-15587191 ] stack commented on HBASE-10656: --- [~mi...@cloudera.com] There was no load on the cluster, right? > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[
https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587146#comment-15587146
]
Misha Dmitriev commented on HBASE-10656:
To be more specific, our RegionServers end up with millions of Counter$Cell
objects in memory:
{code}
#instances Shallow size #instancesShallow size Class name
garbage garbage live live
2,985,951 396,571K (32.8%) 766,919101,856K (8.4%)
org.apache.hadoop.hbase.util.Counter$Cell
2,985,949 69,983K (5.8%) 766,918 17,974K (1.5%)
org.apache.hadoop.hbase.util.Counter$Cell[]
{/code}
I think there is no point in talking about optimizations where we forcefully
prevent two Cell objects from sharing a single cache line where there are so
many of them that they just cause memory blowup.
A simple way of solving this problem would be to just remove the extra padding
long fields. However, I am totally new to HBase and don't know whether a large
number of these objects is always the case. Maybe in some setups there are very
few of them. In that case, maybe it would make sense to have two alternative
implementations of Cell:
- one that assumes a small number of objects and optimized for cache speed, as
now
- another that's just as compact as possible.
> high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
>
>
> Key: HBASE-10656
> URL: https://issues.apache.org/jira/browse/HBASE-10656
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
> Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt,
> HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch,
> MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java,
> MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java,
> output.pdf, output.txt, output2.pdf, output2.txt
>
>
> Cliff's high-scale-lib's Counter is used in important classes (for example,
> HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation
> detail of the Java standard library and belongs to Oracle (Sun). That
> consequently makes HBase depend on the specific JRE Implementation.
> To make matters worse, Counter has a bug and you may get wrong result if you
> mix a reading method into your logic calling writing methods.
> In more detail, I think the bug is caused by reading an internal array field
> without resolving memory caching, which is intentional the comment says, but
> storing the read result into a volatile field. That field may be not changed
> after you can see the true values of the array field, and also may be not
> changed after updating the "next" CAT instance's values in some race
> condition when extending CAT instance chain.
> Anyway, it is possible that you create a new alternative class which only
> depends on the standard library. I know Java8 provides its alternative, but
> HBase should support Java6 and Java7 for some time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587121#comment-15587121 ] Enis Soztutar commented on HBASE-10656: --- Great. Looked at the code for Striped64, it uses Contended annotation though which is Java-8 only AFAIK. So we have have to do the same trickery with our own padding, thus suffering from the same GC problem above. > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[
https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586960#comment-15586960
]
Andrew Purtell commented on HBASE-10656:
Public domain, category A, provided we use the JSR166 sources not what was
donated to OpenJDK and relicensed as GPL 2 (category X)
http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/jsr166e/LongAdder.java?view=co
{noformat}
/*
* Written by Doug Lea with assistance from members of JCP JSR-166
* Expert Group and released to the public domain, as explained at
* http://creativecommons.org/publicdomain/zero/1.0/
*/
{noformat}
> high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
>
>
> Key: HBASE-10656
> URL: https://issues.apache.org/jira/browse/HBASE-10656
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
> Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt,
> HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch,
> MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java,
> MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java,
> output.pdf, output.txt, output2.pdf, output2.txt
>
>
> Cliff's high-scale-lib's Counter is used in important classes (for example,
> HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation
> detail of the Java standard library and belongs to Oracle (Sun). That
> consequently makes HBase depend on the specific JRE Implementation.
> To make matters worse, Counter has a bug and you may get wrong result if you
> mix a reading method into your logic calling writing methods.
> In more detail, I think the bug is caused by reading an internal array field
> without resolving memory caching, which is intentional the comment says, but
> storing the read result into a volatile field. That field may be not changed
> after you can see the true values of the array field, and also may be not
> changed after updating the "next" CAT instance's values in some race
> condition when extending CAT instance chain.
> Anyway, it is possible that you create a new alternative class which only
> depends on the standard library. I know Java8 provides its alternative, but
> HBase should support Java6 and Java7 for some time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586946#comment-15586946 ] Enis Soztutar commented on HBASE-10656: --- bq. Master / 2.0 can use LongAdder, available in JRE 8 We already do so. bq. I don't have a ready answer for 1.x or 0.98 I was suggesting in HBASE-16146 that we can do a backport of LongAdder in HBase. I did not check the license implications though. > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586863#comment-15586863 ] Andrew Purtell commented on HBASE-10656: Master / 2.0 can use LongAdder, available in JRE 8: https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/LongAdder.html . I don't have a ready answer for 1.x or 0.98 (although the latter may be retired soon). > high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug > > > Key: HBASE-10656 > URL: https://issues.apache.org/jira/browse/HBASE-10656 > Project: HBase > Issue Type: Bug >Reporter: Hiroshi Ikeda >Assignee: Hiroshi Ikeda >Priority: Minor > Fix For: 0.96.2, 0.98.1, 0.99.0 > > Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, > HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, > MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, > MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, > output.pdf, output.txt, output2.pdf, output2.txt > > > Cliff's high-scale-lib's Counter is used in important classes (for example, > HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation > detail of the Java standard library and belongs to Oracle (Sun). That > consequently makes HBase depend on the specific JRE Implementation. > To make matters worse, Counter has a bug and you may get wrong result if you > mix a reading method into your logic calling writing methods. > In more detail, I think the bug is caused by reading an internal array field > without resolving memory caching, which is intentional the comment says, but > storing the read result into a volatile field. That field may be not changed > after you can see the true values of the array field, and also may be not > changed after updating the "next" CAT instance's values in some race > condition when extending CAT instance chain. > Anyway, it is possible that you create a new alternative class which only > depends on the standard library. I know Java8 provides its alternative, but > HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[
https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586749#comment-15586749
]
stack commented on HBASE-10656:
---
[~ikeda] Here is an interesting observation by a coworker
[~mi...@cloudera.com]. I can open new issue to discuss but posting here for
moment:
{quote}
To induce high load on MONITORING TOOL in my small 8-machine cluster, V
suggested to create 10 hbase tables with 1K regions each - in this way,
MONITORING TOOL gets 10K new entities to monitor. I've done that and it worked
for MONITORING TOOL as expected. However, one thing that we noticed is that
HBase Region Servers in my cluster are now constantly running GC
I decided to take a quick look, took a heap dump from one of the region servers
and analyzed it with the same tool (http://www.jxray.com) that I use in the
MONITORING TOOL work. The output is attached.
One finding is that 41% of memory is occupied by instances of
org.apache.hadoop.hbase.util.Counter$Cell class, and they seem to be actively
"churned" by GC all the time. I looked at the code of this class, and one thing
that immediately caught my eye is this:
private static class Cell {
// Pads are added around the value to avoid cache-line contention with
// another cell's value. The cache-line size is expected to be equal to or
// less than about 128 Bytes (= 64 Bits * 16).
@SuppressWarnings("unused")
volatile long p0, p1, p2, p3, p4, p5, p6;
volatile long value;
@SuppressWarnings("unused")
volatile long q0, q1, q2, q3, q4, q5, q6;
So, as far as I understand, the only meaningful data field in this class,
'value', is deliberately "padded" with empty fields just to make an instance of
this class big enough to fit the entire 128-byte cache line.
This looks like a very extreme optimization that would work if there were very
few objects in memory, or at least very few of Counter$Cell instances, so that
they were kept in the cache all the time. But clearly in our case making these
objects artificially large greatly increases the GC pressure and ultimately
makes everything much slower.
Can somebody shed some light on this? In particular:
- Why do so many Counter instances are created and destroyed all the time
despite the fact that there is no HBase activity going on?
- I don't think the setup with 10K regions is very unconventional. If so many
Cell objects need to be maintained, then probably it's worth providing e.g.
another implementation that's simply optimized for size rather than for memory
cache performance?
{quote}
On Question #1, it is probably our metrics accounting that is going on. On #2,
you might have input.
> high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
>
>
> Key: HBASE-10656
> URL: https://issues.apache.org/jira/browse/HBASE-10656
> Project: HBase
> Issue Type: Bug
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Fix For: 0.96.2, 0.98.1, 0.99.0
>
> Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt,
> HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch,
> MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java,
> MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java,
> output.pdf, output.txt, output2.pdf, output2.txt
>
>
> Cliff's high-scale-lib's Counter is used in important classes (for example,
> HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation
> detail of the Java standard library and belongs to Oracle (Sun). That
> consequently makes HBase depend on the specific JRE Implementation.
> To make matters worse, Counter has a bug and you may get wrong result if you
> mix a reading method into your logic calling writing methods.
> In more detail, I think the bug is caused by reading an internal array field
> without resolving memory caching, which is intentional the comment says, but
> storing the read result into a volatile field. That field may be not changed
> after you can see the true values of the array field, and also may be not
> changed after updating the "next" CAT instance's values in some race
> condition when extending CAT instance chain.
> Anyway, it is possible that you create a new alternative class which only
> depends on the standard library. I know Java8 provides its alternative, but
> HBase should support Java6 and Java7 for some time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946169#comment-13946169 ] stack commented on HBASE-10656: --- Committed HBASE-10656-addition.patch over in HBASE-10822 Thanks [~ikeda] high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, output.pdf, output.txt, output2.pdf, output2.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13946173#comment-13946173 ] Hudson commented on HBASE-10656: FAILURE: Integrated in HBase-TRUNK #5037 (See [https://builds.apache.org/job/HBase-TRUNK/5037/]) HBASE-10822 Thread local addendum to HBASE-10656 Counter (stack: rev 1581141) * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Counter.java high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, output.pdf, output.txt, output2.pdf, output2.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13944745#comment-13944745 ] Hiroshi Ikeda commented on HBASE-10656: --- Yes, I mean I want to apply the additional patch (HBASE-10656-addition.patch), which reflects MyCounter3.java. I believe the performance cost of ThreadLocal is usually ignorable for usual meaningful logic and unpredictably consuming memory is more serious, but if there is anyone who doubt it, strictly speaking, measuring is required. high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, output.pdf, output.txt, output2.pdf, output2.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13942838#comment-13942838 ] Hiroshi Ikeda commented on HBASE-10656: --- Sorry I just guess it by some trials of my counters with replacing ID and hashcode, and I don't have practical scale HBase for my tests. If in actual usages in HBase the performance overhead of ThreadLocal is not so apparent, I prefer the last counter because of its durability in general situtations in the current and near future HBase. Even if the measuerd consumed memory is low, it may still depend on situations and luck (though using the sequential ID gives more adaptability to restricted situations). high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, output.pdf, output.txt, output2.pdf, output2.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13943339#comment-13943339 ] stack commented on HBASE-10656: --- bq. ... I prefer the last counter You mean MyCounter3.java? I can try it... high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, output.pdf, output.txt, output2.pdf, output2.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13941918#comment-13941918 ] stack commented on HBASE-10656: --- bq. That's cheat! Smile Thank you for the continued investigations [~ikeda] bq. I think high-scale-lib's Counter usually consume more memory than the new Counter This is interesting. Have you looked at how much is consumed in an actual hbase deploy? Can I check for you? high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-addition.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounter3.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, PerformanceTestApp2.java, output.pdf, output.txt, output2.pdf, output2.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13936362#comment-13936362 ] Hudson commented on HBASE-10656: SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #119 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/119/]) HBASE-10656 high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug (stack: rev 1577759) * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Counter.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestCounter.java * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/ExactCounterMetric.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/trunk/pom.xml * /hbase/trunk/src/main/docbkx/developer.xml high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, output.pdf, output.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13935902#comment-13935902 ] Hudson commented on HBASE-10656: FAILURE: Integrated in hbase-0.96-hadoop2 #241 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/241/]) HBASE-10656 high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug (stack: rev 1577760) * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Counter.java * /hbase/branches/0.96/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestCounter.java * /hbase/branches/0.96/hbase-server/pom.xml * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/ExactCounterMetric.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/branches/0.96/pom.xml high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, output.pdf, output.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13935935#comment-13935935 ] Hudson commented on HBASE-10656: FAILURE: Integrated in hbase-0.96 #350 (See [https://builds.apache.org/job/hbase-0.96/350/]) HBASE-10656 high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug (stack: rev 1577760) * /hbase/branches/0.96/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Counter.java * /hbase/branches/0.96/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestCounter.java * /hbase/branches/0.96/hbase-server/pom.xml * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/ExactCounterMetric.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/branches/0.96/pom.xml high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, output.pdf, output.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13935956#comment-13935956 ] Hudson commented on HBASE-10656: FAILURE: Integrated in HBase-TRUNK #5013 (See [https://builds.apache.org/job/HBase-TRUNK/5013/]) HBASE-10656 high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug (stack: rev 1577759) * /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/util/Counter.java * /hbase/trunk/hbase-common/src/test/java/org/apache/hadoop/hbase/util/TestCounter.java * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/metrics/ExactCounterMetric.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionTool.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java * /hbase/trunk/pom.xml * /hbase/trunk/src/main/docbkx/developer.xml high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Assignee: Hiroshi Ikeda Priority: Minor Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: 10656-098.v2.txt, 10656-trunk.v2.patch, 10656.096v2.txt, HBASE-10656-0.96.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, output.pdf, output.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[
https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925593#comment-13925593
]
Hiroshi Ikeda commented on HBASE-10656:
---
bq. Am I reading your table correctly? It says your implementation consistently
faster than cliff click lib and AtomicInteger?
Yes, the table says my counter works well.
For increment, comparing to high-scale-lib, the newly created counter has just
a bit advantage, which is coming from simplicity of logic, and just meaningful
in micro-bench, I think.
For getting a value, high-sacle-lib's counter forgets to skip pads to sum up
values (almost bug). Using a volatile instance variable to cache the sum may
also causes overhead, but anyway, high-scale-lib's counter gives a wrong value
so that I don't want to use it.
{quote}
What happens if the below does no longer holds?
+ // ...The cache-line size is expected to be equal to or
+ // less than about 128 Bytes (= 64 Bits * 16).
{quote}
By the previously uploaded result of my performance test, for AtomicLong, using
more threads requires more time. It starts contention using 2 threads, with
gradually increasing to 8 threads, which causes full contention because my
environment has 8 core.
The same overhead may be occurred for the newly created counter if the actual
cache line is larger than the size I expected.
It seems that only just contention with 2 threads may be relatively large. It
is possible that the performance drastically reduces.
However, in the first place, the effect of adding pads basically depends on the
implementation of JRE. I referred the source code of LongAdder and added pads
like it, and I think it is practical because the LongAdder is written by an
expert, but logically VM may eliminate that unused pads under the name of
optimization.
Also adding pads can waste memory. The newly created counter uses about 16 long
variables for one cell, and supposing the internal array requires 8 length to
avoid contention, it requires the same memory as 128 long variables per a
counter. I think it is already sufficiently done to waste memory.
bq. Looking at LongAdder, its public domain so copying from it is fine (I was
concerned about license issues).
I just borrow a trick of padding and copypaste names of pad variables (that
are never used and any names will go) from LongAdder. There are the other ideas
in LongAdder, but it depends on VM implementation in the end, and performance
costs of the ticks didn't seem trivial comparing to the main logic (just
increment), so I just give priority to make my counter simple. It may be
required to revise if it is too rough.
high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
Key: HBASE-10656
URL: https://issues.apache.org/jira/browse/HBASE-10656
Project: HBase
Issue Type: Bug
Reporter: Hiroshi Ikeda
Priority: Minor
Attachments: HBASE-10656-0.96.patch, HBASE-10656-trunk.patch,
MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java,
PerformanceTestApp.java, output.pdf, output.txt
Cliff's high-scale-lib's Counter is used in important classes (for example,
HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation
detail of the Java standard library and belongs to Oracle (Sun). That
consequently makes HBase depend on the specific JRE Implementation.
To make matters worse, Counter has a bug and you may get wrong result if you
mix a reading method into your logic calling writing methods.
In more detail, I think the bug is caused by reading an internal array field
without resolving memory caching, which is intentional the comment says, but
storing the read result into a volatile field. That field may be not changed
after you can see the true values of the array field, and also may be not
changed after updating the next CAT instance's values in some race
condition when extending CAT instance chain.
Anyway, it is possible that you create a new alternative class which only
depends on the standard library. I know Java8 provides its alternative, but
HBase should support Java6 and Java7 for some time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[
https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923818#comment-13923818
]
Hadoop QA commented on HBASE-10656:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12633340/HBASE-10656-trunk.patch
against trunk revision .
ATTACHMENT ID: 12633340
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8921//console
This message is automatically generated.
high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
Key: HBASE-10656
URL: https://issues.apache.org/jira/browse/HBASE-10656
Project: HBase
Issue Type: Bug
Reporter: Hiroshi Ikeda
Priority: Minor
Attachments: HBASE-10656-0.96.patch, HBASE-10656-trunk.patch,
MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java,
PerformanceTestApp.java, output.pdf, output.txt
Cliff's high-scale-lib's Counter is used in important classes (for example,
HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation
detail of the Java standard library and belongs to Oracle (Sun). That
consequently makes HBase depend on the specific JRE Implementation.
To make matters worse, Counter has a bug and you may get wrong result if you
mix a reading method into your logic calling writing methods.
In more detail, I think the bug is caused by reading an internal array field
without resolving memory caching, which is intentional the comment says, but
storing the read result into a volatile field. That field may be not changed
after you can see the true values of the array field, and also may be not
changed after updating the next CAT instance's values in some race
condition when extending CAT instance chain.
Anyway, it is possible that you create a new alternative class which only
depends on the standard library. I know Java8 provides its alternative, but
HBase should support Java6 and Java7 for some time.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924460#comment-13924460 ] stack commented on HBASE-10656: --- bq. ... because I have no environment to build HBase It looks like you did a good job hand-writing your patch given all tests pass. Am I reading your table correctly? It says your implementation consistently faster than cliff click lib and AtomicInteger? What happens if the below does no longer holds? +// ...The cache-line size is expected to be equal to or +// less than about 128 Bytes (= 64 Bits * 16). We will lose the cache line advantage and stuff will slow? I did a cursory review. This looks great. Looking at LongAdder, its public domain so copying from it is fine (I was concerned about license issues). high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: HBASE-10656-0.96.patch, HBASE-10656-trunk.patch, MyCounter.java, MyCounter2.java, MyCounterTest.java, MyCounterTest.java, PerformanceTestApp.java, output.pdf, output.txt Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13919111#comment-13919111 ] Andrew Purtell commented on HBASE-10656: bq. We should have a fallback implementation as you suggest. +1 high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: MyCounter.java, MyCounterTest.java Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13919140#comment-13919140 ] Hiroshi Ikeda commented on HBASE-10656: --- bq. MyCounter.Cat.CACHE_LINE_SCALE to 4 Sorry, MyCounter.Cat.CACHE_LINE_SCALE to 3 is correct because 8 = 1 3 high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: MyCounter.java, MyCounterTest.java, MyCounterTest.java Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918328#comment-13918328 ] stack commented on HBASE-10656: --- Nice work [~ikeda] We use Unsafe in other areas of the code base too so a total purge would take more than just our undoing use of high-scale-lib counters. Are we susceptible to the bug you've identified? Do we do the write while read w/o protection? Are these high-scale counters used for metrics only or for more critical countings? I've not looked. Thanks you [~ikeda] for digging in here. high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: MyCounter.java, MyCounterTest.java Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918335#comment-13918335 ] Nicolas Liochon commented on HBASE-10656: - Is there any reference to this bug in the high-scale-lib repo? high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: MyCounter.java, MyCounterTest.java Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13918968#comment-13918968 ] Hiroshi Ikeda commented on HBASE-10656: --- [~stack] bq. We use Unsafe in other areas of the code base too so a total purge would take more than just our undoing use of high-scale-lib counters. I know org.apache.hadoop.hbase.util.Bytes uses Unsafe but it swithces logic in runtime so that Oracle JDK is indispensable only for build time. On the contrary, high-scale-lib crachs in runtime when it fails to find Unsafe. I don't find any things like this in code of HBase, though I'm not sure about the other 3rd party libraries. bq. Are we susceptible to the bug you've identified? I worry a little about HBaseServer using the Counter for some threshold, but by and large I agree that the effect of the bug itself doesn't seem so serious. bq. Do we do the write while read w/o protection? Without locks, reading the count is just only weakly consisntent like ConcurrentHashMap.size(). While there are threads chainging the count, the count read is just an estimate, but if there are no such threads while reading the count it should give the correct count at the time. [~nkeywal] bq. Is there any reference to this bug in the high-scale-lib repo? I just found the bug (or just buggy specification that allows losing counts?) when I used it by way of experiment, and I didn't so hardly search information about it in web, because I think the dependency to Oracle JRE is a much more substantial problem. I think this is not listed up on the bug tickets in the sourceforge's high-scale-lib page. high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: MyCounter.java, MyCounterTest.java Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10656) high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug
[ https://issues.apache.org/jira/browse/HBASE-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13919004#comment-13919004 ] stack commented on HBASE-10656: --- bq. On the contrary, high-scale-lib crachs in runtime when it fails to find Unsafe. We should have a fallback implementation as you suggest. high-scale-lib's Counter depends on Oracle (Sun) JRE, and also has some bug Key: HBASE-10656 URL: https://issues.apache.org/jira/browse/HBASE-10656 Project: HBase Issue Type: Bug Reporter: Hiroshi Ikeda Priority: Minor Attachments: MyCounter.java, MyCounterTest.java Cliff's high-scale-lib's Counter is used in important classes (for example, HRegion) in HBase, but Counter uses sun.misc.Unsafe, that is implementation detail of the Java standard library and belongs to Oracle (Sun). That consequently makes HBase depend on the specific JRE Implementation. To make matters worse, Counter has a bug and you may get wrong result if you mix a reading method into your logic calling writing methods. In more detail, I think the bug is caused by reading an internal array field without resolving memory caching, which is intentional the comment says, but storing the read result into a volatile field. That field may be not changed after you can see the true values of the array field, and also may be not changed after updating the next CAT instance's values in some race condition when extending CAT instance chain. Anyway, it is possible that you create a new alternative class which only depends on the standard library. I know Java8 provides its alternative, but HBase should support Java6 and Java7 for some time. -- This message was sent by Atlassian JIRA (v6.2#6252)
