[
https://issues.apache.org/jira/browse/HBASE-21097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591426#comment-16591426
]
Ted Yu commented on HBASE-21097:
--------------------------------
Interesting.
>From
>https://builds.apache.org/job/HBase%20Nightly/job/master/451/testReport/junit/org.apache.hadoop.hbase.regionserver.throttle/TestFlushWithThroughputController/health_checks___yetus_jdk8_hadoop2_checks___testFlushThroughputTuning/
> :
{code}
java.lang.AssertionError: expected:<0.0> but was:<1.2906294173808417E-6>
at
org.apache.hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController.testFlushThroughputTuning(TestFlushWithThroughputController.java:185)
{code}
The above was without change to TestFlushWithThroughputController .
When I add the following line before the assertion:
{code}
long offheap =
hbtu.getConfiguration().getLong(MemorySizeUtil.OFFHEAP_MEMSTORE_SIZE_KEY, 0);
{code}
offheap is shown to be 0. Meaning, the test was only using on-heap memory only.
>From Duo's patch:
{code}
+ accounting.getGlobalMemStoreHeapSize() * 1.0 /
accounting.getGlobalMemStoreLimitLowMark(),
{code}
The above is what getFlushPressure() uses for memType == MemoryType.HEAP case
and that is what the current test executes (since offheap size is 0).
I am not sure what the modified assertion is meant to reveal.
In my debugger, if I set breakpoint within getFlushPressure(), I would see the
following continuously printed:
{code}
2018-08-24 02:43:28,398 DEBUG [regionserver/hw13463:0.Chore.1]
throttle.PressureAwareFlushThroughputController(95): flushPressure is
8.822787000188014E-7, tune flush throughput to 10.00 MB/second
2018-08-24 02:43:31,449 DEBUG [regionserver/hw13463:0.Chore.1]
throttle.PressureAwareFlushThroughputController(95): flushPressure is
8.822787000188014E-7, tune flush throughput to 10.00 MB/second
{code}
If the 1.8159279214625702E-6 is reproducible, do you mind sharing the values
for getGlobalMemStoreHeapSize() and globalMemStoreLimitLowMark (inside
getFlushPressure).
On my Mac, they were 1280 and 1450788736, respectively.
> Flush pressure assertion may fail in testFlushThroughputTuning
> ---------------------------------------------------------------
>
> Key: HBASE-21097
> URL: https://issues.apache.org/jira/browse/HBASE-21097
> Project: HBase
> Issue Type: Test
> Reporter: Ted Yu
> Assignee: Ted Yu
> Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 21097.v1.txt, HBASE-21097.patch
>
>
> From
> https://builds.apache.org/job/PreCommit-HBASE-Build/14137/artifact/patchprocess/patch-unit-hbase-server.txt
> :
> {code}
> [ERROR]
> testFlushThroughputTuning(org.apache.hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController)
> Time elapsed: 17.446 s <<< FAILURE!
> java.lang.AssertionError: expected:<0.0> but was:<1.2906294173808417E-6>
> at
> org.apache.hadoop.hbase.regionserver.throttle.TestFlushWithThroughputController.testFlushThroughputTuning(TestFlushWithThroughputController.java:185)
> {code}
> Here is the related assertion:
> {code}
> assertEquals(0.0, regionServer.getFlushPressure(), EPSILON);
> {code}
> where EPSILON = 1E-6
> In the above case, due to margin of 2.9E-7, the assertion didn't pass.
> It seems the epsilon can be adjusted to accommodate different workload /
> hardware combination.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
