[
https://issues.apache.org/jira/browse/HBASE-27058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17541169#comment-17541169
]
Geoffrey Jacoby edited comment on HBASE-27058 at 5/23/22 9:01 PM:
------------------------------------------------------------------
Some more context: at a high level, what is needed is a way to _synchronously_
request a major compaction in an IT test and not proceed until after the major
compaction is done. And for this to work in a test using the
EnvironmentEdgeManager.
Since HBase doesn't provide a way to do that directly (I think it did a long
time ago on the Admin API?), Phoenix has some test utilities that use the Admin
API to request a major compaction, and then loop on
Admin.getLastMajorCompactionTimestamp waiting for it to update to confirm the
major compaction has taken place.
This is necessary because Phoenix has some coprocessor hooks that modify
compaction behavior that need to be tested. (For example, Phoenix has an option
to block major compaction from reaping deleted / overwritten / TTL expired
Cells for a certain window of time. This is necessary to keep Phoenix global
index verifications and rebuilds from being messed up by an unlucky major
compaction, because HBase has no idea that the base HBase table and index HBase
table are logically linked.)
Tests which exercise this sort of behavior usually use the
EnvironmentEdgeManager a lot, because they're testing precise timestamp
behavior and using the system clock and Thread.sleep would both be slow and
unstable.
was (Author: gjacoby):
Some more context: at a high level, what is needed is a way to _synchronously_
request a major compaction in an IT test and not proceed until after the major
compaction is done.
Since HBase doesn't provide a way to do that directly (I think it did a long
time ago on the Admin API?), Phoenix has some test utilities that use the Admin
API to request a major compaction, and then loop on
Admin.getLastMajorCompactionTimestamp waiting for it to update to confirm the
major compaction has taken place.
This is necessary because Phoenix has some coprocessor hooks that modify
compaction behavior that need to be tested. (For example, Phoenix has an option
to block major compaction from reaping deleted / overwritten / TTL expired
Cells for a certain window of time. This is necessary to keep Phoenix global
index verification and rebuilds from being messed up by an unlucky major
compaction, because HBase has no idea that the base HBase table and index HBase
table are logically linked.)
> Admin#getLastMajorCompactionTimestamp() doesn't get updated when the
> EnvironmentEdgeManager clock is stopped
> ------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-27058
> URL: https://issues.apache.org/jira/browse/HBASE-27058
> Project: HBase
> Issue Type: Bug
> Affects Versions: 2.5.0
> Reporter: Istvan Toth
> Priority: Minor
> Labels: test
>
> In Hbase 2.0-2.4 it is possible to check for a finished compaction by polling
> Admin.getLastMajorCompactionTimestamp() for the table under compaction, even
> when the clock is stopped via EnvironmentEdgeManager.
> However, in Hbase 2.5 the Admin.getLastMajorCompactionTimestamp() will not be
> updated even after the compaction is finished, and getCompactionState()
> returns NONE.
> I am not even sure that this is bug, however, this has broken one of our
> Phoenix tests, and may cause problems for others.
> This is the test code that breaks:
> [https://github.com/apache/phoenix/blob/8aa825ed88828a99d40fdb68eb2f930981cd8a6b/phoenix-core/src/test/java/org/apache/phoenix/util/TestUtil.java#L818]
> Admin.getLastMajorCompactionTimestamp() seems to take the value from the
> Metrics, so I guess that the metrics no longer get updated somewhere when the
> clock is stopped.
> I did not dig deeper than that.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
