[
https://issues.apache.org/jira/browse/PHOENIX-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17530217#comment-17530217
]
ASF GitHub Bot commented on PHOENIX-6699:
-----------------------------------------
joshelser commented on code in PR #1431:
URL: https://github.com/apache/phoenix/pull/1431#discussion_r862109064
##########
phoenix-core/src/main/java/org/apache/phoenix/monitoring/MetricUtil.java:
##########
@@ -38,4 +46,22 @@ public static MetricsStopWatch getMetricsStopWatch(boolean
isRequestMetricsEnabl
return new MetricsStopWatch(true);
}
+ // We need to cover the case when JmxCacheBuster has just stopped the
HBase metrics
+ // system, and not accidentally overwrite the DefaultMetricsSystem
singleton.
+ // See PHOENIX-6699
+ public static boolean isDefaultMetricsInitialized() {
+ try {
+ MetricsSystemImpl metrics = (MetricsSystemImpl)
DefaultMetricsSystem.instance();
+ Field prefixField =
MetricsSystemImpl.class.getDeclaredField("prefix");
Review Comment:
> I don't think we have to worry about a race around prefix, AFAICT by the
time the Phoenix classes are loaded, HBase has long initialized DefaultMetrics,
and prefix has been set.
Ah, ok, I wondered if I was completely worrying about a non-issue.
> Of course, this, like many Metrics stuff, is a huge hack, we shouldn't
need to use reflection to check if DefaultMetricsSystem has been initialized
already.
Yeah, that's life with hadoop metrics2 :)
> Also, we could just to check if we are and RS/Master process, and skip the
initialization, but I don't know a way to do that.
Hrm, I think we could do that (let me see what I can dig up). What about
Geoffrey's point about the server-side metrics stuff we do have in Phoenix?
> Phoenix metrics overwriting DefaultMetricsSystem in RegionServers
> -----------------------------------------------------------------
>
> Key: PHOENIX-6699
> URL: https://issues.apache.org/jira/browse/PHOENIX-6699
> Project: Phoenix
> Issue Type: Bug
> Components: core
> Affects Versions: 5.2.0
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Major
>
> There is a race condition in the Hbase and Phoenix metrics handling.
> Hbase metrics is supposed to run in the daemon processes, while Phoenix
> metrics is supposed to run on the phoenix client side.
> On an RS both HBase and Phoenix tries to register its metrics implementation
> into DefaultMetricsSystem.
>
> On the happy path, when Phoenix tries to register its metrics on the RS, it
> won't have any effect, because HBase has already registered an active Metrics
> implementation.
> However, JmxCacheBuster stops the Hbase metrics system for ~500ms every five
> seconds, and if Phoenix tries to initialize its metrics in this period, then
> it will succeed in initializing its Metrics object, and will overwrite
> theDefaultMetricsSystem object initialized by HBase.
> This disables the HBase metrics system for the RS process.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
