[
https://issues.apache.org/jira/browse/PHOENIX-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17530094#comment-17530094
]
ASF GitHub Bot commented on PHOENIX-6699:
-----------------------------------------
joshelser commented on code in PR #1431:
URL: https://github.com/apache/phoenix/pull/1431#discussion_r861966333
##########
phoenix-core/src/main/java/org/apache/phoenix/monitoring/MetricUtil.java:
##########
@@ -38,4 +46,22 @@ public static MetricsStopWatch getMetricsStopWatch(boolean
isRequestMetricsEnabl
return new MetricsStopWatch(true);
}
+ // We need to cover the case when JmxCacheBuster has just stopped the
HBase metrics
+ // system, and not accidentally overwrite the DefaultMetricsSystem
singleton.
+ // See PHOENIX-6699
+ public static boolean isDefaultMetricsInitialized() {
+ try {
+ MetricsSystemImpl metrics = (MetricsSystemImpl)
DefaultMetricsSystem.instance();
+ Field prefixField =
MetricsSystemImpl.class.getDeclaredField("prefix");
Review Comment:
`prefix` isn't `volatile` and is set from within the `synchronized`
`init(String)` method. This would mean that we may be initialized and not yet
be able to observe it from this separate thread.
That said, I don't know how likely this race condition would be to hit.
We could also check the `monitoring` boolean instead of `prefix`. This would
mean that we already have atomic access to that variable. I am assuming that
the synchronization on the methods in MetricsSystemImpl would not affect this
reflection, meaning we would still have a (small) window where `stop()` is
being called but not yet complete.
Maybe you have already thought about this :)
> Phoenix metrics overwriting DefaultMetricsSystem in RegionServers
> -----------------------------------------------------------------
>
> Key: PHOENIX-6699
> URL: https://issues.apache.org/jira/browse/PHOENIX-6699
> Project: Phoenix
> Issue Type: Bug
> Components: core
> Affects Versions: 5.2.0
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Major
>
> There is a race condition in the Hbase and Phoenix metrics handling.
> Hbase metrics is supposed to run in the daemon processes, while Phoenix
> metrics is supposed to run on the phoenix client side.
> On an RS both HBase and Phoenix tries to register its metrics implementation
> into DefaultMetricsSystem.
>
> On the happy path, when Phoenix tries to register its metrics on the RS, it
> won't have any effect, because HBase has already registered an active Metrics
> implementation.
> However, JmxCacheBuster stops the Hbase metrics system for ~500ms every five
> seconds, and if Phoenix tries to initialize its metrics in this period, then
> it will succeed in initializing its Metrics object, and will overwrite
> theDefaultMetricsSystem object initialized by HBase.
> This disables the HBase metrics system for the RS process.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
