[
https://issues.apache.org/jira/browse/PHOENIX-5069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16736531#comment-16736531
]
ASF GitHub Bot commented on PHOENIX-5069:
-----------------------------------------
Github user BinShi-SecularBird commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/425#discussion_r245847336
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/query/GuidePostsCache.java ---
@@ -85,39 +99,47 @@ public GuidePostsCache(ConnectionQueryServices
queryServices, Configuration conf
})
// Log removals at TRACE for debugging
.removalListener(new PhoenixStatsCacheRemovalListener())
- // Automatically load the cache when entries are missing
- .build(isStatsEnabled ? new StatsLoader() : new
EmptyStatsLoader());
+ // Automatically load the cache when entries need to be
refreshed
+ .build(cacheLoader);
}
/**
- * {@link CacheLoader} implementation for the Phoenix Table Stats
cache.
+ * {@link PhoenixStatsLoader} implementation for the Stats Loader.
*/
- protected class StatsLoader extends CacheLoader<GuidePostsKey,
GuidePostsInfo> {
+ protected class StatsLoaderImpl implements PhoenixStatsLoader {
@Override
- public GuidePostsInfo load(GuidePostsKey statsKey) throws
Exception {
+ public boolean needsLoad() {
+ // Whenever it's called, we try to load stats from stats table
+ // no matter it has been updated or not.
+ return true;
+ }
+
+ @Override
+ public GuidePostsInfo loadStats(GuidePostsKey statsKey,
GuidePostsInfo prevGuidepostInfo) throws Exception {
@SuppressWarnings("deprecation")
- Table statsHTable =
queryServices.getTable(SchemaUtil.getPhysicalName(
+ TableName tableName = SchemaUtil.getPhysicalName(
PhoenixDatabaseMetaData.SYSTEM_STATS_NAME_BYTES,
- queryServices.getProps()).getName());
+ queryServices.getProps());
+ Table statsHTable =
queryServices.getTable(tableName.getName());
try {
GuidePostsInfo guidePostsInfo =
StatisticsUtil.readStatistics(statsHTable, statsKey,
HConstants.LATEST_TIMESTAMP);
traceStatsUpdate(statsKey, guidePostsInfo);
return guidePostsInfo;
} catch (TableNotFoundException e) {
// On a fresh install, stats might not yet be created,
don't warn about this.
- logger.debug("Unable to locate Phoenix stats table", e);
- return GuidePostsInfo.NO_GUIDEPOST;
+ logger.debug("Unable to locate Phoenix stats table: " +
tableName.toString(), e);
+ return prevGuidepostInfo;
} catch (IOException e) {
- logger.warn("Unable to read from stats table", e);
+ logger.warn("Unable to read from stats table: " +
tableName.toString(), e);
--- End diff --
Comment from @karanmehta93 (copied from prev PR
https://github.com/apache/phoenix/pull/416): nit: remove this comment
> Use asynchronous refresh to provide non-blocking Phoenix Stats Client Cache
> ---------------------------------------------------------------------------
>
> Key: PHOENIX-5069
> URL: https://issues.apache.org/jira/browse/PHOENIX-5069
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Bin Shi
> Assignee: Bin Shi
> Priority: Major
>
> The current Phoenix Stats Cache uses TTL based eviction policy. A cached
> entry will expire after a given amount of time (900s by default) passed since
> the entry's been created. This will lead to cache miss when
> Compiler/Optimizer fetches stats from cache at the next time. As you can see
> from the above graph, fetching stats from the cache is a blocking operation —
> when there is cache miss, it has a round trip over the wire to scan the
> SYSTEM.STATS Table and to get the latest stats info, rebuild the cache and
> finally return the stats to the Compiler/Optimizer. Whenever there is a cache
> miss, this blocking call causes significant performance penalty and see
> periodic spikes.
> *This Jira suggests to use asynchronous refresh mechanism to provide a
> non-blocking cache. For details, please see the linked design document below.*
> [~karanmehta93] [[email protected]] [~dbwong] [~elserj] [[email protected]]
> [~sergey soldatov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
