[
https://issues.apache.org/jira/browse/PHOENIX-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17630663#comment-17630663
]
ASF GitHub Bot commented on PHOENIX-6761:
-----------------------------------------
palashc commented on code in PR #1506:
URL: https://github.com/apache/phoenix/pull/1506#discussion_r1017227237
##########
phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaDataImpl.java:
##########
@@ -46,36 +48,52 @@ public class PMetaDataImpl implements PMetaData {
private PMetaDataCache metaData;
private final TimeKeeper timeKeeper;
private final PTableRefFactory tableRefFactory;
+ private final long updateCacheFrequency;
private HashMap<String, PTableKey> physicalNameToLogicalTableMap = new
HashMap<>();
+ private final Object metricsLock = new Object();
- public PMetaDataImpl(int initialCapacity, ReadOnlyProps props) {
- this(initialCapacity, TimeKeeper.SYSTEM, props);
+ public PMetaDataImpl(int initialCapacity, long updateCacheFrequency,
ReadOnlyProps props) {
+ this(initialCapacity, updateCacheFrequency, TimeKeeper.SYSTEM, props);
}
- public PMetaDataImpl(int initialCapacity, TimeKeeper timeKeeper,
ReadOnlyProps props) {
+ public PMetaDataImpl(int initialCapacity, long updateCacheFrequency,
TimeKeeper timeKeeper, ReadOnlyProps props) {
+
this(new PMetaDataCache(initialCapacity, props.getLong(
QueryServices.MAX_CLIENT_METADATA_CACHE_SIZE_ATTRIB,
- QueryServicesOptions.DEFAULT_MAX_CLIENT_METADATA_CACHE_SIZE),
timeKeeper,
- PTableRefFactory.getFactory(props)), timeKeeper,
PTableRefFactory.getFactory(props));
+ QueryServicesOptions.DEFAULT_MAX_CLIENT_METADATA_CACHE_SIZE),
timeKeeper),
+ timeKeeper, PTableRefFactory.getFactory(props),
+ updateCacheFrequency);
}
- private PMetaDataImpl(PMetaDataCache metaData, TimeKeeper timeKeeper,
PTableRefFactory tableRefFactory) {
+ private PMetaDataImpl(PMetaDataCache metaData, TimeKeeper timeKeeper,
+ PTableRefFactory tableRefFactory, long
updateCacheFrequency) {
this.timeKeeper = timeKeeper;
this.metaData = metaData;
this.tableRefFactory = tableRefFactory;
+ this.updateCacheFrequency = updateCacheFrequency;
}
- @Override
- public PMetaDataImpl clone() {
- return new PMetaDataImpl(new PMetaDataCache(this.metaData),
this.timeKeeper, this.tableRefFactory);
+ private void updateGlobalMetric(PTableRef pTableRef) {
Review Comment:
If multiple threads share the same CQSI object, will incrementing counters
for cache access without synchronizing be correct?
> Phoenix Client Side Metadata Caching Improvement
> ------------------------------------------------
>
> Key: PHOENIX-6761
> URL: https://issues.apache.org/jira/browse/PHOENIX-6761
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Kadir Ozdemir
> Assignee: Palash Chauhan
> Priority: Major
> Attachments: PHOENIX-6761.master.initial.patch
>
>
> CQSI maintains a client-side metadata cache, i.e., schemas, tables, and
> functions, that evicts the last recently used table entries when the cache
> size grows beyond the configured size.
> Each time a Phoenix connection is created, the client-side metadata cache
> maintained by the CQSI object creating this connection is cloned for the
> connection. Thus, we have two levels of caches, one at the Phoenix connection
> level and the other at the CQSI level.
> When a Phoenix client needs to update the client side cache, it updates both
> caches (on the connection object and on the CQSI object). The Phoenix client
> attempts to retrieve a table from the connection level cache. If this table
> is not there then the Phoenix client does not check the CQSI level cache,
> instead it retrieves the object from the server and finally updates both the
> connection and CQSI level cache.
> PMetaDataCache provides caching for tables, schemas and functions but it
> maintains separate caches internally, one cache for each type of metadata.
> The cache for the tables is actually a cache of PTableRef objects. PTableRef
> holds a reference to the table object as well as the estimated size of the
> table object, the create time, last access time, and resolved time. The
> create time is set to the last access time value provided when the PTableRef
> object is inserted into the cache. The resolved time is also provided when
> the PTableRef object is inserted into the cache. Both the created time and
> resolved time are final fields (i.e., they are not updated). PTableRef
> provide a setter method to update the last access time. PMetaDataCache
> updates the last access time whenever the table is retrieved from the cache.
> The LRU eviction policy is implemented using the last access time. The
> eviction policy is not implemented for schemas and functions. The
> configuration parameter for the frequency of updating cache is
> phoenix.default.update.cache.frequency. This can be defined at the cluster or
> table level. When it is set to zero, it means cache would not be used.
> Obviously the eviction of the cache is to limit the memory consumed by the
> cache. The expected behavior is that when a table is removed from the cache,
> the table (PTableImpl) object is also garbage collected. However, this does
> not really happen because multiple caches make references to the same object
> and each cache maintains its own table refs and thus access times. This means
> that the access time for the same table may differ from one cache to another;
> and when one cache can evict an object, another cache will hold on the same
> object.
> Although individual caches implements the LRU eviction policy, the overall
> memory eviction policy for the actual table objects is more like age based
> cache. If a table is frequently accessed from the connection level caches,
> the last access time maintained by the corresponding table ref objects for
> this table will be updated. However, these updates on the access times will
> not be visible to the CQSI level cache. The table refs in the CQSI level
> cache have the same create time and access time.
> Since whenever an object is inserted into the local cache of a connection
> object, it is also inserted the cache on the CSQI object, the CQSI level
> cache will grow faster than the caches on the connection objects. When the
> cache reaches its maximum size, the newly inserted tables will result in
> evicting one of the existing tables in the cache. Since the access time of
> these tables are not updated on the CQSI level cache, it is likely that the
> table that has stayed in the cache for the longest period of time will be
> evicted (regardless of whether the same table is frequently accessed via the
> connection level caches). This obviously defeats the purpose of an LRU cache.
> Another problem with the current cache is related to the choice of its
> internal data structures and its eviction implementation. The table refs in
> the cache are maintained in a hash map which maps a table key (which is pair
> of a tenant id and table name) to a table ref. When the size of a cache (the
> total byte size of the table objects referred by the cache) reaches its
> configured limit, how much overage adding a new table would cause is
> computed. Then all the table refs in this cache are cloned into a priority
> queue as well as a new cache. This queue uses the access time to determine
> the order of its elements (i.e., table refs). The table refs that should not
> be evicted are removed from the queue, which leaves the table refs to be
> evicted in the queue. Finally, the table refs left in the queue are removed
> from the new cache. The new cache replaces the old one. It clear that this is
> an expensive operation in terms of memory allocations and CPU time. The bad
> news is that when the cache reaches its limit, every insertion would likely
> cause an eviction and this expensive operation will be repeated for each such
> insertion.
> Since Phoenix connections are supposed to be short lived, maintaining a
> separate cache for each connection object and especially cloning entire cache
> content (and then pruning the entries belonging to other tenants when the
> connection is a tenant specific connection) are not justified. The cost of
> such a clone operation by itself would offset the gain of not accessing the
> CQSI level cache as the number of such accesses per connection should be
> small because of short lived Phoenix connections.
> Also the impact of Phoenix connection leaks, the connections that are not
> closed by applications and simply long lived connections will be exacerbated
> since these connections will have references to the large set of table
> objects.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
