CalvinKirs commented on code in PR #66717:
URL: https://github.com/apache/doris/pull/66717#discussion_r3826927096
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/AbstractExternalMetaCache.java:
##########
@@ -283,23 +403,53 @@ private <K, V> void
invalidateEntryIfMatched(CatalogEntryGroup group, MetaCacheE
}
}
- private CatalogEntryGroup buildCatalogEntryGroup(Map<String, String>
catalogProperties) {
+ private CatalogEntryGroup buildCatalogEntryGroup(long catalogId,
Map<String, String> catalogProperties) {
CatalogEntryGroup group = new CatalogEntryGroup();
- metaCacheEntryDefs.values()
- .forEach(entryDef -> group.put(entryDef.getName(),
newMetaCacheEntry(entryDef, catalogProperties)));
- return group;
+ try {
+ metaCacheEntryDefs.values().forEach(entryDef -> group.put(
+ entryDef.getName(), newMetaCacheEntry(catalogId, entryDef,
catalogProperties)));
+ return group;
+ } catch (RuntimeException | Error e) {
+ group.close();
+ throw e;
+ }
}
@SuppressWarnings("unchecked")
private <K, V> MetaCacheEntry<K, V> newMetaCacheEntry(
- MetaCacheEntryDef<?, ?> rawEntryDef, Map<String, String>
catalogProperties) {
+ long catalogId, MetaCacheEntryDef<?, ?> rawEntryDef, Map<String,
String> catalogProperties) {
MetaCacheEntryDef<K, V> entryDef = (MetaCacheEntryDef<K, V>)
rawEntryDef;
CacheSpec cacheSpec = CacheSpec.fromProperties(
catalogProperties, engine, entryDef.getName(),
entryDef.getDefaultCacheSpec());
- return new MetaCacheEntry<>(entryDef.getName(),
- wrapSchemaValidator(entryDef.getLoader(),
entryDef.getValueType()),
- cacheSpec,
- refreshExecutor, entryDef.isAutoRefresh(),
entryDef.isContextualOnly());
+ OptionalLong catalogMaxWeight =
budgetManager.parseCatalogMaxWeight(catalogProperties);
+ if (cacheSpec.isWeightBounded() && entryDef.getSizeEstimator() ==
null) {
+ throw new IllegalArgumentException(String.format(
+ "Entry '%s' for engine '%s' configures max-weight but has
no estimator.",
+ entryDef.getName(), engine));
+ }
+ boolean enableWeight = entryDef.getSizeEstimator() != null
Review Comment:
Schema and view entries are count- and TTL-bounded by design in this PR: the
weight budget governs the dominant retained graphs (table generations, snapshot
projections, manifests), matching the converged coarse-governance scope
reviewed in the architecture follow-up. Extending byte estimators to every
remaining entry type widens the estimator surface this PR just reduced, so we
prefer to keep that as an explicit follow-up if operational data shows schema
entries matter at scale. The dimensions this thread lists (converted
Column/Type graphs, view definitions) are bounded per entry by the existing
capacity/TTL limits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
