github-actions[bot] commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4068078554
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java:
##########
@@ -1228,10 +1241,46 @@ public void unregisterDatabase(String dbName) {
if (LOG.isDebugEnabled()) {
LOG.debug("unregister database [{}]", dbName);
}
- if (isInitialized()) {
- metaCache.invalidate(dbName, Util.genIdByName(name, dbName));
+ // Resolve the canonical database object before removing it from the
local metadata cache.
+ // The row-count cache can outlive that object and must be invalidated
by its numeric id.
+ boolean catalogInitialized = isInitialized();
+ String resolvedLocalDbName = catalogInitialized ?
getLocalDatabaseName(dbName, true) : null;
+ String localDbName = resolvedLocalDbName == null ? dbName :
resolvedLocalDbName;
+ Optional<ExternalDatabase<? extends ExternalTable>> db =
catalogInitialized
+ ? metaCache.tryGetMetaObj(localDbName) : Optional.empty();
+ long dbId = db.map(ExternalDatabase::getId).orElseGet(() ->
Util.genIdByName(name, localDbName));
+ boolean hasCanonicalLocalIdentity = db.isPresent()
+ || resolvedLocalDbName != null && getLowerCaseDatabaseNames()
!= 0;
+ try {
+ if (hasCanonicalLocalIdentity) {
+ Env.getCurrentEnv().getExtMetaCacheMgr().invalidateDb(getId(),
dbId, localDbName);
+ } else {
+ Env.getCurrentEnv().getExtMetaCacheMgr().invalidateDb(getId(),
dbName);
+ }
+ } finally {
+ if (catalogInitialized) {
+ invalidatingDatabaseMetaCache.set(true);
+ try {
+ metaCache.invalidate(localDbName, dbId);
Review Comment:
[P1] Publish the DB row-count fence after removing the local DB. The
invalidateDb call above completes the only fence while this DB/table is still
resolvable, so a concurrent miss can start afterward, capture an initialized
HMS table, and compute from its retained remoteTable state. This invalidate
then suppresses row-count work in the synchronous removal callback, leaving
that load/value publishable after DROP and reusable by a same-name recreation.
This is distinct from the earlier double-invalidation thread: split routed
invalidation from row-count ownership, remove the local DB, then issue the
single DB fence, with a latching regression.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java:
##########
@@ -430,7 +434,7 @@ private void buildMetaCache() {
localDbName -> Optional.ofNullable(
buildDbForInit(null, localDbName,
Util.genIdByName(name, localDbName), logType,
true)),
- (key, value, cause) -> value.ifPresent(v ->
v.resetMetaToUninitialized()),
+ (key, value, cause) ->
handleDatabaseMetaCacheRemoval(value),
Review Comment:
[P1] Avoid a global row-count scan on ordinary DB-cache eviction. This
listener receives RemovalCause but discards it, so SIZE/EXPIRED removal calls
resetMetaToUninitialized with row-count invalidation enabled; invalidateDb then
scans up to 100,000 row-count entries under the global publication write lock.
With a 1,000-object DB cache, routine catalog churn can repeatedly block every
row-count reader and evict otherwise valid counts. The existing bulk-reset
thread only suppresses invalidateAll; pass the cause through and disable
row-count invalidation for nonsemantic eviction while retaining routed cleanup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
