924060929 commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4128749721
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java:
##########
@@ -1268,6 +1322,43 @@ public void
retireAllDatabaseObjectsWithoutEngineInvalidation() {
metaCache.invalidateObjects();
} finally {
invalidateEngineCacheOnDatabaseRemoval.remove();
+ // The removal callbacks above suppress every per-database
row-count fence, so publish
+ // one catalog-wide fence here; otherwise a recreated same-name
object can reuse a stale
+ // count through its deterministic table id.
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateRowCountCache(getId());
Review Comment:
Fixed in 82403079607. The shared catalog object-retirement helper now opens
a catalog row-count fence before `invalidateObjects` and keeps its completion
fence. Added a latch test at the database-object swap.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java:
##########
@@ -619,15 +640,36 @@ public boolean unregisterTableForReplay(String tableName)
{
// Check whether the table still exists in the local replay cache.
ExternalTable dorisTable = getTableForReplay(tableName).orElse(null);
if (dorisTable == null) {
+ // A mode-2 table-name mapping can disappear while the old object
stays resident in
+ // MetaCache.metaObjCache; getTableForReplay then misses only
because the mapping is
+ // absent. Retire the hidden table-object generation so a
same-name recreation cannot
+ // reuse the prior incarnation, then widen the engine and
row-count fence.
+ if (!hasLocalTableName(tableName)) {
+ retireAllTableObjectsWithoutEngineInvalidation();
Review Comment:
Fixed in 82403079607. The shared
`retireAllTableObjectsWithoutEngineInvalidation` helper now fences that DB
before and after swapping its table-object generation. Added a latch test at
the unresolved-table swap.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/PaimonExternalMetaCache.java:
##########
@@ -150,7 +150,10 @@ public void invalidateTable(ExternalTable table) {
@Override
public void invalidateDb(long catalogId, String dbName) {
try {
- invalidatePaimonTables(catalogId, nameMapping ->
matchDb(nameMapping, dbName));
+ // The SDK cache can be populated without any Doris table entry
(for example by a
+ // metadata TVF). A name-only database invalidation cannot recover
the remote name
+ // from an evicted database object, so fence the SDK catalog
independently.
+ tableLoader.invalidateCatalog(catalogId);
Review Comment:
Fixed in 82403079607. Name-based Paimon invalidation now matches SDK entries
by the affected local DB name, including SDK-only cold entries, and preserves
unrelated DB table/partition caches. Explicit DROP routes engine invalidation
once instead of duplicating a warm typed callback; the two-DB batch-cache and
DROP tests pass.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalCatalog.java:
##########
@@ -1247,10 +1273,38 @@ public void unregisterDatabase(String dbName) {
if (LOG.isDebugEnabled()) {
LOG.debug("unregister database [{}]", dbName);
}
- if (isInitialized()) {
- metaCache.invalidate(dbName, Util.genIdByName(name, dbName));
+ if (!isInitialized()) {
+ Env.getCurrentEnv().getExtMetaCacheMgr().invalidateDb(getId(),
dbName);
+ return;
}
- Env.getCurrentEnv().getExtMetaCacheMgr().invalidateDb(getId(), dbName);
+ String localDbName = getLocalDatabaseName(dbName, true);
+ if (localDbName == null) {
+ // A mode-2 remote-to-local mapping can disappear (for example
after a names refresh)
+ // while the resident database object survives. The canonical key
is then unknown, so
+ // treat the scope as unknown: retire every cached database object
and flush the engine
+ // caches and row counts catalog-wide instead of evicting the
wrong local key.
+ retireAllDatabaseObjectsWithoutEngineInvalidation();
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateCatalog(getId());
+ return;
+ }
+ long dbId = Util.genIdByName(name, localDbName);
+ metaCache.invalidate(localDbName, dbId);
Review Comment:
Fixed in 82403079607. `unregisterDatabase` now fences the known DB ID before
`metaCache.invalidate` releases its name slot, and the explicit routed
invalidation retains the completion fence. Added a cold-DROP latch test at name
removal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
