924060929 commented on code in PR #69632:
URL: https://github.com/apache/doris/pull/69632#discussion_r4385786316
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalDatabase.java:
##########
@@ -721,6 +712,70 @@ public boolean unregisterTableForReplay(String tableName) {
return true;
}
+ public void invalidateTableLookup(String tableName) {
+ if (metaCache != null) {
+ String localName =
getTableForReplay(tableName).map(ExternalTable::getName).orElse(tableName);
+ metaCache.invalidateLookup(localName,
extCatalog.getLowerCaseTableNames() != 0);
+ }
+ }
+
+ /** Historical spellings require typed engine invalidation and an
independent DB row-count fence. */
+ public void invalidateTableLookupDependencies() {
+ ExternalMetaCacheMgr cacheMgr =
Env.getCurrentEnv().getExtMetaCacheMgr();
+ long catalogId = getCatalog().getId();
+ cacheMgr.invalidateRowCountCache(catalogId, getId());
+ try {
+ cacheMgr.invalidateDb(this);
+ } finally {
+ cacheMgr.invalidateRowCountCache(catalogId, getId());
+ }
+ }
+
+ public Optional<Runnable> beginTableMetadataRefresh() {
+ return metaCache == null ? Optional.empty() :
Optional.of(metaCache.beginMetadataRefresh());
+ }
+
+ /** A missing result is authorized only against the retained DB, names and
object generations. */
+ @SuppressWarnings("unchecked")
+ public Optional<Runnable> observeTableLookup(ExternalTable table) {
+ MetaCache<T> cache = metaCache;
+ if (!initialized || cache == null
+ ||
extCatalog.shouldBypassTableNameCache(SessionContext.current())) {
+ return Optional.empty();
+ }
+ // Observation must not initialize a retired catalog before a pinned
local schema read.
+ TableMetadataLoadEpoch retainedEpoch = metadataLoadEpoch.get();
+ if
(!extCatalog.isMetadataLoadEpochCurrent(retainedEpoch.catalogEpoch)) {
+ return Optional.empty();
+ }
+ long epoch = retainedEpoch.value;
+ boolean allObjects = extCatalog.getLowerCaseTableNames() != 0;
+ Optional<Runnable> observed = cache.observeLookup(table.getName(),
table.getRemoteName(), (T) table,
+ allObjects, () -> {
+ if (allObjects) {
+ invalidateTableLookupDependencies();
+ } else {
+
Env.getCurrentEnv().getExtMetaCacheMgr().invalidateTableCache(table);
+ }
+ });
+ return observed.map(action -> () ->
MetaCacheEntry.runWithDeferredRemovalCallbacks(() -> {
+ // A parent object loader can acquire the catalog monitor while
holding its key lock.
+ // Pin that key first, then enter the same catalog -> database ->
table-cache order.
+ extCatalog.executeIfDatabaseCurrent(this, () -> {
+ synchronized (extCatalog) {
Review Comment:
Fixed in 0664ce0b4f50. Native and HMS Iceberg database-resolution helpers no
longer hold the catalog monitor across getDbNullable(). They check the expected
runtime under the monitor before and after resolution, preserving the
parent-key -> catalog order and rejecting a reset during the lookup. The test
uses the real CREATE resolution path, real parent MetaCache, a real
observeTableLookup repair callback, and another DB's publication to
capacity-evict the parent after repair has retained its key. Original source
times out; both native/HMS races now finish. Runtime-change and stale-runtime
cases verify SDK calls are rejected.
Validation: 124 focused FE cases passed; FE build and Checkstyle passed. SDK
existence is mocked; no live remote DDL or actual statistics-worker integration
claim.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java:
##########
@@ -662,6 +687,99 @@ public void invalidate(String localName, long id) {
}
}
+ /** Fences old object loads while retaining unrelated cached objects and
their ID mappings. */
+ public void invalidateLookup(String localName, boolean allObjects) {
+ metaObjLifecycleLock.writeLock().lock();
+ try {
+ synchronized (namesMutationLock) {
+ invalidateNames();
+ metaObjGeneration.incrementAndGet();
+ lookupObservationGeneration++;
+ if (allObjects) {
+ metaObjCache.invalidateAll();
+ idToName.clear();
Review Comment:
Fixed in 0664ce0b4f50. Wide targeted retirement still fences/evicts all old
objects, but now removes only the target's canonical/folded ID routes.
Unrelated sibling routes remain available for fresh numeric reload. Tests
refresh names and reload B solely by ID without a named B registration; an old
in-flight sibling load is rejected and retried fresh, and a target load cannot
restore its removed ID. Matching follows the existing ROOT-locale name policy,
including the distinct Unicode-name case.
Validation: 124 focused FE cases passed; FE build and Checkstyle passed. SDK
existence is mocked; no live remote DDL or actual statistics-worker integration
claim.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
