github-actions[bot] commented on code in PR #68196:
URL: https://github.com/apache/doris/pull/68196#discussion_r4140009153
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/ExternalRowCountCache.java:
##########
@@ -120,19 +330,35 @@ static Optional<Long> loadRowCount(RowCountKey
rowCountKey, boolean fillMetaCach
* @return Cached row count or -1 if not exist
*/
public long getCachedRowCount(long catalogId, long dbId, long tableId,
boolean fillMetaCache) {
- RowCountKey key = new RowCountKey(catalogId, dbId, tableId);
try {
- CompletableFuture<Optional<Long>> f = fillMetaCache
- ? rowCountCache.get(key, (rowCountKey, executor) ->
CompletableFuture.supplyAsync(
- () -> loadRowCount(rowCountKey, true), executor))
- : rowCountCache.get(key);
+ CompletableFuture<Optional<Long>> f;
+ long generation;
+ long dbGeneration;
+ publicationLock.readLock().lock();
+ try {
+ generation = currentCatalogGeneration(catalogId);
Review Comment:
[P3] Keep permanent catalog retirement effective for late held-table reads.
`releaseCatalog` removes the generation, but a statement that already holds an
initialized `ExternalTable` can call `getCachedRowCount` after DROP cleanup;
`currentCatalogGeneration` then recreates an entry for the retired ID, and no
later cleanup removes it. Repeated CREATE/DROP with overlapping statements
still grows this unbounded map. This is a distinct post-release admission path
from the earlier generation-release thread. Track retired IDs or reject
generation creation once the catalog is gone, and test a read that begins after
permanent cleanup.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/metacache/MetaCache.java:
##########
@@ -553,6 +553,20 @@ public Optional<T> getMetaObjById(long id) {
return name == null ? Optional.empty() : getMetaObj(name, id);
}
+ /** Return the retained canonical name without loading an evicted metadata
object. */
+ public Optional<String> getNameByIdIfPresent(long id) {
+ return Optional.ofNullable(withMetaObjLifecycleReadLock(() ->
idToName.get(id)));
+ }
+
+ /** Snapshot resident object identities matching a DROP alias under the
catalog's name mode. */
+ public List<Pair<String, Long>> getCachedIdentitiesMatching(String name,
boolean ignoreCase) {
+ return withMetaObjLifecycleReadLock(() -> idToName.entrySet().stream()
+ .filter(entry -> ignoreCase ?
entry.getValue().equalsIgnoreCase(name)
Review Comment:
[P3] Use the catalog's name fold for no-op DROP matching. In mode 2, an
Iceberg catalog can cache `I` while a quoted `DROP DATABASE IF EXISTS` for
dotless `ı` resolves no database: `Locale.ROOT` folds these to distinct `i` and
`ı`. This new `equalsIgnoreCase` predicate still matches them, so the no-op
hook unregisters live `I`, removes it from a complete database-name snapshot,
and clears its engine and row-count caches until refresh. The prior absent-DROP
thread addressed a catalog-wide fallback; this is a false match in the
replacement scan. Compare names with the same ROOT fold used by lookup and test
this pair.
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/RefreshManager.java:
##########
@@ -300,6 +385,24 @@ private void
invalidatePaimonCatalogForUnresolvedReplay(ExternalCatalog catalog)
}
}
+ private void invalidateColdReplayDatabase(ExternalCatalog catalog,
ExternalObjectLog log) {
+ Optional<Pair<String, Long>> identity =
catalog.getDbIdentityForReplay(log.getDbName(), log.getDbId());
Review Comment:
[P1] Preserve the logged identity on cold REFRESH replay. In a mode-2 Paimon
catalog with SDK caching enabled, direct table access can retain an SDK-only
handle for `Foo` while a names refresh rebinds `foo` to replacement `FOO`. If
that new `FOO` object is still cold when a queued REFRESH log for `Foo`
replays, `getDbIdentityForReplay` selects `FOO`, and the new DB-scoped
invalidation misses the SDK's case-sensitive `Foo` handle; the previous cold
path flushed the Paimon SDK catalog. When `Foo` returns, direct metadata access
can reuse stale table metadata. This is distinct from the DROP and ordinary
DB-scope threads. Preserve the logged DB identity or widen ambiguous replay
cleanup, and test the rebound with an SDK-only entry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
