github-actions[bot] commented on code in PR #66913:
URL: https://github.com/apache/doris/pull/66913#discussion_r3899907762
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalCatalog.java:
##########
@@ -73,8 +86,30 @@ public class HMSExternalCatalog extends ExternalCatalog {
//for "type" = "hms" , but is iceberg table.
private IcebergMetadataOps icebergMetadataOps;
+ private IcebergCatalogResourceTracker icebergResourceTracker = new
IcebergCatalogResourceTracker();
private volatile AbstractHiveProperties hmsProperties;
+ private AtomicLong runtimeGeneration = new AtomicLong();
+
+ public synchronized long getRuntimeGeneration() {
+ return runtimeGeneration.get();
+ }
+
+ /** Captures the authentication context and generation used by one Hudi
scan. */
+ public synchronized HudiScanRuntimeContext getHudiScanRuntimeContext() {
+ makeSureInitialized();
+ HudiExternalMetaCache hudiCache =
Env.getCurrentEnv().getExtMetaCacheMgr().hudi(getId());
Review Comment:
[P1] Avoid taking the lifecycle stripe under the catalog monitor
This synchronized method holds the HMS catalog monitor while `hudi(getId())`
may cold-initialize the engine group by waiting for the per-catalog lifecycle
stripe. ALTER/replay takes that stripe before validation and later calls
synchronized `modifyCatalogProps()`. If a first Hudi scan enters here during
validation, the scan waits for the stripe while ALTER waits for this monitor
forever (and keeps the catalog-manager write lock). The cold fs-view test
pre-initializes the engine group, so it misses this boundary. Prepare/resolve
the group before entering the catalog monitor, or otherwise enforce
lifecycle-stripe -> catalog-monitor ordering, and add an initially-absent-group
ALTER race.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalMetaCache.java:
##########
@@ -170,19 +206,81 @@ public Table getWritableIcebergTable(ExternalTable
dorisTable, @Nullable Iceberg
return table;
}
+ WritableTableLease acquireWritableIcebergTable(
Review Comment:
[P1] Keep the remaining writable callers on this generation lease
Only `IcebergMetadataOps` uses this scoped owner. The action hierarchy and
fallback transaction overloads still call the public raw
`getWritableIcebergTable()`; for example, `IcebergExpireSnapshotsAction`
retains that table through manifest reads, delete callbacks, and `commit()`. If
ALTER/reset lands after the helper's post-load identity check, no tracker
reference remains, so catalog retirement can close the newly owned Hive/Hadoop
shared FileIO while the action is still using it. This is outside the earlier
cache-escape thread because these handles bypass the table cache and its
removal listener. Migrate actions and transactions to an owner held through
their last commit/delete callback, and add reset barriers for both paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
