924060929 commented on code in PR #66913:
URL: https://github.com/apache/doris/pull/66913#discussion_r3900555298
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSExternalCatalog.java:
##########
@@ -73,8 +86,30 @@ public class HMSExternalCatalog extends ExternalCatalog {
//for "type" = "hms" , but is iceberg table.
private IcebergMetadataOps icebergMetadataOps;
+ private IcebergCatalogResourceTracker icebergResourceTracker = new
IcebergCatalogResourceTracker();
private volatile AbstractHiveProperties hmsProperties;
+ private AtomicLong runtimeGeneration = new AtomicLong();
+
+ public synchronized long getRuntimeGeneration() {
+ return runtimeGeneration.get();
+ }
+
+ /** Captures the authentication context and generation used by one Hudi
scan. */
+ public synchronized HudiScanRuntimeContext getHudiScanRuntimeContext() {
+ makeSureInitialized();
+ HudiExternalMetaCache hudiCache =
Env.getCurrentEnv().getExtMetaCacheMgr().hudi(getId());
Review Comment:
Fixed in c8d6f1c1d5c. HMSExternalCatalog now enters the per-catalog
lifecycle stripe before taking the catalog monitor for both runtime capture and
reset, so first-use Hudi group initialization and ALTER/replay follow one lock
order. The deterministic lifecycle test verifies the catalog monitor is not
held at stripe entry and that Hudi cache retirement runs while both lifecycle
ownership and the catalog monitor are held. The focused 151-test suite and
./build.sh --fe pass on the rebased branch-4.1 head.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergExternalMetaCache.java:
##########
@@ -170,19 +206,81 @@ public Table getWritableIcebergTable(ExternalTable
dorisTable, @Nullable Iceberg
return table;
}
+ WritableTableLease acquireWritableIcebergTable(
Review Comment:
Fixed in c8d6f1c1d5c. Iceberg actions and transactions now hold a
reference-counted WritableTableLease through their final manifest/delete
callback and commit or rollback. Rewrite planning, the transaction, and every
asynchronous rewrite task share the same exact generation; each task owns a
child lease and releases it on success, failure, timeout, cancellation, or
pre-start cancellation. Cancelled tasks are removed from TransientTaskManager
before task.cancel(), closing the scheduler-map retention path. Unit tests
cover action/transaction lifetime, task cancellation, scheduler cleanup
ordering, and generation retention; the focused 151-test suite and ./build.sh
--fe pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
