924060929 commented on code in PR #66914:
URL: https://github.com/apache/doris/pull/66914#discussion_r3820670022
##########
fe/fe-connector/fe-connector-hudi/src/main/java/org/apache/doris/connector/hudi/HudiScanPlanProvider.java:
##########
@@ -294,30 +294,33 @@ basePath, inputFormat, serdeLib, columnNames,
columnTypes, partitionFieldNames(m
HoodieLocalEngineContext engineCtx = new
HoodieLocalEngineContext(metaClient.getStorageConf());
HoodieTableFileSystemView fsView =
FileSystemViewManager.createInMemoryFileSystemView(
engineCtx, metaClient, metadataConfig);
+ try {
+ // Resolve partitions
+ List<String> partitionPaths = resolvePartitions(hudiHandle,
metaClient);
Review Comment:
Fixed on current head a55eb2af80e9c0880775050c65811dce3f3b9a12. The
independently-created HoodieTableMetadata is now owned by a try-with-resources
boundary after partition paths are fully materialized; the filesystem view
keeps its separate lifecycle. Added HudiResourceLifecycleTest to verify close.
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergTableCache.java:
##########
@@ -61,10 +62,15 @@ final class IcebergTableCache {
private final MetaCacheEntry<TableIdentifier, Table> entry;
IcebergTableCache(long ttlSeconds, int maxSize) {
+ this(ttlSeconds, maxSize, table -> { });
+ }
+
+ IcebergTableCache(long ttlSeconds, int maxSize, Consumer<Table> cleaner) {
// "<= 0 disables" connector TTL contract, folded to CacheSpec's
disable sentinel (CacheSpec.ofConnectorTtl).
CacheSpec spec = CacheSpec.ofConnectorTtl(ttlSeconds, maxSize);
this.entry = new MetaCacheEntry<>("iceberg-table", null, spec,
- ForkJoinPool.commonPool(), false, true, 0L, true);
+ ForkJoinPool.commonPool(), false, true, 0L, true,
Review Comment:
Fixed on current head a55eb2af80e9c0880775050c65811dce3f3b9a12. Cache
ownership and statement borrowers now use ref-counted TableOwner/TableLease
boundaries; invalidation, disabled-cache loads, suppressed publication,
TTL/capacity eviction, and connector teardown release only their own reference.
REST ownership is classified at load time and captures a stable cleanup action,
so an active statement can safely outlive connector teardown and closes its
per-table FileIO exactly once on final lease release. Null/NONE scopes use
direct loaders and never drop a lease. Added deterministic
lifecycle/race/teardown tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
