924060929 commented on code in PR #67545:
URL: https://github.com/apache/doris/pull/67545#discussion_r3954381116
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonConnector.java:
##########
@@ -465,11 +469,32 @@ private Catalog createCatalog() {
hmsAuth, storageHadoopConfig,
"Failed to create Paimon catalog with HMS metastore");
}
+ case PaimonCatalogProperties.DLF: {
+ // Legacy DLF catalogs often expose OSS only through dlf.*
aliases and an oss:// warehouse.
+ // Check the resolved storage bindings here so those catalogs
remain valid while non-OSS
+ // backends cannot be passed to Paimon's DLF Hive catalog.
+ if
(!hasDlfCompatibleStorage(storage().getStorageProperties())) {
+ throw new IllegalStateException("Paimon DLF metastore
requires OSS storage properties.");
+ }
+ DlfMetaStoreProperties dlf = (DlfMetaStoreProperties)
+ MetaStoreProviders.bind(catalogProps.getRaw(),
storageHadoopConfig);
+ Map<String, String> dlfConf = new
HashMap<>(dlf.toDlfCatalogConf());
+ dlfConf.put(PaimonCatalogFactory.DLF_CLIENT_POOL_IDENTITY,
+ PaimonCatalogFactory.dlfClientPoolIdentity(dlfConf));
Review Comment:
[P2] Retire the Paimon DLF client pool when the catalog closes
The new hashed DLF identity correctly prevents an ALTERed catalog from
reusing stale credentials, but the resulting pool has no catalog-close path. In
the bundled Paimon 1.3.1 dependency, HiveCatalog.close() is a no-op, while
CachedClientPool retains every HiveClientPool in a JVM-static Caffeine cache
until expireAfterAccess (5 minutes by default, and configurable). Repeated
ALTER/DROP can therefore retain multiple client generations and their transport
resources until expiry.
Please give the DLF pool catalog-generation ownership, for example through a
dedicated closeable pool or reference-counted eviction of the shared key, and
add a lifecycle test proving the last owner closes it. Unconditional key
invalidation would be unsafe when two live catalogs share the same identity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
