924060929 commented on code in PR #66914:
URL: https://github.com/apache/doris/pull/66914#discussion_r3838742129
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/ReauthenticatingRestSessionCatalog.java:
##########
@@ -124,9 +134,32 @@ private synchronized void
reauthenticate(RESTSessionCatalog attemptedOn, Runtime
+ "then retrying the request once.", name(), cause);
RESTSessionCatalog replacement = delegateBuilder.get();
RESTSessionCatalog wedged = delegate;
- delegate = replacement;
+ if (resourceTracker == null) {
+ delegate = replacement;
+ closeReplacedDelegate(wedged);
+ return;
+ }
+ try {
+ invalidateTables.run();
Review Comment:
已修复。REST delegate rotation 的 publication 与 cache invalidation/admission 已按同一
generation 协调，旧 delegate load 不会在 rotation 后发布为可达 cache owner。
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnector.java:
##########
@@ -896,13 +901,50 @@ boolean isUserSessionEnabled() {
&&
IcebergCatalogProperties.TYPE_REST.equals(catalogProps.getFlavor());
}
- private Catalog getOrCreateCatalog() {
- if (icebergCatalog == null) {
- synchronized (this) {
- if (icebergCatalog == null) {
- icebergCatalog = createCatalog();
- }
+ /**
+ * Closes a table's FileIO when the cached raw table owns it. Glue and
S3Tables create a per-table
+ * S3FileIO; REST tables are closed only when they do not share the
catalog-level FileIO. Other catalog
+ * flavors are left untouched because they may share a catalog-level
FileIO.
+ */
+ private Runnable cachedTableCleanup(Table table) {
+ return cachedTableCleanup(table, catalogProps.getFlavor());
+ }
+
+ static Runnable cachedTableCleanup(Table table, String flavor) {
+ if (table == null) {
+ return () -> { };
+ }
+ boolean tableOwned = false;
+ try {
+ if (IcebergCatalogProperties.TYPE_GLUE.equals(flavor)
+ || IcebergCatalogProperties.TYPE_S3_TABLES.equals(flavor))
{
+ tableOwned = true;
+ } else if (IcebergCatalogProperties.TYPE_REST.equals(flavor)
+ && table.io() instanceof SupportsStorageCredentials) {
+ tableOwned = !((SupportsStorageCredentials)
table.io()).credentials().isEmpty();
Review Comment:
已修复。REST table FileIO 是否独占改为按实际 IO identity/producing catalog IO
判断，config-only response 产生的独立 FileIO 会作为 table-owned 关闭。
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnector.java:
##########
@@ -896,13 +902,88 @@ boolean isUserSessionEnabled() {
&&
IcebergCatalogProperties.TYPE_REST.equals(catalogProps.getFlavor());
}
- private Catalog getOrCreateCatalog() {
- if (icebergCatalog == null) {
- synchronized (this) {
- if (icebergCatalog == null) {
- icebergCatalog = createCatalog();
- }
+ /**
+ * Closes a table's FileIO when the cached raw table owns it. Glue and
S3Tables create a per-table
+ * S3FileIO; REST tables are closed only when they do not share the
catalog-level FileIO. Other catalog
+ * flavors are left untouched because they may share a catalog-level
FileIO.
+ */
+ private Runnable cachedTableCleanup(Table table) {
+ return cachedTableCleanup(table, catalogProps.getFlavor(),
restSessionCatalog);
+ }
+
+ static Runnable cachedTableCleanup(Table table, String flavor) {
+ return cachedTableCleanup(table, flavor, null);
+ }
+
+ private static Runnable cachedTableCleanup(Table table, String flavor,
Object catalog) {
+ if (table == null) {
+ return () -> { };
+ }
+ boolean tableOwned = false;
+ try {
+ if (IcebergCatalogProperties.TYPE_GLUE.equals(flavor)
+ || IcebergCatalogProperties.TYPE_S3_TABLES.equals(flavor))
{
+ tableOwned = true;
+ } else if (IcebergCatalogProperties.TYPE_REST.equals(flavor)) {
+ FileIO catalogFileIO = restCatalogFileIO(catalog);
Review Comment:
已修复。table load 捕获 producing delegate generation 的 catalog FileIO，cleanup
分类不再读取 currentDelegate；rotation 后不会误关仍有 borrower 的旧 shared IO。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
