yayaha commented on code in PR #18817:
URL: https://github.com/apache/iceberg/pull/18817#discussion_r4498793170
##########
core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java:
##########
@@ -176,11 +176,6 @@ DelegateFileIO io(String location) {
}
}
- if (io instanceof SupportsStorageCredentials
Review Comment:
**note to reviewers**: This check ran in io(), so it ran on every file
operation. The equals test stopped it from resetting the delegate on each call.
Since #15678 and #15696, a delegate refreshes its own credentials, so after the
first refresh the lists always differ. Each call then reset the delegate to the
credentials from table load, and S3FileIO.setCredentials and
GCSFileIO.setCredentials cancel the scheduled refresh and close the storage
clients.
##########
core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java:
##########
@@ -279,11 +274,24 @@ public void deletePrefix(String prefix) {
io(prefix).deletePrefix(prefix);
}
+ /**
+ * Sets the storage credentials for new delegates and for existing delegates
that support storage
+ * credentials.
+ *
+ * <p>Call this method before other threads use this FileIO. A delegate that
another thread
+ * creates during this call can keep the previous credentials.
+ */
@Override
public void setCredentials(List<StorageCredential> credentials) {
Preconditions.checkArgument(credentials != null, "Invalid storage
credentials: null");
// copy credentials into a modifiable collection for Kryo serde
this.storageCredentials = Lists.newArrayList(credentials);
+ // A delegate can refresh its own credentials, so io() does not replace
them with this list.
+ for (DelegateFileIO io : ioInstances.values()) {
Review Comment:
**Note to reviewers**: The update of existing delegates moves here from
io(), so new credentials from a caller still reach them. This loop does not
need the equals test, because it runs only when a caller passes credentials,
and those are the credentials to use. CatalogUtil.loadFileIO and
RESTSessionCatalog call setCredentials before any delegate exists, so today the
loop does not run there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]