brunsgaard commented on PR #10141: URL: https://github.com/apache/paimon/pull/10141#issuecomment-5815134446
Thanks, that is a good catch. You are right that the stream keeps the credential it was opened with. `fileIOWithToken` merges the token into the options and builds the `FileIO` from literal keys, so a refresh produces a new `FileIO` for later calls and cannot reach a stream that is already open. Iceberg does not have this exposure because `S3FileIO` holds an `AwsCredentialsProvider` and the SDK resolves credentials per request. One thing I am unsure about is how much of this the client can own. A refresh returns whatever the server vends. A server that mints per request returns a near full token, so a large window does give long streams a margin. A server that caches a source credential and vends from it returns the remainder, and then the client receives a short token however early it asks. In that case the margin can only come from the server side, as a floor below which it never vends. `RESTTokenFileIO` already carries a `minimumValidityMillis` for `createBlobPresignedUrl`, where the caller states how long it needs. Stream opens do not use it. So I would rather not pick a default by guesswork. How do you think this should be approached? I am happy to change the default, to take the stream side further in this PR, or to split it, whichever fits the direction you have in mind. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
