Hi Keith, You are right about point 2. We should include more information, specifically the authority, in the ObtainedSecurityToken to ensure the SecurityTokenReceiver has enough context to construct the map to credentials.
I will update the FIP to reflect this change in the ObtainedSecurityToken structure. Best regards, Liebing Yu On Tue, 13 Jan 2026 at 20:11, Keith Lee <[email protected]> wrote: > Thank you Liebing for your response! Follow up question & clarifications. > > > 2. As described in the FIP, we will hold a map from FSKey to Credentials > in SecurityTokenReceiver. > > Currently the listening method `onNewTokensObtained(ObtainedSecurityToken)` > does not contain FSKey argument [1]. Without that, there is not enough > information for SecurityTokenReceiver implementation to construct map of > FSKey to Credentials. > > > 3. Hadoop-fs provides a mechanism to pass the FileSystem URI into the > CredentialsProvider[1]. We can simply capture this URI in > DynamicTemporaryOssCredentialsProvider and use it to retrieve the > corresponding credentials when getCredentials() is invoked. > > This clarifies it. Thank you! > > [1] > > https://github.com/apache/fluss/blob/main/fluss-common/src/main/java/org/apache/fluss/fs/token/SecurityTokenReceiver.java#L34 > > Best regards > Keith Lee > > > On Tue, Jan 13, 2026 at 11:37 AM Liebing Yu <[email protected]> wrote: > > > Hi Keith, > > > > Thank you for your thoughtful comments. Here are my thoughts on the > points > > you raised: > > > > 1. Weight-based distribution is actually a critical requirement in our > > production environment. We frequently manage multiple DFS clusters of > > varying sizes, where throughput must be allocated proportionally to their > > capacities. That said, the weighting strategy is designed to be an > optional > > feature; it won't interfere with the standard round-robin behavior, > giving > > users the flexibility to choose the strategy that best fits their needs. > > > > 2. As described in the FIP, we will hold a map from FSKey to Credentials > > in SecurityTokenReceiver. > > > > 3. Hadoop-fs provides a mechanism to pass the FileSystem URI into the > > CredentialsProvider[1]. We can simply capture this URI in > > DynamicTemporaryOssCredentialsProvider and use it to retrieve the > > corresponding credentials when getCredentials() is invoked. > > > > [1] > > > > > https://github.com/apache/hadoop/blob/42f0e79bff0ceb950be174fb6d3b00a113246dcb/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSUtils.java#L123 > > > > Best regards, > > Liebing Yu > > > > > > On Mon, 12 Jan 2026 at 20:22, Keith Lee <[email protected]> > > wrote: > > > > > Hello Liebing, > > > > > > Thank you for the proposal. I have some questions below > > > > > > 1. Can you elaborate the motivation or use-case for weighting? IMO > > > weighting adds a small layer of complexity and unless there is a strong > > > use-case for it we should opt for just plain round-robin. > > > 2. In order to populate cache by FsKey, TokenReceiver implementations > > need > > > to know which token belong to which particular FsKey. Does > TokenReceiver > > > interface need to change to include FsPath information? Or is the plan > to > > > have FsPath info contained within ObtainedSecurityToken.additionInfos? > > > 3. DynamicTemporaryOssCredentialsProvider is currently FsPath agnostic > > and > > > it is also dynamically instantiated by hadoop-fs. What is your planned > > > changes around here specifically to ensure that hadoop-fs instantiate a > > > CredentialsProvider that calls > > > OSSSecurityTokenReceiver.getCredentials(fsPath) > > > for the correct FsPath? > > > > > > Best regards > > > Keith Lee > > > > > > > > > On Thu, Jan 8, 2026 at 12:10 PM Liebing Yu <[email protected]> > wrote: > > > > > > > Hi devs, > > > > > > > > I propose initiating discussion on FIP-25[1]. Fluss leverages remote > > > > storage systems—such as Amazon S3, HDFS, and Alibaba Cloud OSS—to > > > deliver a > > > > cost-efficient, highly available, and fault-tolerant storage solution > > > > compared to local disk. *However, in production environments, we > often > > > find > > > > that the bandwidth of a single remote storage becomes a bottleneck. > > > *Taking > > > > OSS[2] as an example, the typical upload bandwidth limit for a single > > > > account is 20 Gbit/s (Internal) and 10 Gbit/s (Public). So I > initiated > > > this > > > > FIP which aims to introduce support for multiple remote storage paths > > and > > > > enables the dynamic addition of new storage paths without service > > > > interruption. > > > > > > > > Any feedback and suggestions on this proposal are welcome! > > > > > > > > [1] > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-25%3A+Support+Multi-Location+for+Remote+Storage > > > > [2] > > > > > > > > > > > > > > https://www.alibabacloud.com/help/en/oss/user-guide/limits?spm=a2c63.l28256.help-menu-31815.d_0_0_5.2ac34d06oZYFvK > > > > > > > > Best regards, > > > > Liebing Yu > > > > > > > > > >
