Thank you Liebing for your response! Follow up question & clarifications.
> 2. As described in the FIP, we will hold a map from FSKey to Credentials in SecurityTokenReceiver. Currently the listening method `onNewTokensObtained(ObtainedSecurityToken)` does not contain FSKey argument [1]. Without that, there is not enough information for SecurityTokenReceiver implementation to construct map of FSKey to Credentials. > 3. Hadoop-fs provides a mechanism to pass the FileSystem URI into the CredentialsProvider[1]. We can simply capture this URI in DynamicTemporaryOssCredentialsProvider and use it to retrieve the corresponding credentials when getCredentials() is invoked. This clarifies it. Thank you! [1] https://github.com/apache/fluss/blob/main/fluss-common/src/main/java/org/apache/fluss/fs/token/SecurityTokenReceiver.java#L34 Best regards Keith Lee On Tue, Jan 13, 2026 at 11:37 AM Liebing Yu <[email protected]> wrote: > Hi Keith, > > Thank you for your thoughtful comments. Here are my thoughts on the points > you raised: > > 1. Weight-based distribution is actually a critical requirement in our > production environment. We frequently manage multiple DFS clusters of > varying sizes, where throughput must be allocated proportionally to their > capacities. That said, the weighting strategy is designed to be an optional > feature; it won't interfere with the standard round-robin behavior, giving > users the flexibility to choose the strategy that best fits their needs. > > 2. As described in the FIP, we will hold a map from FSKey to Credentials > in SecurityTokenReceiver. > > 3. Hadoop-fs provides a mechanism to pass the FileSystem URI into the > CredentialsProvider[1]. We can simply capture this URI in > DynamicTemporaryOssCredentialsProvider and use it to retrieve the > corresponding credentials when getCredentials() is invoked. > > [1] > > https://github.com/apache/hadoop/blob/42f0e79bff0ceb950be174fb6d3b00a113246dcb/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSUtils.java#L123 > > Best regards, > Liebing Yu > > > On Mon, 12 Jan 2026 at 20:22, Keith Lee <[email protected]> > wrote: > > > Hello Liebing, > > > > Thank you for the proposal. I have some questions below > > > > 1. Can you elaborate the motivation or use-case for weighting? IMO > > weighting adds a small layer of complexity and unless there is a strong > > use-case for it we should opt for just plain round-robin. > > 2. In order to populate cache by FsKey, TokenReceiver implementations > need > > to know which token belong to which particular FsKey. Does TokenReceiver > > interface need to change to include FsPath information? Or is the plan to > > have FsPath info contained within ObtainedSecurityToken.additionInfos? > > 3. DynamicTemporaryOssCredentialsProvider is currently FsPath agnostic > and > > it is also dynamically instantiated by hadoop-fs. What is your planned > > changes around here specifically to ensure that hadoop-fs instantiate a > > CredentialsProvider that calls > > OSSSecurityTokenReceiver.getCredentials(fsPath) > > for the correct FsPath? > > > > Best regards > > Keith Lee > > > > > > On Thu, Jan 8, 2026 at 12:10 PM Liebing Yu <[email protected]> wrote: > > > > > Hi devs, > > > > > > I propose initiating discussion on FIP-25[1]. Fluss leverages remote > > > storage systems—such as Amazon S3, HDFS, and Alibaba Cloud OSS—to > > deliver a > > > cost-efficient, highly available, and fault-tolerant storage solution > > > compared to local disk. *However, in production environments, we often > > find > > > that the bandwidth of a single remote storage becomes a bottleneck. > > *Taking > > > OSS[2] as an example, the typical upload bandwidth limit for a single > > > account is 20 Gbit/s (Internal) and 10 Gbit/s (Public). So I initiated > > this > > > FIP which aims to introduce support for multiple remote storage paths > and > > > enables the dynamic addition of new storage paths without service > > > interruption. > > > > > > Any feedback and suggestions on this proposal are welcome! > > > > > > [1] > > > > > > > > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-25%3A+Support+Multi-Location+for+Remote+Storage > > > [2] > > > > > > > > > https://www.alibabacloud.com/help/en/oss/user-guide/limits?spm=a2c63.l28256.help-menu-31815.d_0_0_5.2ac34d06oZYFvK > > > > > > Best regards, > > > Liebing Yu > > > > > >
