Hi Keith, Thank you for your thoughtful comments. Here are my thoughts on the points you raised:
1. Weight-based distribution is actually a critical requirement in our production environment. We frequently manage multiple DFS clusters of varying sizes, where throughput must be allocated proportionally to their capacities. That said, the weighting strategy is designed to be an optional feature; it won't interfere with the standard round-robin behavior, giving users the flexibility to choose the strategy that best fits their needs. 2. As described in the FIP, we will hold a map from FSKey to Credentials in SecurityTokenReceiver. 3. Hadoop-fs provides a mechanism to pass the FileSystem URI into the CredentialsProvider[1]. We can simply capture this URI in DynamicTemporaryOssCredentialsProvider and use it to retrieve the corresponding credentials when getCredentials() is invoked. [1] https://github.com/apache/hadoop/blob/42f0e79bff0ceb950be174fb6d3b00a113246dcb/hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSUtils.java#L123 Best regards, Liebing Yu On Mon, 12 Jan 2026 at 20:22, Keith Lee <[email protected]> wrote: > Hello Liebing, > > Thank you for the proposal. I have some questions below > > 1. Can you elaborate the motivation or use-case for weighting? IMO > weighting adds a small layer of complexity and unless there is a strong > use-case for it we should opt for just plain round-robin. > 2. In order to populate cache by FsKey, TokenReceiver implementations need > to know which token belong to which particular FsKey. Does TokenReceiver > interface need to change to include FsPath information? Or is the plan to > have FsPath info contained within ObtainedSecurityToken.additionInfos? > 3. DynamicTemporaryOssCredentialsProvider is currently FsPath agnostic and > it is also dynamically instantiated by hadoop-fs. What is your planned > changes around here specifically to ensure that hadoop-fs instantiate a > CredentialsProvider that calls > OSSSecurityTokenReceiver.getCredentials(fsPath) > for the correct FsPath? > > Best regards > Keith Lee > > > On Thu, Jan 8, 2026 at 12:10 PM Liebing Yu <[email protected]> wrote: > > > Hi devs, > > > > I propose initiating discussion on FIP-25[1]. Fluss leverages remote > > storage systems—such as Amazon S3, HDFS, and Alibaba Cloud OSS—to > deliver a > > cost-efficient, highly available, and fault-tolerant storage solution > > compared to local disk. *However, in production environments, we often > find > > that the bandwidth of a single remote storage becomes a bottleneck. > *Taking > > OSS[2] as an example, the typical upload bandwidth limit for a single > > account is 20 Gbit/s (Internal) and 10 Gbit/s (Public). So I initiated > this > > FIP which aims to introduce support for multiple remote storage paths and > > enables the dynamic addition of new storage paths without service > > interruption. > > > > Any feedback and suggestions on this proposal are welcome! > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-25%3A+Support+Multi-Location+for+Remote+Storage > > [2] > > > > > https://www.alibabacloud.com/help/en/oss/user-guide/limits?spm=a2c63.l28256.help-menu-31815.d_0_0_5.2ac34d06oZYFvK > > > > Best regards, > > Liebing Yu > > >
