litiliu opened a new issue, #3524: URL: https://github.com/apache/fluss/issues/3524
### Search before asking - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar. ### Fluss version 1.0-snapshot ### Please describe the bug PR #938 introduced `client.fs.*` forwarding so Fluss clients can pass client-side filesystem options when they need to access remote files, such as KV snapshots or remote log segments. The intended scope appears to be necessary client-side filesystem runtime options, not filesystem credential configuration. In the PR discussion, exposing credential-related filesystem options such as `*.accessKeySecret`, `*.roleArn`, and `*.credentials.provider` was explicitly called out as dangerous, and the PR was updated to forward only the `client.fs.*` namespace. However, the current implementation still forwards all `client.fs.*` options to `FileSystem.initialize` after stripping the `client.fs.` prefix. This means credential-bearing options can be provided by the client and can affect client-side filesystem initialization. For S3, this can bypass the intended server-issued temporary token flow. Fluss clients are expected to obtain filesystem security tokens from Fluss servers through `getFileSystemSecurityToken` and use `DynamicTemporaryAWSCredentialsProvider` locally. Long-lived credentials, AssumeRole, and STS token generation should be managed by Fluss servers. For example, a client can configure: ```text client.fs.s3a.access.key client.fs.s3a.secret.key client.fs.s3a.access-key client.fs.s3a.secret-key client.fs.s3a.session.token client.fs.s3a.assumed.role.arn client.fs.s3a.assumed.role.* client.fs.s3a.aws.credentials.provider ``` These options are stripped to `fs.s3a.*` and passed to the S3 filesystem plugin. If `client.fs.s3a.access.key` and `client.fs.s3a.secret.key` are configured, `S3FileSystemPlugin` sees static credentials and skips configuring `DynamicTemporaryAWSCredentialsProvider`. Expected behavior: - Fluss clients should fail fast when credential-bearing filesystem options are configured through `client.fs.*`. - Fluss clients should continue to allow non-credential filesystem runtime options, such as endpoint, region, path-style access, or download tuning options. Examples of options that should still be allowed: ```text client.fs.s3a.endpoint client.fs.s3a.region client.fs.s3a.path-style-access client.fs.s3a.path.style.access client.fs.oss.endpoint client.fs.oss.multipart.download.threads ``` ### Solution Add validation before forwarding `client.fs.*` options to `FileSystem.initialize`. Reject credential-bearing client-side filesystem options fail-fast, at least for S3: ```text client.fs.s3a.aws.credentials.provider client.fs.s3a.access.key client.fs.s3a.secret.key client.fs.s3a.access-key client.fs.s3a.secret-key client.fs.s3a.session.token client.fs.s3a.assumed.role.arn client.fs.s3a.assumed.role.* ``` The error message should clearly state that filesystem credentials must be configured on Fluss servers, and Fluss clients should use server-issued temporary filesystem tokens. Suggested tests: 1. Non-credential `client.fs.*` options are still forwarded to the filesystem. 2. `client.fs.s3a.access.key` and `client.fs.s3a.secret.key` fail fast. 3. `client.fs.s3a.aws.credentials.provider` fails fast. 4. `client.fs.s3a.assumed.role.arn` and `client.fs.s3a.assumed.role.*` fail fast. 5. Normal S3 client access without credential-bearing client options still uses `DynamicTemporaryAWSCredentialsProvider`. Related issues and PRs: - #935 - #938 ### Are you willing to submit a PR? - [ ] I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
