This is an automated email from the ASF dual-hosted git repository. fresh-borzoni pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/fluss.git
commit c5a0c00a22999e375afc2ac3e50f5c6cb26886a1 Author: Anton Borisov <[email protected]> AuthorDate: Tue Jul 7 02:01:53 2026 +0100 [docs] Document S3 custom AWS credentials provider mode --- .../maintenance/tiered-storage/filesystems/s3.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/website/docs/maintenance/tiered-storage/filesystems/s3.md b/website/docs/maintenance/tiered-storage/filesystems/s3.md index ca22dd339..311573a59 100644 --- a/website/docs/maintenance/tiered-storage/filesystems/s3.md +++ b/website/docs/maintenance/tiered-storage/filesystems/s3.md @@ -86,7 +86,7 @@ Without `s3.assumed.role.arn`, Fluss falls back to `GetSessionToken` (the defaul When running Fluss on Kubernetes with [IAM Roles for Service Accounts (IRSA)](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) or on EC2 with instance profiles, you can omit `s3.access-key` and `s3.secret-key`. The server will authenticate using the default AWS credential chain. :::note -The explicit server-side credential provider mode (`s3.aws.credentials.provider`) only supports providers that return long-term access key credentials and cannot be combined with `s3.assumed.role.arn`. IRSA and instance profile providers return session credentials, so they must use the default AWS credential chain described in this section instead of the explicit provider mode. +IRSA and instance profiles return session credentials, so they must use this default chain — not the explicit [Custom AWS Credentials Provider](#custom-aws-credentials-provider) mode, which requires long-term credentials. ::: In this mode, `s3.assumed.role.arn` is required — the server uses `AssumeRole` to generate temporary credentials for clients (Flink/Spark connectors) that read tiered data from S3. @@ -104,3 +104,21 @@ The server's IAM role (e.g., the IRSA service account role) must have: :::note The server authenticates using its own credentials (static keys, IRSA, instance profile, or environment variables) for S3 data access. For delegation, the server calls AssumeRole with the configured `s3.assumed.role.arn`, so clients receive credentials for that role — which can have different permissions (e.g., read-only). Note that the server uses the same identity for both its own S3 access and the STS AssumeRole call. Further decoupling is planned. ::: + +### Custom AWS Credentials Provider + +For deployments that manage long-term credentials through a standard AWS SDK or Hadoop S3A credentials provider (for example, a rotating profile), set `s3.aws.credentials.provider` to the provider class. Fluss then treats it as the authoritative server-side credential source, so rotated credentials are picked up without restarting the servers. + +```yaml +remote.data.dir: s3://<your-bucket>/path/to/remote/storage +s3.region: <your-s3-region> +s3.aws.credentials.provider: com.amazonaws.auth.profile.ProfileCredentialsProvider +``` + +| Configuration | Description | +|---|---| +| `s3.aws.credentials.provider` | Fully qualified class name of an AWS SDK / Hadoop S3A credentials provider. It is used both for the server's own S3 access and for resolving the credentials used to mint temporary client credentials via `GetSessionToken`. Takes precedence over `s3.access-key`/`s3.secret-key` when both are configured. | + +:::note +This mode only supports providers that return **long-term** access-key credentials, and it cannot be combined with `s3.assumed.role.arn`. Providers that return session credentials — including IRSA and EC2 instance profiles — must use the [default AWS credential chain](#default-aws-credential-chain-irsa-instance-profiles) instead. +:::
