NathanCYee opened a new issue, #16667: URL: https://github.com/apache/iceberg/issues/16667
### Feature Request / Improvement Currently, [`RESTSigV4AuthSession`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/RESTSigV4AuthSession.java#L83) calls the [`restCredentialsProvider` function in `AwsProperties`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L451) to retrieve the credentials provider to sign the sigv4 request. The `restCredentialsProvider` function currently follows the following decision chain: 1. [If `accessKeyId`, `secretAccessKey`, and optionally `sessionToken` are set -> return a `StaticCredentialsProvider`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L482) 2. [If `clientCredentialsProvider` is set, build the class using `DynClasses` and pass the property map `clientCredentialsProviderProperties` into the create function of the class](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L500) 3. Otherwise [return the default credential chain using `DefaultCredentialsProvider`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L497). However, this decision chain does not consider the credentials if `client.factory=org.apache.iceberg.aws.AssumeRoleAwsClientFactory`. The factory will define a role to assume that ISN'T used on REST catalog requests but IS used for S3FileIO, Glue Catalog, KMS, and DynamoDB operations causing divergent permissions for REST catalog operations. **Workarounds attempted:** AwsProperties [allows the definition of alternative credential providers](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsClientProperties.java#L45) under `client.credentials-provider`. When this property is set, `restCredentialsProvider` will [return the custom provider instead of the default chain](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L492). However, this property is not compatible with `software.amazon.awssdk.services.sts.auth.StsAssumeRoleCredentialsProvider` since this property requires the credentials provider to have a `create()` or `create(Map)` function, while the [assume role provider uses the builder pattern](https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/services/sts/auth/StsAssumeRoleCredentialsProvider.html). **Potential Design:** Fastest path would be to update the [`credentialsProvider(String accessKeyId, String secretAccessKey, String sessionToken)`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java#L491) function of `AwsProperties` to create and return a `StsAssumeRoleCredentialsProvider` if `this.clientAssumeRoleArn` is set. The `AwsProperties` class would be [duplicating the `createCredentialsProvider` function of `AssumeRoleAwsClientFactory`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/AssumeRoleAwsClientFactory.java#L176). **This could go between step 2 and 3 of the decision chain of `restCredentialsProvider`** (before returning default credentials chain, check for `clientAssumeRoleArn` first and if it is set return the assume role credentials provider). A more complete path would be to define another interface (possibly named `CanCreateAwsCredentialsProvider`?) that has a public function `AwsCredentialsProvider createCredentialsProvider()`. `AssumeRoleAwsClientFactory` can implement this class (and possibly other client factories such as [`LakeFormationAwsClientFactory`](https://github.com/apache/iceberg/blob/26a57711d990c695915afb5ce14f00736325d547/aws/src/main/java/org/apache/iceberg/aws/lakeformation/LakeFormationAwsClientFactory.java#L144)). When `CLIENT_FACTORY` is set to a class that inherits this interface, instantiate the client factory and call the function between step 2 and 3 of the decision chain of `restCredentialsProvider`. ### Query engine Spark ### Willingness to contribute - [x] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [ ] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
