MarquisC commented on issue #7344: URL: https://github.com/apache/iceberg/issues/7344#issuecomment-1646560968
hey @raviranak @stevenzwu what we're seeing something similar EKS as well via the Iceberg Flink path (wanted to get your thoughts): - We're using the aws sdk bundle jar [tested against 2.17.257 and 2.20.99] [The default credential provider ](https://github.com/aws/aws-sdk-java-v2/blob/2.17.257/core/auth/src/main/java/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.java) _should_ have by precedent attempted to leverage the WebIdentity path right (before it defaults to the EC2 instance role)? If I kubectl exec in and install the aws-cli within the container, the the result of `aws sts get-caller-identity` correctly identifies the hierarchy and selects the kubernetes service account -> IAM role (WebIdentity Path) I couldn't find an easy way in the time that I looked to directly influence/configure the Glue client from the Iceberg lib. What I ended up doing was just letting the EC2 instance role assume the role it needs via: ``` Create Catalog ... 'catalog-impl'='org.apache.iceberg.aws.glue.GlueCatalog', 'io-impl'='org.apache.iceberg.aws.s3.S3FileIO', 'client.assume-role.region' = 'us-east-1', 'client.factory' = 'org.apache.iceberg.aws.AssumeRoleAwsClientFactory', 'client.assume-role.arn' = 'arn:aws:iam::${aws account number}:role/${the role that should of worked from web identity perms')"}" ``` The particular where it wasn't working use case was enabling Flink Session clusters on kubernetes and the Flink SQL Gateway to chat with Glue correctly. The weird part about this is, our fat jar Flink jobs (same deps), successfully leverage the WebIdentity path (we allow our jobs to dynamically create the tables and databases in glue if they don't exist). Hopefully this is helpful for you @raviranak -when I get more time I'll keep fiddling and try to see what I can see (I might be able to for example step through the SQL Gateway Impl and see what happened). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
