Hi, I have a flink job running on EKS, reading and writing data records to S3 buckets. I am trying to set up access credentials via AWS IAM. I followed this: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
I have configured: com.amazonaws.auth.WebIdentityTokenCredentialsProvider as the credential provider in flink-conf.yaml for hadoop s3a connector, and annotated my service account with the role. When running the job, i am getting access denied error Exception: Caused by: com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: Not authorized to perform sts:AssumeRoleWithWebIdentity (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: 923df33a-802e-47e2-a203-0841aca03dd8; Proxy: null) I have tried to access S3 buckets from AWS CLI running in a pod with the same service account and that works. Am I using the correct credential provider for IAM integration, not sure if Hadoop S3a supports it. https://issues.apache.org/jira/browse/HADOOP-18154 Please advise if I am doing anything wrong in setting up credentials via IAM. Regards Anuj Jain
