jclarysse opened a new pull request #4049: URL: https://github.com/apache/hadoop/pull/4049
### Description of PR The PR addresses a requirement to comply with AWS security concept [IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) (IRSA) while operating [Delta sharing](https://github.com/delta-io/delta-sharing) in Amazon Elastic Kubernetes Service (EKS). The code change consists in adding a new credentials provider class `org.apache.hadoop.fs.s3a.OIDCTokenCredentialsProvider` to the module [hadoop-aws](https://hadoop.apache.org/docs/current/hadoop-aws/tools/hadoop-aws/index.html) in [Hadoop release 2.10.1](https://github.com/apache/hadoop/tree/rel/release-2.10.1). In addition, the dependency aws-java-sdk-bundle-1.11.271 was upgraded to its latest version 1.12.167 as [AWS WebIdentityTokenCredentialsProvider class](https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/WebIdentityTokenCredentialsProvider.html%E2%80%A6) was not yet available in original version. ### How was this patch tested? No new unit-test or integration-test was created on-purpose. The patch was "only" tested as part of our specific use-case, using Delta sharing server 0.4.0 with the following Hadoop configuration (core-site.xml): ``` <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>fs.s3a.aws.credentials.provider</name> <value>org.apache.hadoop.fs.s3a.OIDCTokenCredentialsProvider</value> </property> <property> <name>fs.s3a.jwt.path</name> <value>/var/run/secrets/kubernetes.io/serviceaccount/token</value> </property> <property> <name>fs.s3a.role.arn</name> <value>my_iam_role_arn</value> </property> <property> <name>fs.s3a.session.name</name> <value></value> </property> <property> <name>fs.s3a.server-side-encryption-algorithm</name> <value>SSE-KMS</value> </property> <property> <name>fs.s3a.server-side-encryption.key</name> <value>my_kms_key_id</value> </property> </configuration> ``` ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [X] No new dependency was added to the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
