Kazuyuki Tanimura created HADOOP-14237:
------------------------------------------
Summary: S3A Support Shared Instance Profile Credentials Across
All Instances
Key: HADOOP-14237
URL: https://issues.apache.org/jira/browse/HADOOP-14237
Project: Hadoop Common
Issue Type: Bug
Components: fs/s3
Affects Versions: 3.0.0-alpha2, 3.0.0-alpha1, 2.8.0, 2.8.1
Environment: EC2, AWS
Reporter: Kazuyuki Tanimura
When I run a large Hadoop cluster on EC2 instances with IAM Role, it fails
getting the instance profile credentials, eventually all jobs on the cluster
fail. Since a number of S3A clients (all mappers and reducers) try to get the
credentials, the AWS credential endpoint starts responding 5xx and 4xx error
codes.
SharedInstanceProfileCredentialsProvider.java is sort of trying to solve it,
but it still does not share the credentials with other EC2 instances /
processes.
This issue prevents users from creating Hadoop clusters on EC2
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
