Hi,
I tried doing what Vladimir suggested. But no luck there either. My guess
is that it has something to do with securityContext.fsGroup. I am trying to
pass yaml file path along with spark submit command. My yaml file content
is
```
apiVersion: v1
kind: Pod
spec:
securityContext:
fsGroup: 65534
serviceAccount: <service accoun>
serviceAccountName: <service account name>
```
Is there anything wrong with this yaml file?
~
*Thanks,*
Rishabh Jain
Application Developer
Email rishabh.j...@thoughtworks.com
Telephone +91 6264277897 <+91+626+427+7897>
[image: ThoughtWorks]
<http://www.thoughtworks.com/?utm_campaign=prajwal-boloor-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator>
On Tue, Feb 9, 2021 at 10:44 PM Vladimir Prus <vladimir.p...@gmail.com>
wrote:
>
>
> On 9 Feb 2021, at 19:46, Rishabh Jain <rishabh.j...@thoughtworks.com>
> wrote:
>
> Hi,
>
> We are trying to access S3 from spark job running on EKS cluster pod. I
> have a service account that has an IAM role attached with full S3
> permission. We are using DefaultCredentialsProviderChain. But still we are
> getting 403 Forbidden from S3.
>
>
> It’s hard to say without any information, but some things you might want
> to double-check
>
> - Make sure the Spark job is using sufficiently new AWS SDK, so that IAM
> for service account is supported
> - Modify your job to print the effective role, e.g.
>
> val stsClient =
> AWSSecurityTokenServiceClientBuilder.standard().build();
> val request = new GetCallerIdentityRequest()
> val identity = stsClient.getCallerIdentity(request)
> println(identity.getArn())
>
> - If the above does not print the expected role, verify that the pods
> actually have the right service account, and
> that AWS_ROLE_ARN/AWS_WEB_IDENTITY_TOKEN_FILE variables are set on the
> pod, and that
> the assume policy for the role does allow EKS to assume that role.
> - If the above prints the expected role, then 403 error means you did not
> setup IAM policies on your role/bucket.
>
>
> Is there anything wrong with our approach?
>
> Generally speaking, IAM for service accounts in EKS + Spark works, it's
> just there's a lot of things that can go wrong the first time you do it.
>
>
> HTH,
>
