You could use IAM roles in AWS to access the data in S3 without credentials. See this link <https://www.cloudera.com/documentation/enterprise/5-5-x/topics/spark_s3.html> and this link <http://parthicloud.com/how-to-access-s3-bucket-from-application-on-amazon-ec2-without-access-credentials/> for an example.
Regards, Srinath. On Thu, May 10, 2018 at 7:04 AM, Mina Aslani <aslanim...@gmail.com> wrote: > Hi, > > I am trying to load a ML model from AWS S3 in my spark app running in a > docker container, however I need to pass the AWS credentials. > My questions is, why do I need to pass the credentials in the path? > And what is the workaround? > > Best regards, > Mina >
