> On 23 Jul 2015, at 01:50, Ewan Leith <ewan.le...@realitymine.com> wrote: > > I think the standard S3 driver used in Spark from the Hadoop project (S3n) > doesn't support IAM role based authentication. > > However, S3a should support it. If you're running Hadoop 2.6 via the > spark-ec2 scripts (I'm not sure what it launches with by default) try > accessing your bucket via s3a:// URLs instead of s3n:// > > http://wiki.apache.org/hadoop/AmazonS3 > > https://issues.apache.org/jira/browse/HADOOP-10400 > > Thanks, > Ewan >
s3a should support roles. note that it isn't ready for production use before Hadoop 2.7.1, various scaiability and performance problems surfaced after 2.6 shipped --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
