Thank you for the help. I am using the jetS3t support. I will look into setting up emrfs support today, though, and share anything I come up with.
Sent via the Samsung Galaxy Note® 4, an AT&T 4G LTE smartphone -------- Original message -------- From: David Tucker <[email protected]> Date: 05/12/2015 1:18 AM (GMT-05:00) To: [email protected] Subject: Re: Drill connect to S3 with AWS EMR role The current version of Hadoop in EMR (both Apache and MapR) does not support the IAM authentication to S3 without the credentials in core-site. I believe the support has been integrated into Hadoop 2.6 … so when the EMR distributions upgrade to that level, the access you request should be supported. Did you successfully configure the drill-bit to use the full EMRFS jars, or did you default to the older jets3t support ? If you have the classpath settings for full emrfs support, please share them with the group (and I will integrate that support into the MapR EMR bootstrap action under development for Drill). Regards, David On May 11, 2015, at 6:55 PM, Alonzo Barnett <[email protected]> wrote: > I am looking into using Drill with AWS EMR. My organization uses IAM roles > with EMR in order to rotate security credentials. > > In a first round of testing I successfully connected Drill to S3 on a > cluster without IAM role based credential rotation which required placing > access and secret keys in core-site.xml. > > It is possible to still use Drill with S3 without hardcoding credentials > into core-site? With emrfs there is some work by AWS to rotate > credentials, and I would like to exploit the modification to core-site if > possible. > > -- > v/r, > Alonzo Barnett
