Hi,
I've set these properties both in core-site.xml and hdfs-site.xml with no luck.
Thank you.
Daniel
> On 9 Jun 2016, at 01:11, Steve Loughran <ste...@hortonworks.com> wrote:
>
>
>> On 8 Jun 2016, at 16:34, Daniel Haviv <daniel.ha...@veracity-group.com>
>> wrote:
>>
>> Hi,
>> I'm trying to create a table on s3a but I keep hitting the following error:
>> Exception in thread "main" org.apache.hadoop.hive.ql.metadata.HiveException:
>> MetaException(message:com.cloudera.com.amazonaws.AmazonClientException:
>> Unable to load AWS credentials from any provider in the chain)
>>
>> I tried setting the s3a keys using the configuration object but I might be
>> hitting SPARK-11364 :
>> conf.set("fs.s3a.access.key", accessKey)
>> conf.set("fs.s3a.secret.key", secretKey)
>> conf.set("spark.hadoop.fs.s3a.access.key",accessKey)
>> conf.set("spark.hadoop.fs.s3a.secret.key",secretKey)
>> val sc = new SparkContext(conf)
>>
>> I tried setting these propeties in hdfs-site.xml but i'm still getting this
>> error.
>
>
>
> try core-site.xml rather than hdfs-site.xml; the latter only gets loaded when
> an HdfsConfiguration() instances is created; it may be a bit too late.
>
>> Finally I tried to set the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
>> environment variables but with no luck.
>
> Those env vars aren't picked up directly by S3a (well, that was fixed over
> the weekend https://issues.apache.org/jira/browse/HADOOP-12807 ); There's
> some fixup in spark ( see
> SparkHadoopUtil.appendS3AndSparkHadoopConfigurations() ); I don't know if
> that is a factor;
>
>> Any ideas on how to resolve this issue ?
>>
>> Thank you.
>> Daniel
>>
>> Thank you.
>> Daniel
>
