[
https://issues.apache.org/jira/browse/DRILL-6511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-6511:
-------------------------------
Description:
When using the above approach, it appears that one must specify the endpoint:
{code}
"connection": "s3a://<bucket-name>/",
"config": {
"fs.s3a.access.key": "<key>",
"fs.s3a.secret.key": "<key>",
"fs.s3a.endpoint": "s3.us-west-1.amazonaws.com"
},
{code}
I could not get the above to work using the pattern in the default S3 config:
{code}
connection: "s3a://my.bucket.location.com",
{code}
Using the endpoint is how all S3a examples I could find described the usage. It
is also consistent with the [s3a
documentation|https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A].
Note that the {{my.bucket.location.com}} example is never right. The format
must be valid for Amazon.
Since the above is not secure at all, it is better to specify all the
properties in the {{core-site.xml}} file, either the one already provided in
Hadoop, or one created for Drill:
{code}
<configuration>
<property>
<name>fs.s3a.access.key</name>
<value>ACCESS-KEY</value>
</property>
<property>
<name>fs.s3a.secret.key</name>
<value>SECRET-KEY</value>
</property>
<property>
<name>fs.s3a.endpoint</name>
<value>s3.REGION.amazonaws.com</value>
</property>
</configuration>
{code}
The {{core-site-example.xml}} file already provides the first two properties.
The specific request here is to add the third item.
was:
When using the above approach, it appears that one must specify the endpoint:
{code}
"connection": "s3a://<bucket-name>/",
"config": {
"fs.s3a.access.key": "<key>",
"fs.s3a.secret.key": "<key>",
"fs.s3a.endpoint": "s3.us-west-1.amazonaws.com"
},
{code}
I could not get the above to work using the pattern in the default S3 config:
{code}
connection: "s3a://my.bucket.location.com",
{code}
Using the endpoint is how all S3a examples I could find described the usage. It
is also consistent with the [s3a
documentation|https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A].
Note that the {{my.bucket.location.com}} example is never right. The format
must be valid for Amazon.
> Bootstrap S3 storage plugin is misleading
> -----------------------------------------
>
> Key: DRILL-6511
> URL: https://issues.apache.org/jira/browse/DRILL-6511
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.13.0
> Reporter: Paul Rogers
> Priority: Minor
>
> When using the above approach, it appears that one must specify the endpoint:
> {code}
> "connection": "s3a://<bucket-name>/",
> "config": {
> "fs.s3a.access.key": "<key>",
> "fs.s3a.secret.key": "<key>",
> "fs.s3a.endpoint": "s3.us-west-1.amazonaws.com"
> },
> {code}
> I could not get the above to work using the pattern in the default S3 config:
> {code}
> connection: "s3a://my.bucket.location.com",
> {code}
> Using the endpoint is how all S3a examples I could find described the usage.
> It is also consistent with the [s3a
> documentation|https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A].
> Note that the {{my.bucket.location.com}} example is never right. The format
> must be valid for Amazon.
> Since the above is not secure at all, it is better to specify all the
> properties in the {{core-site.xml}} file, either the one already provided in
> Hadoop, or one created for Drill:
> {code}
> <configuration>
> <property>
> <name>fs.s3a.access.key</name>
> <value>ACCESS-KEY</value>
> </property>
> <property>
> <name>fs.s3a.secret.key</name>
> <value>SECRET-KEY</value>
> </property>
> <property>
> <name>fs.s3a.endpoint</name>
> <value>s3.REGION.amazonaws.com</value>
> </property>
> </configuration>
> {code}
> The {{core-site-example.xml}} file already provides the first two properties.
> The specific request here is to add the third item.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
