[
https://issues.apache.org/jira/browse/HADOOP-16393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943031#comment-16943031
]
Steve Loughran edited comment on HADOOP-16393 at 10/2/19 5:46 PM:
------------------------------------------------------------------
still seeing this
{{```}}
{{ {bin/hadoop s3guard init s3a://hwdev-steve-ireland-new/}}
{{ 2019-10-02 18:42:22,019 [main] DEBUG s3guard.S3GuardTool
(S3GuardTool.java:run(1758)) - Executing command init}}
{{ 2019-10-02 18:42:22,045 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1137)) - Propagating entries under
fs.s3a.bucket.hwdev-steve-ireland-new.}}
{{ 2019-10-02 18:42:22,047 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating fs.s3a.committer.name
from [core-site.xml]}}
{{ 2019-10-02 18:42:22,048 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating fs.s3a.endpoint from
[core-site.xml]}}
{{ 2019-10-02 18:42:22,048 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating
fs.s3a.committer.magic.enabled from [core-site.xml]}}
{{ 2019-10-02 18:42:22,048 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating
fs.s3a.metadatastore.impl from [core-site.xml]}}
{{ java.lang.IllegalArgumentException: No DynamoDB table name configured}}
{{ at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:141)}}
{{ at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:484)}}
{{ at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.initMetadataStore(S3GuardTool.java:320)}}
{{ at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$Init.run(S3GuardTool.java:542)}}
{{ at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:427)}}
{{ at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)}}
{{ at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:1797)}}
{{ at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.main(S3GuardTool.java:1806)}}
{{ 2019-10-02 18:42:22,071 [main] INFO util.ExitUtil
(ExitUtil.java:terminate(210)) - Exiting with status -1:
java.lang.IllegalArgumentException: No DynamoDB table name configured}}
{{ }}}}
{{ ```}}
was (Author: [email protected]):
still seeing this
```
bin/hadoop s3guard init s3a://hwdev-steve-ireland-new/
2019-10-02 18:42:22,019 [main] DEBUG s3guard.S3GuardTool
(S3GuardTool.java:run(1758)) - Executing command init
2019-10-02 18:42:22,045 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1137)) - Propagating entries under
fs.s3a.bucket.hwdev-steve-ireland-new.
2019-10-02 18:42:22,047 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating fs.s3a.committer.name
from [core-site.xml]
2019-10-02 18:42:22,048 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating fs.s3a.endpoint from
[core-site.xml]
2019-10-02 18:42:22,048 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating
fs.s3a.committer.magic.enabled from [core-site.xml]
2019-10-02 18:42:22,048 [main] DEBUG s3a.S3AUtils
(S3AUtils.java:propagateBucketOptions(1158)) - Updating
fs.s3a.metadatastore.impl from [core-site.xml]
java.lang.IllegalArgumentException: No DynamoDB table name configured
at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:141)
at
org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:484)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.initMetadataStore(S3GuardTool.java:320)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$Init.run(S3GuardTool.java:542)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:427)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:1797)
at
org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.main(S3GuardTool.java:1806)
2019-10-02 18:42:22,071 [main] INFO util.ExitUtil
(ExitUtil.java:terminate(210)) - Exiting with status -1:
java.lang.IllegalArgumentException: No DynamoDB table name configured
```
> S3Guard init command uses global settings, not those of target bucket
> ---------------------------------------------------------------------
>
> Key: HADOOP-16393
> URL: https://issues.apache.org/jira/browse/HADOOP-16393
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Fix For: 3.3.0
>
>
> If you call {{s3guard init s3a://name/}} then the custom bucket options of
> fs.s3a.bucket.name are not picked up, instead the global value is used.
> Fix: take the name of the bucket and use that to eval properties and patch
> the config used for the init command.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
