[jira] [Updated] (HADOOP-14013) S3Guard: fix multi-bucket integration tests

2017-02-06 Thread Aaron Fabbri (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Fabbri updated HADOOP-14013:
--
   Resolution: Fixed
Fix Version/s: HADOOP-13345
   Status: Resolved  (was: Patch Available)

Committed to feature branch.. Thanks [~ste...@apache.org] and [~liuml07] for 
review and suggestions on this one.

> S3Guard: fix multi-bucket integration tests
> ---
>
> Key: HADOOP-14013
> URL: https://issues.apache.org/jira/browse/HADOOP-14013
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Fix For: HADOOP-13345
>
> Attachments: HADOOP-14013-HADOOP-13345.001.patch
>
>
> Clone of HADOOP-13876 to address multi-bucket test failures separately.
> One of the issues is with accessing read-only buckets.  If a user accesses a 
> read-only bucket with credentials that do not have DynamoDB write 
> permissions, they will get errors when trying to access the read-only bucket. 
>  This manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.
> Goals for this JIRA:
> - Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the 
> real use-case.
> - Document limitations etc. in the s3guard.md site doc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14013) S3Guard: fix multi-bucket integration tests

2017-02-02 Thread Aaron Fabbri (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Fabbri updated HADOOP-14013:
--
Attachment: HADOOP-14013-HADOOP-13345.001.patch

Attaching v1 patch, based on suggestions from [~steve_l] and [~liuml07].

Also removes example s3guard test settings from the test core-site.xml since we 
have the maven profiles now.

Tested via 
{noformat}
mvn clean verify -Ddynamo -Ds3guard -Dit.test=ITestS3AAWSCredentialsProvider 
-Dtest=none
{noformat}

Without the patch we see the familiar failure:
{noformat}
ITestS3AAWSCredentialsProvider.testAnonymousProvider:132 ยป AWSServiceIO initTa..
{noformat}

With the patch it succeeds, as expected.

> S3Guard: fix multi-bucket integration tests
> ---
>
> Key: HADOOP-14013
> URL: https://issues.apache.org/jira/browse/HADOOP-14013
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Attachments: HADOOP-14013-HADOOP-13345.001.patch
>
>
> Clone of HADOOP-13876 to address multi-bucket test failures separately.
> One of the issues is with accessing read-only buckets.  If a user accesses a 
> read-only bucket with credentials that do not have DynamoDB write 
> permissions, they will get errors when trying to access the read-only bucket. 
>  This manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.
> Goals for this JIRA:
> - Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the 
> real use-case.
> - Document limitations etc. in the s3guard.md site doc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14013) S3Guard: fix multi-bucket integration tests

2017-02-02 Thread Aaron Fabbri (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Fabbri updated HADOOP-14013:
--
Status: Patch Available  (was: Open)

> S3Guard: fix multi-bucket integration tests
> ---
>
> Key: HADOOP-14013
> URL: https://issues.apache.org/jira/browse/HADOOP-14013
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
> Attachments: HADOOP-14013-HADOOP-13345.001.patch
>
>
> Clone of HADOOP-13876 to address multi-bucket test failures separately.
> One of the issues is with accessing read-only buckets.  If a user accesses a 
> read-only bucket with credentials that do not have DynamoDB write 
> permissions, they will get errors when trying to access the read-only bucket. 
>  This manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.
> Goals for this JIRA:
> - Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the 
> real use-case.
> - Document limitations etc. in the s3guard.md site doc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Updated] (HADOOP-14013) S3Guard: fix multi-bucket integration tests

2017-01-20 Thread Aaron Fabbri (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HADOOP-14013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aaron Fabbri updated HADOOP-14013:
--
Description: 
Clone of HADOOP-13876 to address multi-bucket test failures separately.

One of the issues is with accessing read-only buckets.  If a user accesses a 
read-only bucket with credentials that do not have DynamoDB write permissions, 
they will get errors when trying to access the read-only bucket.  This 
manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.

Goals for this JIRA:
- Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the real 
use-case.
- Document limitations etc. in the s3guard.md site doc.



  was:
HADOOP-13449 adds support for DynamoDBMetadataStore.

The code currently supports two options for choosing DynamoDB table names:
1. Use name of each s3 bucket and auto-create a DynamoDB table for each.
2. Configure a table name in the {{fs.s3a.s3guard.ddb.table}} parameter.

One of the issues is with accessing read-only buckets.  If a user accesses a 
read-only bucket with credentials that do not have DynamoDB write permissions, 
they will get errors when trying to access the read-only bucket.  This 
manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.

Goals for this JIRA:
- Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the real 
use-case.
- Allow for a "one DynamoDB table per cluster" configuration with a way to 
chose which credentials are used for DynamoDB.
- Document limitations etc. in the s3guard.md site doc.




> S3Guard: fix multi-bucket integration tests
> ---
>
> Key: HADOOP-14013
> URL: https://issues.apache.org/jira/browse/HADOOP-14013
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/s3
>Affects Versions: HADOOP-13345
>Reporter: Aaron Fabbri
>Assignee: Aaron Fabbri
>
> Clone of HADOOP-13876 to address multi-bucket test failures separately.
> One of the issues is with accessing read-only buckets.  If a user accesses a 
> read-only bucket with credentials that do not have DynamoDB write 
> permissions, they will get errors when trying to access the read-only bucket. 
>  This manifests causes test failures for {{ITestS3AAWSCredentialsProvider}}.
> Goals for this JIRA:
> - Fix {{ITestS3AAWSCredentialsProvider}} in a way that makes sense for the 
> real use-case.
> - Document limitations etc. in the s3guard.md site doc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org