[jira] [Updated] (HADOOP-13904) DynamoDBMetadataStore to handle DDB throttling failures through retry policy

2017-01-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13904: -- Attachment: HADOOP-13904-HADOOP-13345.001.patch Attaching v1 patch. It adds new scale tests

[jira] [Commented] (HADOOP-13904) DynamoDBMetadataStore to handle DDB throttling failures through retry policy

2017-01-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829036#comment-15829036 ] Aaron Fabbri commented on HADOOP-13904: --- Taking this JIRA since [~liuml07] is on vacation. I'll

[jira] [Assigned] (HADOOP-13904) DynamoDBMetadataStore to handle DDB throttling failures through retry policy

2017-01-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13904: - Assignee: Aaron Fabbri (was: Mingliang Liu) > DynamoDBMetadataStore to handle DDB

[jira] [Commented] (HADOOP-13904) DynamoDBMetadataStore to handle DDB throttling failures through retry policy

2017-01-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828829#comment-15828829 ] Aaron Fabbri commented on HADOOP-13904: --- I'm creating a new scale test on top of MetadataStore so

[jira] [Commented] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828801#comment-15828801 ] Aaron Fabbri commented on HADOOP-13877: --- Hey [~ste...@apache.org] can I get a quick review on

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828564#comment-15828564 ] Aaron Fabbri commented on HADOOP-13589: --- I'm happy to test and commit this if that helps. Will

[jira] [Commented] (HADOOP-13995) s3guard cli: make tests easier to run and address failure

2017-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826810#comment-15826810 ] Aaron Fabbri commented on HADOOP-13995: --- [~mackrorysd] wants to work on this, but shout if you

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826812#comment-15826812 ] Aaron Fabbri commented on HADOOP-13650: --- Created HADOOP-13995. > S3Guard: Provide command line

[jira] [Created] (HADOOP-13995) s3guard cli: make tests easier to run and address failure

2017-01-17 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13995: - Summary: s3guard cli: make tests easier to run and address failure Key: HADOOP-13995 URL: https://issues.apache.org/jira/browse/HADOOP-13995 Project: Hadoop Common

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826761#comment-15826761 ] Aaron Fabbri commented on HADOOP-13650: --- Hi [~cnauroth]. I also hit some errors in my testing.

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15822564#comment-15822564 ] Aaron Fabbri commented on HADOOP-13650: --- Thanks for the followup patch [~eddyxu]. +1 on the code

[jira] [Updated] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13877: -- Status: Patch Available (was: Open) > S3Guard: fix TestDynamoDBMetadataStore when

[jira] [Comment Edited] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15822506#comment-15822506 ] Aaron Fabbri edited comment on HADOOP-13877 at 1/13/17 11:15 PM: -

[jira] [Updated] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13877: -- Status: Open (was: Patch Available) > S3Guard: fix TestDynamoDBMetadataStore when

[jira] [Updated] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13877: -- Attachment: HADOOP-13877-HADOOP-13345.003.patch Attaching v3 patch. Adds [~liuml07]'s

[jira] [Commented] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820036#comment-15820036 ] Aaron Fabbri commented on HADOOP-13877: --- +1 v2 patch, pending testing. I'm cool with the

[jira] [Commented] (HADOOP-13981) S3Guard CLI: Add documentation

2017-01-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820005#comment-15820005 ] Aaron Fabbri commented on HADOOP-13981: --- Is there a central place for CLI documentation, or would

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15820003#comment-15820003 ] Aaron Fabbri commented on HADOOP-13650: --- I created another JIRA HADOOP-13980 for the fsck stuff.

[jira] [Created] (HADOOP-13981) S3Guard CLI: Add documentation

2017-01-11 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13981: - Summary: S3Guard CLI: Add documentation Key: HADOOP-13981 URL: https://issues.apache.org/jira/browse/HADOOP-13981 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-13980) S3Guard CLI: Add fsck check command

2017-01-11 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13980: - Summary: S3Guard CLI: Add fsck check command Key: HADOOP-13980 URL: https://issues.apache.org/jira/browse/HADOOP-13980 Project: Hadoop Common Issue Type:

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819990#comment-15819990 ] Aaron Fabbri commented on HADOOP-13650: --- Also wanted to comment on the addition of fsck features.

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819969#comment-15819969 ] Aaron Fabbri commented on HADOOP-13650: --- Nice work on this [~eddyxu]. Overall I think it looks

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815897#comment-15815897 ] Aaron Fabbri commented on HADOOP-13345: --- +1 merging in trunk.. Works for me [~ste...@apache.org]

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813303#comment-15813303 ] Aaron Fabbri commented on HADOOP-13589: --- This looks awesome [~ste...@apache.org]. One minor nit:

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-01-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813283#comment-15813283 ] Aaron Fabbri commented on HADOOP-13914: --- Ok, thanks for clarifying. Enjoy your break! > s3guard:

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-01-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813249#comment-15813249 ] Aaron Fabbri commented on HADOOP-13914: --- Thanks for uploading an RFC patch [~liuml07]. Did you

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2017-01-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813216#comment-15813216 ] Aaron Fabbri commented on HADOOP-13914: --- Thanks [~ste...@apache.org]. I'm not sure I'm parsing

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805850#comment-15805850 ] Aaron Fabbri commented on HADOOP-13336: --- +1 (binding only on HADOOP-13345 feature branch).

[jira] [Commented] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805631#comment-15805631 ] Aaron Fabbri commented on HADOOP-13877: --- Agreed, but different test cases use different bucket

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805590#comment-15805590 ] Aaron Fabbri commented on HADOOP-13336: --- +1 (non-binding) once you remove the change of debug log

[jira] [Updated] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13877: -- Attachment: HADOOP-13877-HADOOP-13345.001.patch Attaching patch. Fix is just to be

[jira] [Updated] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13877: -- Status: Patch Available (was: Open) > S3Guard: fix TestDynamoDBMetadataStore when

[jira] [Commented] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803408#comment-15803408 ] Aaron Fabbri commented on HADOOP-13877: --- Here are the two failures I see when I set

[jira] [Commented] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2017-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803075#comment-15803075 ] Aaron Fabbri commented on HADOOP-13877: --- Yep.. sorry for delay.. Let me reproduce this on latest

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802338#comment-15802338 ] Aaron Fabbri commented on HADOOP-13345: --- Thanks [~ste...@apache.org], this is good stuff. Sounds

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802302#comment-15802302 ] Aaron Fabbri commented on HADOOP-13345: --- Hi [~cnauroth]. Just to clarify, you can run all the S3A

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1577#comment-1577 ] Aaron Fabbri commented on HADOOP-13345: --- Thanks for your input [~liuml07]. The reason I propose

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799249#comment-15799249 ] Aaron Fabbri commented on HADOOP-13345: --- Happy new year [~ste...@apache.org] and [~liuml07]. I'd

[jira] [Commented] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2016-12-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756111#comment-15756111 ] Aaron Fabbri commented on HADOOP-13914: --- [~liuml07] [~steve_l] for your reading pleasure. I will

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2016-12-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: test-only-HADOOP-13914.patch Attaching patch for new integration test I wrote

[jira] [Updated] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2016-12-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13914: -- Attachment: s3guard-empty-dirs.md Attaching s3guard-empty-dirs.md doc which goes into

[jira] [Created] (HADOOP-13914) s3guard: improve S3AFileStatus#isEmptyDirectory handling

2016-12-16 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13914: - Summary: s3guard: improve S3AFileStatus#isEmptyDirectory handling Key: HADOOP-13914 URL: https://issues.apache.org/jira/browse/HADOOP-13914 Project: Hadoop Common

[jira] [Commented] (HADOOP-13455) S3Guard: Write end user docs, change table autocreate default.

2016-12-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750280#comment-15750280 ] Aaron Fabbri commented on HADOOP-13455: --- Committed to HADOOP-13345 feature branch. Thanks for

[jira] [Updated] (HADOOP-13455) S3Guard: Write end user docs, change table autocreate default.

2016-12-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13455: -- Resolution: Fixed Status: Resolved (was: Patch Available) > S3Guard: Write end user

[jira] [Commented] (HADOOP-13455) S3Guard: Write end user docs, change table autocreate default.

2016-12-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15750195#comment-15750195 ] Aaron Fabbri commented on HADOOP-13455: --- +1 Thanks [~liuml07]. I assumed the default for table

[jira] [Updated] (HADOOP-13455) S3Guard: Write end user docs, change table autocreate default.

2016-12-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13455: -- Summary: S3Guard: Write end user docs, change table autocreate default. (was: S3Guard: Write

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2016-12-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749803#comment-15749803 ] Aaron Fabbri commented on HADOOP-13336: --- Putting the bucket name at the end seems more correct,

[jira] [Commented] (HADOOP-13899) tune dynamodb client & tests

2016-12-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746771#comment-15746771 ] Aaron Fabbri commented on HADOOP-13899: --- +1, modulo [~liuml07]'s comments on the DDB stuff. One

[jira] [Commented] (HADOOP-13455) S3Guard: Write end user documentation.

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744117#comment-15744117 ] Aaron Fabbri commented on HADOOP-13455: --- Notice I also added a missing DynamoDB parameter to

[jira] [Comment Edited] (HADOOP-13455) S3Guard: Write end user documentation.

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744117#comment-15744117 ] Aaron Fabbri edited comment on HADOOP-13455 at 12/13/16 4:27 AM: -

[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13455: -- Affects Version/s: HADOOP-13345 Status: Patch Available (was: In Progress) >

[jira] [Updated] (HADOOP-13455) S3Guard: Write end user documentation.

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13455: -- Attachment: HADOOP-13455-HADOOP-13345.001.patch [~steve_l] ask and you shall receive. I took

[jira] [Work started] (HADOOP-13455) S3Guard: Write end user documentation.

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-13455 started by Aaron Fabbri. - > S3Guard: Write end user documentation. >

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743591#comment-15743591 ] Aaron Fabbri commented on HADOOP-13336: --- Just noticed you suggest the same thing about having a

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743567#comment-15743567 ] Aaron Fabbri commented on HADOOP-13336: --- Great summary [~steve_l]. I think being

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743497#comment-15743497 ] Aaron Fabbri commented on HADOOP-13449: --- Thanks [~steve_l]. Those tests that make assertions

[jira] [Commented] (HADOOP-13886) s3guard: ITestS3AFileOperationCost.testFakeDirectoryDeletion failure

2016-12-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736762#comment-15736762 ] Aaron Fabbri commented on HADOOP-13886: --- [~liuml07] wrote an initial analysis of this failure

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736704#comment-15736704 ] Aaron Fabbri commented on HADOOP-13449: --- You are welcome! Thanks also to [~eddyxu] for his help

[jira] [Updated] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13449: -- Resolution: Fixed Fix Version/s: HADOOP-13345 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736682#comment-15736682 ] Aaron Fabbri commented on HADOOP-13449: --- I committed this to the HADOOP-13345 feature branch.

[jira] [Created] (HADOOP-13886) s3guard: ITestS3AFileOperationCost.testFakeDirectoryDeletion failure

2016-12-09 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13886: - Summary: s3guard: ITestS3AFileOperationCost.testFakeDirectoryDeletion failure Key: HADOOP-13886 URL: https://issues.apache.org/jira/browse/HADOOP-13886 Project:

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734138#comment-15734138 ] Aaron Fabbri commented on HADOOP-13449: --- Sounds good. Thanks for analysis on that failure. I will

[jira] [Created] (HADOOP-13877) S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set

2016-12-08 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13877: - Summary: S3Guard: fix TestDynamoDBMetadataStore when fs.s3a.s3guard.ddb.table is set Key: HADOOP-13877 URL: https://issues.apache.org/jira/browse/HADOOP-13877

[jira] [Created] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2016-12-08 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-13876: - Summary: S3Guard: better support for multi-bucket access including read-only Key: HADOOP-13876 URL: https://issues.apache.org/jira/browse/HADOOP-13876 Project:

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733756#comment-15733756 ] Aaron Fabbri commented on HADOOP-13449: --- I'm in the process of testing the v12 patch. [~liuml07]

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2016-12-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733598#comment-15733598 ] Aaron Fabbri commented on HADOOP-13345: --- Looks like you did the commit *and* you also did a merge

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2016-12-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15730226#comment-15730226 ] Aaron Fabbri commented on HADOOP-13345: --- Sounds good. > S3Guard: Improved Consistency for S3A >

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2016-12-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15729951#comment-15729951 ] Aaron Fabbri commented on HADOOP-13345: --- +1 on adding patch v2 from HADOOP-13852 to the s3guard

[jira] [Comment Edited] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727382#comment-15727382 ] Aaron Fabbri edited comment on HADOOP-13449 at 12/7/16 1:55 AM: If it is

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727382#comment-15727382 ] Aaron Fabbri commented on HADOOP-13449: --- If it is MetadataStoreTestBase line 154

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727368#comment-15727368 ] Aaron Fabbri commented on HADOOP-13449: --- What line does testPutNew() fail on for you?

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727226#comment-15727226 ] Aaron Fabbri commented on HADOOP-13449: --- This looks pretty good [~liuml07], thank you. A couple

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-12-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Attachment: HADOOP-13793-HADOOP-13345.003.patch Attaching v3 patch addressing [~liuml07]'s

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-12-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Patch Available (was: Open) > s3guard: add inconsistency injection, integration

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-12-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Open (was: Patch Available) > s3guard: add inconsistency injection, integration

[jira] [Commented] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-12-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723584#comment-15723584 ] Aaron Fabbri commented on HADOOP-13793: --- Thanks for the review [~liuml07]. These are good

[jira] [Commented] (HADOOP-13075) Add support for SSE-KMS and SSE-C in s3a filesystem

2016-12-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15722930#comment-15722930 ] Aaron Fabbri commented on HADOOP-13075: --- I'm interested in getting this stuff in. We can probably

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717021#comment-15717021 ] Aaron Fabbri commented on HADOOP-13449: --- I did a little research on #3. It looks like you cannot

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15716696#comment-15716696 ] Aaron Fabbri commented on HADOOP-13449: --- I think this is the list of outstanding items to get

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-12-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713155#comment-15713155 ] Aaron Fabbri commented on HADOOP-13449: --- That use case for Mock S3 Client + Real DDB client (for

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710879#comment-15710879 ] Aaron Fabbri commented on HADOOP-13449: --- FYI so we don't duplicate effort: I'm looking at

[jira] [Commented] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15710216#comment-15710216 ] Aaron Fabbri commented on HADOOP-13793: --- All s3a integration tests passed with default endpoint in

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Patch Available (was: Open) > s3guard: add inconsistency injection, integration

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Attachment: HADOOP-13793-HADOOP-13345.002.patch Reattaching patch: removing "Created by

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Attachment: (was: HADOOP-13793-HADOOP-13345.002.patch) > s3guard: add inconsistency

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Open (was: Patch Available) > s3guard: add inconsistency injection, integration

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Attachment: HADOOP-13793-HADOOP-13345.002.patch Attaching updated patch: rebased on latest

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Patch Available (was: Open) > s3guard: add inconsistency injection, integration

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Open (was: Patch Available) > s3guard: add inconsistency injection, integration

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15709115#comment-15709115 ] Aaron Fabbri commented on HADOOP-13449: --- Finished a run of the S3A integration tests. I see that

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707310#comment-15707310 ] Aaron Fabbri commented on HADOOP-13449: --- This fixes that failed unit test for me: {code} ---

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15707232#comment-15707232 ] Aaron Fabbri commented on HADOOP-13449: --- Congrats on the clean jenkins run [~liuml07]. I'm

[jira] [Comment Edited] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688223#comment-15688223 ] Aaron Fabbri edited comment on HADOOP-13449 at 11/22/16 11:05 PM: -- Yes,

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688223#comment-15688223 ] Aaron Fabbri commented on HADOOP-13449: --- Yes, I believe that is what Steve L was suggesting. Go

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688188#comment-15688188 ] Aaron Fabbri commented on HADOOP-13449: --- Sounds good [~liuml07]. Last time I did a rebase but

[jira] [Updated] (HADOOP-13651) S3Guard: S3AFileSystem Integration with MetadataStore

2016-11-21 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13651: -- Resolution: Fixed Fix Version/s: HADOOP-13345 Status: Resolved (was: Patch

[jira] [Updated] (HADOOP-13793) s3guard: add inconsistency injection, integration tests

2016-11-21 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13793: -- Status: Patch Available (was: In Progress) > s3guard: add inconsistency injection,

[jira] [Commented] (HADOOP-13449) S3Guard: Implement DynamoDBMetadataStore.

2016-11-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677910#comment-15677910 ] Aaron Fabbri commented on HADOOP-13449: --- Hey [~liuml07]. I am available early next week to help

[jira] [Commented] (HADOOP-13651) S3Guard: S3AFileSystem Integration with MetadataStore

2016-11-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677632#comment-15677632 ] Aaron Fabbri commented on HADOOP-13651: --- Sorry I just noticed your question. I just committed

[jira] [Commented] (HADOOP-13651) S3Guard: S3AFileSystem Integration with MetadataStore

2016-11-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15677627#comment-15677627 ] Aaron Fabbri commented on HADOOP-13651: --- Committed to HADOOP-13345 branch. Thank you everyone who

<    5   6   7   8   9   10   11   12   13   >