[jira] [Updated] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-08-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14154: -- Attachment: run-dir-perf-itest.sh > Persist isAuthoritative bit in DynamoDBMetaStore

[jira] [Updated] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-08-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14154: -- Attachment: perf-eval-v1.diff > Persist isAuthoritative bit in DynamoDBMetaStore

[jira] [Commented] (HADOOP-15400) Improve S3Guard documentation on Authoritative Mode implementation

2018-08-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572678#comment-16572678 ] Aaron Fabbri commented on HADOOP-15400: --- Sorry for the delay on review but wanted to say this

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-08-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572676#comment-16572676 ] Aaron Fabbri commented on HADOOP-14154: --- v3 patch looks good.  I ran through all the integration

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-08-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571116#comment-16571116 ] Aaron Fabbri commented on HADOOP-14154: --- Thanks for the patch.  Running through some tests..

[jira] [Commented] (HADOOP-15651) NPE in S3AInputStream.read() in ITestS3AInconsistency.testOpenFailOnRead

2018-08-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570738#comment-16570738 ] Aaron Fabbri commented on HADOOP-15651: --- Yeah, this one is not obvious.  Let's see: * Stack

[jira] [Commented] (HADOOP-15626) FileContextMainOperationsBaseTest.testBuilderCreateAppendExistingFile fails on filesystems without append.

2018-08-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567776#comment-16567776 ] Aaron Fabbri commented on HADOOP-15626: --- +1 LTGM thanks for fixing this. >

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-08-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567775#comment-16567775 ] Aaron Fabbri commented on HADOOP-14154: --- Thanks for the v2 patch.  The DDBPathMetadata stuff

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564734#comment-16564734 ] Aaron Fabbri commented on HADOOP-14154: --- Ok.. I actually looked at the right patch today ;). 

[jira] [Commented] (HADOOP-15642) Update to latest/recent version of aws-sdk for Hadoop 3.2

2018-07-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16564588#comment-16564588 ] Aaron Fabbri commented on HADOOP-15642: --- ok, on my list. keep bugging me if needed.. I also need

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563165#comment-16563165 ] Aaron Fabbri commented on HADOOP-14154: --- Yes. you are right. I just thought of this before going

[jira] [Comment Edited] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562984#comment-16562984 ] Aaron Fabbri edited comment on HADOOP-14154 at 7/31/18 5:46 AM: Thanks

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16562984#comment-16562984 ] Aaron Fabbri commented on HADOOP-14154: --- Thanks for the patch. {quote} * Added 

[jira] [Commented] (HADOOP-15634) LocalDirAllocator using up local nonDFS when set to S3

2018-07-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558954#comment-16558954 ] Aaron Fabbri commented on HADOOP-15634: --- {quote}why do we need local disk. {quote} For future

[jira] [Commented] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-07-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558924#comment-16558924 ] Aaron Fabbri commented on HADOOP-15426: --- This sounds right based on the testing I did in the

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558656#comment-16558656 ] Aaron Fabbri commented on HADOOP-14154: --- {quote} As my current understanding using a dynamodb

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-20 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551078#comment-16551078 ] Aaron Fabbri commented on HADOOP-14154: --- Sounds good to me. I'll leave the decision how to split

[jira] [Commented] (HADOOP-14154) Persist isAuthoritative bit in DynamoDBMetaStore (authoritative mode support)

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549960#comment-16549960 ] Aaron Fabbri commented on HADOOP-14154: --- Doc looks great [~gabor.bota]. One minor comment:

[jira] [Updated] (HADOOP-15621) s3guard: implement time-based (TTL) expiry for DynamoDB Metadata Store

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15621: -- Description: Similar to HADOOP-13649, I think we should add a TTL (time to live) feature to

[jira] [Updated] (HADOOP-15621) s3guard: implement time-based (TTL) expiry for DynamoDB Metadata Store

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15621: -- Fix Version/s: (was: 3.2.0) > s3guard: implement time-based (TTL) expiry for DynamoDB

[jira] [Created] (HADOOP-15621) s3guard: implement time-based (TTL) expiry for DynamoDB Metadata Store

2018-07-19 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-15621: - Summary: s3guard: implement time-based (TTL) expiry for DynamoDB Metadata Store Key: HADOOP-15621 URL: https://issues.apache.org/jira/browse/HADOOP-15621 Project:

[jira] [Assigned] (HADOOP-14109) improvements to S3GuardTool destroy command

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14109: - Assignee: (was: Abraham Fine) > improvements to S3GuardTool destroy command >

[jira] [Assigned] (HADOOP-14757) S3AFileSystem.innerRename() to size metadatastore lists better

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14757: - Assignee: (was: Abraham Fine) > S3AFileSystem.innerRename() to size metadatastore

[jira] [Assigned] (HADOOP-15140) S3guard mistakes root URI without / as non-absolute path

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-15140: - Assignee: (was: Abraham Fine) > S3guard mistakes root URI without / as

[jira] [Assigned] (HADOOP-14734) add option to tag DDB table(s) created

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14734: - Assignee: (was: Abraham Fine) > add option to tag DDB table(s) created >

[jira] [Assigned] (HADOOP-13980) S3Guard CLI: Add fsck check command

2018-07-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13980: - Assignee: (was: Abraham Fine) > S3Guard CLI: Add fsck check command >

[jira] [Commented] (HADOOP-15489) S3Guard to self update on directory listings of S3

2018-07-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548655#comment-16548655 ] Aaron Fabbri commented on HADOOP-15489: --- [~ste...@apache.org] see my comment above. Is there

[jira] [Commented] (HADOOP-14757) S3AFileSystem.innerRename() to size metadatastore lists better

2018-07-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16548635#comment-16548635 ] Aaron Fabbri commented on HADOOP-14757: --- [~abrahamfine] I'll unassign these S3A JIRAs from you,

[jira] [Updated] (HADOOP-15215) s3guard set-capacity command to fail on read/write of 0

2018-07-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15215: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: (was: image-2018-07-02-11-08-41-302.png) > Remove HTrace support >

[jira] [Commented] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530265#comment-16530265 ] Aaron Fabbri commented on HADOOP-15566: --- Nice work [~jojochuang].  I had fun hacking on this for

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: ss-trace-s3a.png > Remove HTrace support > - > >

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: (was: ss-trace-s3a.png) > Remove HTrace support > - > >

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: image-2018-07-02-11-08-41-302.png > Remove HTrace support >

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: ss-trace-s3a.png > Remove HTrace support > - > >

[jira] [Commented] (HADOOP-15566) Remove HTrace support

2018-06-28 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526619#comment-16526619 ] Aaron Fabbri commented on HADOOP-15566: --- Agreed.  Htrace is great but suffered from everyone

[jira] [Updated] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

2018-06-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15563: -- Description: To keep costs down on DDB, autoscaling is a key feature: you set the max values

[jira] [Updated] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

2018-06-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15563: -- Environment: (was: To keep costs down on DDB, autoscaling is a key feature: you set the

[jira] [Commented] (HADOOP-15215) s3guard set-capacity command to fail on read/write of 0

2018-06-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525367#comment-16525367 ] Aaron Fabbri commented on HADOOP-15215: --- LGTM +1.  Good work here [~gabor.bota]. I think findbugs

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506723#comment-16506723 ] Aaron Fabbri commented on HADOOP-15525: --- Actually, refreshing my memory here,

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506712#comment-16506712 ] Aaron Fabbri commented on HADOOP-15525: --- The existing docs

[jira] [Updated] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15525: -- Description: Scenario: customer wants to only give a Hadoop cluster access to a subtree of

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506673#comment-16506673 ] Aaron Fabbri commented on HADOOP-15525: --- Thanks [~poeppt]. We do have assume role support in S3A

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506574#comment-16506574 ] Aaron Fabbri commented on HADOOP-15525: --- Assigning to me for now. I'd like to write a doc here to

[jira] [Assigned] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-15525: - Assignee: Aaron Fabbri > s3a: clarify / improve support for mixed ACL buckets >

[jira] [Updated] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15525: -- Description: Scenario: customer wants to only give a Hadoop cluster access to a subtree of

[jira] [Created] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-15525: - Summary: s3a: clarify / improve support for mixed ACL buckets Key: HADOOP-15525 URL: https://issues.apache.org/jira/browse/HADOOP-15525 Project: Hadoop Common

[jira] [Resolved] (HADOOP-11697) Use larger value for fs.s3a.connection.timeout.

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri resolved HADOOP-11697. --- Resolution: Duplicate Fix Version/s: 3.0.0 > Use larger value for

[jira] [Commented] (HADOOP-11697) Use larger value for fs.s3a.connection.timeout.

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506552#comment-16506552 ] Aaron Fabbri commented on HADOOP-11697: --- I think we can close this, since HADOOP-12346 bumped

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-06-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503693#comment-16503693 ] Aaron Fabbri commented on HADOOP-15407: --- Thanks.. was still reading through latest patch. Just

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-06-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503682#comment-16503682 ] Aaron Fabbri commented on HADOOP-15407: --- Thanks for the update. In terms of us maintaining this

[jira] [Commented] (HADOOP-15507) Add MapReduce counters about EC bytes read

2018-06-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500916#comment-16500916 ] Aaron Fabbri commented on HADOOP-15507: --- +1 assuming you manually tested (i.e. your screenshot is

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-05-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16495944#comment-16495944 ] Aaron Fabbri commented on HADOOP-14918: --- Shout if you need any help testing this [~gabor.bota].

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494331#comment-16494331 ] Aaron Fabbri commented on HADOOP-15480: --- Thanks [~gabor.bota], and thank you for letting me make

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.005.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: (was: HADOOP-15480.005.patch) > AbstractS3GuardToolTestBase.testDiffCommand

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.005.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.004.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thank

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494099#comment-16494099 ] Aaron Fabbri commented on HADOOP-14946: --- I'll commit the v3 patch once yetus is clean.  (same as

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Attachment: HADOOP-14946.003.patch > S3Guard testPruneCommandCLI can fail >

[jira] [Comment Edited] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491473#comment-16491473 ] Aaron Fabbri edited comment on HADOOP-15480 at 5/26/18 2:38 AM: thanks

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491473#comment-16491473 ] Aaron Fabbri commented on HADOOP-15480: --- thanks for the update [~gabor.bota], good work. Tests

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.003.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490288#comment-16490288 ] Aaron Fabbri commented on HADOOP-14918: --- Thanks for making progress on this, [~gabor.bota], it

[jira] [Comment Edited] (HADOOP-15489) S3Guard to self update on directory listings of S3

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490281#comment-16490281 ] Aaron Fabbri edited comment on HADOOP-15489 at 5/25/18 6:08 AM: Look at

[jira] [Commented] (HADOOP-15489) S3Guard to self update on directory listings of S3

2018-05-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16490281#comment-16490281 ] Aaron Fabbri commented on HADOOP-15489: --- Look at S3Guard#dirListingUnion(). We used to always

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489949#comment-16489949 ] Aaron Fabbri commented on HADOOP-15480: --- Thanks for the info [~ste...@apache.org] {quote} if we

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489671#comment-16489671 ] Aaron Fabbri commented on HADOOP-14946: --- Thanks guys. {quote}NTP logs? {quote} Attempt at a joke /

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488508#comment-16488508 ] Aaron Fabbri commented on HADOOP-14946: --- I hope you don't mind [~gabor.bota], I've attached a v2

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Attachment: HADOOP-14946.002.patch > S3Guard testPruneCommandCLI can fail >

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-23 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488430#comment-16488430 ] Aaron Fabbri commented on HADOOP-14946: --- Looking at this again.. since I was able to reproduce an

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-23 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486803#comment-16486803 ] Aaron Fabbri commented on HADOOP-14946: --- Thank you for the patch [~gabor.bota]. +1 LGTM. I believe

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486747#comment-16486747 ] Aaron Fabbri commented on HADOOP-15480: --- Looks good [~gabor.bota], glad the hint was helpful. One

[jira] [Commented] (HADOOP-15437) AzureBlobFS - Services

2018-05-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484600#comment-16484600 ] Aaron Fabbri commented on HADOOP-15437: --- Thank you for the effort to split up the patch. Can you

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480961#comment-16480961 ] Aaron Fabbri commented on HADOOP-15480: --- I also reproduced this yesterday.  Apologies for letting

[jira] [Commented] (HADOOP-15469) S3A directory committer commit job fails if _temporary directory created under dest

2018-05-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479458#comment-16479458 ] Aaron Fabbri commented on HADOOP-15469: --- And even before change, files could arrive in job output

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478404#comment-16478404 ] Aaron Fabbri commented on HADOOP-15430: --- Great work here. Test code is very nice. {quote}+ // here

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478062#comment-16478062 ] Aaron Fabbri commented on HADOOP-14946: --- [~gabor.bota] do you mind looking at this one? If not

[jira] [Assigned] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14946: - Assignee: Gabor Bota > S3Guard testPruneCommandCLI can fail >

[jira] [Commented] (HADOOP-15469) S3A directory committer commit job fails if _temporary directory created under dest

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478057#comment-16478057 ] Aaron Fabbri commented on HADOOP-15469: --- Interesting. The argument for this seems to be (1) this

[jira] [Commented] (HADOOP-15460) S3A FS to add "s3a:no-existence-checks" to the builder file creation option set

2018-05-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475164#comment-16475164 ] Aaron Fabbri commented on HADOOP-15460: --- Interesting. Any thoughts on mkdirs()? > S3A FS to add

[jira] [Commented] (HADOOP-15356) Make HTTP timeout configurable in ADLS Connector

2018-05-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469413#comment-16469413 ] Aaron Fabbri commented on HADOOP-15356: --- Thanks for adding the extra validation of the timeout

[jira] [Commented] (HADOOP-15356) Make HTTP timeout configurable in ADLS Connector

2018-05-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468245#comment-16468245 ] Aaron Fabbri commented on HADOOP-15356: --- Patch looks good to me, +1. One question, what if

[jira] [Updated] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-05-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15420: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Updated] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13649: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Comment Edited] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466798#comment-16466798 ] Aaron Fabbri edited comment on HADOOP-13649 at 5/8/18 3:07 AM: --- Nice work

[jira] [Updated] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13649: -- Attachment: HADOOP-13649.003.patch > s3guard: implement time-based (TTL) expiry for

[jira] [Commented] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466798#comment-16466798 ] Aaron Fabbri commented on HADOOP-13649: --- Nice work [~gabor.bota], this looks good. I hope you

[jira] [Comment Edited] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466764#comment-16466764 ] Aaron Fabbri edited comment on HADOOP-15420 at 5/8/18 2:44 AM: --- Thank you

[jira] [Commented] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-05-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16466764#comment-16466764 ] Aaron Fabbri commented on HADOOP-15420: --- Thank you for working on this issue [~gabor.bota]. Good

[jira] [Updated] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15444: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Updated] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15444: -- Attachment: HADOOP-15444.001.patch > ITestS3GuardToolDynamo should only run with -Ddynamo >

[jira] [Updated] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15444: -- Status: Patch Available (was: Open) attached patch.  Tested (us-west-2) with -Ds3guard >

[jira] [Assigned] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-15444: - Assignee: Aaron Fabbri > ITestS3GuardToolDynamo should only run with -Ddynamo >

[jira] [Created] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-02 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-15444: - Summary: ITestS3GuardToolDynamo should only run with -Ddynamo Key: HADOOP-15444 URL: https://issues.apache.org/jira/browse/HADOOP-15444 Project: Hadoop Common

[jira] [Commented] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461946#comment-16461946 ] Aaron Fabbri commented on HADOOP-14927: --- Looking at this again.. and noticing

[jira] [Commented] (HADOOP-15356) Make HTTP timeout configurable in ADLS Connector

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461714#comment-16461714 ] Aaron Fabbri commented on HADOOP-15356: --- Thanks for the contributions guys. For the v2 patch,

[jira] [Commented] (HADOOP-15434) Upgrade to ADLS SDK that exposes current timeout

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461709#comment-16461709 ] Aaron Fabbri commented on HADOOP-15434: --- The patch is trivial (one line bump to pom version).

<    1   2   3   4   5   6   7   8   9   10   >