[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480961#comment-16480961 ] Aaron Fabbri commented on HADOOP-15480: --- I also reproduced this yesterday.  Apologies for letting

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478404#comment-16478404 ] Aaron Fabbri commented on HADOOP-15430: --- Great work here. Test code is very nice. {quote}+ // here

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478062#comment-16478062 ] Aaron Fabbri commented on HADOOP-14946: --- [~gabor.bota] do you mind looking at this one? If not

[jira] [Assigned] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14946: - Assignee: Gabor Bota > S3Guard testPruneCommandCLI can fail >

[jira] [Commented] (HADOOP-15469) S3A directory committer commit job fails if _temporary directory created under dest

2018-05-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478057#comment-16478057 ] Aaron Fabbri commented on HADOOP-15469: --- Interesting. The argument for this seems to be (1) this

[jira] [Updated] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15525: -- Description: Scenario: customer wants to only give a Hadoop cluster access to a subtree of

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506673#comment-16506673 ] Aaron Fabbri commented on HADOOP-15525: --- Thanks [~poeppt]. We do have assume role support in S3A

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506712#comment-16506712 ] Aaron Fabbri commented on HADOOP-15525: --- The existing docs

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506723#comment-16506723 ] Aaron Fabbri commented on HADOOP-15525: --- Actually, refreshing my memory here,

[jira] [Updated] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15525: -- Description: Scenario: customer wants to only give a Hadoop cluster access to a subtree of

[jira] [Commented] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506574#comment-16506574 ] Aaron Fabbri commented on HADOOP-15525: --- Assigning to me for now. I'd like to write a doc here to

[jira] [Assigned] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-15525: - Assignee: Aaron Fabbri > s3a: clarify / improve support for mixed ACL buckets >

[jira] [Commented] (HADOOP-11697) Use larger value for fs.s3a.connection.timeout.

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506552#comment-16506552 ] Aaron Fabbri commented on HADOOP-11697: --- I think we can close this, since HADOOP-12346 bumped

[jira] [Resolved] (HADOOP-11697) Use larger value for fs.s3a.connection.timeout.

2018-06-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-11697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri resolved HADOOP-11697. --- Resolution: Duplicate Fix Version/s: 3.0.0 > Use larger value for

[jira] [Created] (HADOOP-15525) s3a: clarify / improve support for mixed ACL buckets

2018-06-08 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-15525: - Summary: s3a: clarify / improve support for mixed ACL buckets Key: HADOOP-15525 URL: https://issues.apache.org/jira/browse/HADOOP-15525 Project: Hadoop Common

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Attachment: HADOOP-14946.003.patch > S3Guard testPruneCommandCLI can fail >

[jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494099#comment-16494099 ] Aaron Fabbri commented on HADOOP-14946: --- I'll commit the v3 patch once yetus is clean.  (same as

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-05-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16495944#comment-16495944 ] Aaron Fabbri commented on HADOOP-14918: --- Shout if you need any help testing this [~gabor.bota].

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.005.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: (was: HADOOP-15480.005.patch) > AbstractS3GuardToolTestBase.testDiffCommand

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.005.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Updated] (HADOOP-14946) S3Guard testPruneCommandCLI can fail

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14946: -- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thank

[jira] [Commented] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16494331#comment-16494331 ] Aaron Fabbri commented on HADOOP-15480: --- Thanks [~gabor.bota], and thank you for letting me make

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Attachment: HADOOP-15480.004.patch > AbstractS3GuardToolTestBase.testDiffCommand fails when

[jira] [Updated] (HADOOP-15480) AbstractS3GuardToolTestBase.testDiffCommand fails when using dynamo

2018-05-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15480: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-15507) Add MapReduce counters about EC bytes read

2018-06-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500916#comment-16500916 ] Aaron Fabbri commented on HADOOP-15507: --- +1 assuming you manually tested (i.e. your screenshot is

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-06-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503682#comment-16503682 ] Aaron Fabbri commented on HADOOP-15407: --- Thanks for the update. In terms of us maintaining this

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-06-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16503693#comment-16503693 ] Aaron Fabbri commented on HADOOP-15407: --- Thanks.. was still reading through latest patch. Just

[jira] [Commented] (HADOOP-15215) s3guard set-capacity command to fail on read/write of 0

2018-06-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525367#comment-16525367 ] Aaron Fabbri commented on HADOOP-15215: --- LGTM +1.  Good work here [~gabor.bota]. I think findbugs

[jira] [Updated] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

2018-06-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15563: -- Description: To keep costs down on DDB, autoscaling is a key feature: you set the max values

[jira] [Updated] (HADOOP-15563) s3guard init and set-capacity to support DDB autoscaling

2018-06-27 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15563: -- Environment: (was: To keep costs down on DDB, autoscaling is a key feature: you set the

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: (was: image-2018-07-02-11-08-41-302.png) > Remove HTrace support >

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: ss-trace-s3a.png > Remove HTrace support > - > >

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: (was: ss-trace-s3a.png) > Remove HTrace support > - > >

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: image-2018-07-02-11-08-41-302.png > Remove HTrace support >

[jira] [Commented] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530265#comment-16530265 ] Aaron Fabbri commented on HADOOP-15566: --- Nice work [~jojochuang].  I had fun hacking on this for

[jira] [Updated] (HADOOP-15566) Remove HTrace support

2018-07-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15566: -- Attachment: ss-trace-s3a.png > Remove HTrace support > - > >

[jira] [Commented] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461946#comment-16461946 ] Aaron Fabbri commented on HADOOP-14927: --- Looking at this again.. and noticing

[jira] [Assigned] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-15444: - Assignee: Aaron Fabbri > ITestS3GuardToolDynamo should only run with -Ddynamo >

[jira] [Created] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-02 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-15444: - Summary: ITestS3GuardToolDynamo should only run with -Ddynamo Key: HADOOP-15444 URL: https://issues.apache.org/jira/browse/HADOOP-15444 Project: Hadoop Common

[jira] [Updated] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15444: -- Status: Patch Available (was: Open) attached patch.  Tested (us-west-2) with -Ds3guard >

[jira] [Updated] (HADOOP-15444) ITestS3GuardToolDynamo should only run with -Ddynamo

2018-05-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15444: -- Attachment: HADOOP-15444.001.patch > ITestS3GuardToolDynamo should only run with -Ddynamo >

[jira] [Commented] (HADOOP-15434) Upgrade to ADLS SDK that exposes current timeout

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461709#comment-16461709 ] Aaron Fabbri commented on HADOOP-15434: --- The patch is trivial (one line bump to pom version).

[jira] [Commented] (HADOOP-15356) Make HTTP timeout configurable in ADLS Connector

2018-05-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461714#comment-16461714 ] Aaron Fabbri commented on HADOOP-15356: --- Thanks for the contributions guys. For the v2 patch,

[jira] [Commented] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-04-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453540#comment-16453540 ] Aaron Fabbri commented on HADOOP-13649: --- Hey thanks for the patch [~gabor.bota]. Should have more

[jira] [Commented] (HADOOP-13756) LocalMetadataStore#put(DirListingMetadata) should also put file metadata into fileHash.

2018-04-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455637#comment-16455637 ] Aaron Fabbri commented on HADOOP-13756: --- +1.  You know it is a good patch when it is a one line

[jira] [Created] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-04-26 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-15420: - Summary: s3guard ITestS3GuardToolLocal failures in diff tests Key: HADOOP-15420 URL: https://issues.apache.org/jira/browse/HADOOP-15420 Project: Hadoop Common

[jira] [Commented] (HADOOP-15416) s3guard diff assert failure

2018-04-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455698#comment-16455698 ] Aaron Fabbri commented on HADOOP-15416: --- It looks like the S3Guard Tool diff testing needs some

[jira] [Assigned] (HADOOP-14918) remove the Local Dynamo DB test option

2018-04-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14918: - Assignee: Gabor Bota (was: Steve Loughran) > remove the Local Dynamo DB test option >

[jira] [Updated] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-04-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15420: -- Issue Type: Sub-task (was: Improvement) Parent: HADOOP-15226 > s3guard

[jira] [Updated] (HADOOP-15420) s3guard ITestS3GuardToolLocal failures in diff tests

2018-04-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15420: -- Description: Noticed this when testing the patch for HADOOP-13756.   {code:java} [ERROR]

[jira] [Commented] (HADOOP-13649) s3guard: implement time-based (TTL) expiry for LocalMetadataStore

2018-04-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455742#comment-16455742 ] Aaron Fabbri commented on HADOOP-13649: --- A couple of comments on the patch.  Looks pretty good

[jira] [Updated] (HADOOP-15239) S3ABlockOutputStream.flush() be no-op when stream closed

2018-04-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15239: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-04-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458970#comment-16458970 ] Aaron Fabbri commented on HADOOP-15430: --- I guess we need a test case for the CLI mkdir, as a

[jira] [Comment Edited] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211899#comment-16211899 ] Aaron Fabbri edited comment on HADOOP-13786 at 10/20/17 1:14 AM: - I was

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211899#comment-16211899 ] Aaron Fabbri commented on HADOOP-13786: --- I was unable to reproduce the du failure again FYI. The

[jira] [Commented] (HADOOP-14965) s3a input stream "normal" fadvise mode to be adaptive

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211405#comment-16211405 ] Aaron Fabbri commented on HADOOP-14965: --- Seems like a good idea to make it adaptive. Assuming the

[jira] [Commented] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220862#comment-16220862 ] Aaron Fabbri commented on HADOOP-14973: --- I've seen the toString() stuff, but it doesn't give us a

[jira] [Updated] (HADOOP-15215) s3guard set-capacity command to fail on read/write of 0

2018-07-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15215: -- Resolution: Fixed Fix Version/s: 3.2.0 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-15006) Encrypt S3A data client-side with Hadoop libraries & Hadoop KMS

2018-01-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16314171#comment-16314171 ] Aaron Fabbri commented on HADOOP-15006: --- Thanks again for writing this up [~moist]--it is very

[jira] [Commented] (HADOOP-15141) Support IAM Assumed roles in S3A

2018-01-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317596#comment-16317596 ] Aaron Fabbri commented on HADOOP-15141: --- Thanks for the contribution. I just ran integration tests

[jira] [Commented] (HADOOP-15141) Support IAM Assumed roles in S3A

2018-01-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319502#comment-16319502 ] Aaron Fabbri commented on HADOOP-15141: --- Javac are deprecation warnings on AWS API. Checkstyle

[jira] [Commented] (HADOOP-15140) S3guard mistakes root URI without / as non-absolute path

2018-01-09 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16319489#comment-16319489 ] Aaron Fabbri commented on HADOOP-15140: --- I believe [~abrahamfine] was looking for noob JIRAs to

[jira] [Commented] (HADOOP-15079) ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch

2018-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321714#comment-16321714 ] Aaron Fabbri commented on HADOOP-15079: --- Integration tests all passed with S3Guard (local) and

[jira] [Commented] (HADOOP-15079) ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch

2018-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321527#comment-16321527 ] Aaron Fabbri commented on HADOOP-15079: --- Fragile bits of code.. thanks for digging in to this.

[jira] [Commented] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321643#comment-16321643 ] Aaron Fabbri commented on HADOOP-13761: --- Reminder comment here: We should also: - Update

[jira] [Commented] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2018-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16321710#comment-16321710 ] Aaron Fabbri commented on HADOOP-14927: --- Just hit this again so I will put up a patch soon...

[jira] [Assigned] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2018-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14927: - Assignee: Aaron Fabbri > ITestS3GuardTool failures in testDestroyNoBucket() >

[jira] [Updated] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2018-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14927: -- Issue Type: Sub-task (was: Bug) Parent: HADOOP-14825 > ITestS3GuardTool failures in

[jira] [Commented] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328003#comment-16328003 ] Aaron Fabbri commented on HADOOP-13761: --- [~ste...@apache.org] sure. > S3Guard: implement retries

[jira] [Commented] (HADOOP-12502) SetReplication OutOfMemoryError

2018-01-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332968#comment-16332968 ] Aaron Fabbri commented on HADOOP-12502: --- Thank you for the new patch [~vinayrpet] and for working

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-01-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338413#comment-16338413 ] Aaron Fabbri commented on HADOOP-14918: --- Patch looks good except we want to keep

[jira] [Commented] (HADOOP-15176) Enhance IAM assumed role support in S3A client

2018-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329310#comment-16329310 ] Aaron Fabbri commented on HADOOP-15176: --- {quote} Patch 001, still WiP {quote} Ok, I won't

[jira] [Assigned] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13761: - Assignee: Aaron Fabbri (was: Steve Loughran) > S3Guard: implement retries for DDB

[jira] [Commented] (HADOOP-15141) Support IAM Assumed roles in S3A

2018-01-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328290#comment-16328290 ] Aaron Fabbri commented on HADOOP-15141: --- v6 patch: just fixing checkstyle and whitespace.  Tested

[jira] [Updated] (HADOOP-15141) Support IAM Assumed roles in S3A

2018-01-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15141: -- Attachment: HADOOP-15141-006.patch > Support IAM Assumed roles in S3A >

[jira] [Updated] (HADOOP-15141) Support IAM Assumed roles in S3A

2018-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15141: -- Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-15141) Support IAM Assumed roles in S3A

2018-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328447#comment-16328447 ] Aaron Fabbri commented on HADOOP-15141: --- Test failures look unrelated (TestCopyFromLocal works

[jira] [Commented] (HADOOP-13974) S3Guard CLI to support list/purge of pending multipart commits

2018-01-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329685#comment-16329685 ] Aaron Fabbri commented on HADOOP-13974: --- Hey [~ste...@apache.org] just noticed some unrelated

[jira] [Commented] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338852#comment-16338852 ] Aaron Fabbri commented on HADOOP-13761: --- Wrote a test to check above hypothesis on batch size vs.

[jira] [Commented] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338609#comment-16338609 ] Aaron Fabbri commented on HADOOP-13761: --- {quote} Managed to break tests when working with a bucket

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2018-01-24 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338645#comment-16338645 ] Aaron Fabbri commented on HADOOP-14918: --- You should be able to just change it to an ITest and set

[jira] [Updated] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13761: -- Attachment: HADOOP-13761.001.patch > S3Guard: implement retries for DDB failures and

[jira] [Commented] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340137#comment-16340137 ] Aaron Fabbri commented on HADOOP-13761: --- Attached v1 RFC patch that just adds a test that attempts

[jira] [Commented] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341807#comment-16341807 ] Aaron Fabbri commented on HADOOP-13761: --- v2 patch is same as v1 (just a test which tries to

[jira] [Updated] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13761: -- Attachment: HADOOP-13761.002.patch > S3Guard: implement retries for DDB failures and

[jira] [Comment Edited] (HADOOP-13761) S3Guard: implement retries for DDB failures and throttling; translate exceptions

2018-01-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341807#comment-16341807 ] Aaron Fabbri edited comment on HADOOP-13761 at 1/26/18 11:52 PM: - v2

[jira] [Commented] (HADOOP-15079) ITestS3AFileOperationCost#testFakeDirectoryDeletion failing after OutputCommitter patch

2018-01-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16324829#comment-16324829 ] Aaron Fabbri commented on HADOOP-15079: --- I'm +1 on v2 patch assuming clean Yetus (i can restart

[jira] [Commented] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-01-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346069#comment-16346069 ] Aaron Fabbri commented on HADOOP-15040: --- Doing some testing with latest SDK 1.11.271 today.

[jira] [Work started] (HADOOP-15040) AWS SDK NPE bug spams logs w/ Yarn Log Aggregation

2018-01-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-15040 started by Aaron Fabbri. - > AWS SDK NPE bug spams logs w/ Yarn Log Aggregation >

[jira] [Updated] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-01-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15040: -- Summary: Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation (was: AWS SDK NPE bug

[jira] [Updated] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-02-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15040: -- Attachment: HADOOP-15040.001.patch > Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log

[jira] [Updated] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-02-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15040: -- Attachment: HADOOP-12502-07.patch > Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log

[jira] [Updated] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-02-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15040: -- Attachment: (was: HADOOP-12502-07.patch) > Upgrade AWS SDK: NPE bug spams logs w/ Yarn

[jira] [Updated] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-02-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-15040: -- Status: Patch Available (was: In Progress) Attaching patch to bump AWS SDK to latest

[jira] [Commented] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-02-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351133#comment-16351133 ] Aaron Fabbri commented on HADOOP-15040: --- Ping folks for easy review. [~sean_impala_9b93]

[jira] [Commented] (HADOOP-15040) Upgrade AWS SDK: NPE bug spams logs w/ Yarn Log Aggregation

2018-02-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351122#comment-16351122 ] Aaron Fabbri commented on HADOOP-15040: --- Tests w/o S3Guard look good.  I think we should get this

[jira] [Comment Edited] (HADOOP-15191) Add Private/Unstable BulkDelete operations to supporting object stores for DistCP

2018-02-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349840#comment-16349840 ] Aaron Fabbri edited comment on HADOOP-15191 at 2/2/18 6:17 AM: --- I like the

[jira] [Commented] (HADOOP-15191) Add Private/Unstable BulkDelete operations to supporting object stores for DistCP

2018-02-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349840#comment-16349840 ] Aaron Fabbri commented on HADOOP-15191: --- I like the idea of an optional batch delete interface,

[jira] [Resolved] (HADOOP-15207) Hadoop performance Issues

2018-02-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri resolved HADOOP-15207. --- Resolution: Invalid Can someone please ban this user? Similar SEO spam has been pasted on

<    5   6   7   8   9   10   11   12   13   >