[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-11-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16235099#comment-16235099 ] Aaron Fabbri commented on HADOOP-14971: --- Think I found the cause of the job not finishing for that

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-11-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234906#comment-16234906 ] Aaron Fabbri commented on HADOOP-14971: --- Just noticed {{AbstractCommitTest#waitForConsistency()}}.

[jira] [Comment Edited] (HADOOP-14971) Merge S3A committers into trunk

2017-11-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234821#comment-16234821 ] Aaron Fabbri edited comment on HADOOP-14971 at 11/1/17 10:02 PM: -

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-11-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234821#comment-16234821 ] Aaron Fabbri commented on HADOOP-14971: --- Brought that {{testMRJob}} back to my laptop for

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-11-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234740#comment-16234740 ] Aaron Fabbri commented on HADOOP-14971: --- Integration tests passed in us-west-2 except this one,

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-11-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234583#comment-16234583 ] Aaron Fabbri commented on HADOOP-14971: --- Weird. I had a bunch of whitespace issues when I applied

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-10-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233627#comment-16233627 ] Aaron Fabbri commented on HADOOP-14971: --- looking at latest patch (via github actually, so I can

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-10-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227420#comment-16227420 ] Aaron Fabbri commented on HADOOP-14971: --- Thanks [~ste...@apache.org]. I will build this latest

[jira] [Commented] (HADOOP-14971) Merge S3A committers into trunk

2017-10-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225666#comment-16225666 ] Aaron Fabbri commented on HADOOP-14971: --- Thanks for the notes [~ste...@apache.org]. Looks like

[jira] [Commented] (HADOOP-14973) [s3a] Log StorageStatistics

2017-10-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220862#comment-16220862 ] Aaron Fabbri commented on HADOOP-14973: --- I've seen the toString() stuff, but it doesn't give us a

[jira] [Comment Edited] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211899#comment-16211899 ] Aaron Fabbri edited comment on HADOOP-13786 at 10/20/17 1:14 AM: - I was

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211899#comment-16211899 ] Aaron Fabbri commented on HADOOP-13786: --- I was unable to reproduce the du failure again FYI. The

[jira] [Commented] (HADOOP-14098) AliyunOSS: improve the performance of object metadata operation

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211442#comment-16211442 ] Aaron Fabbri commented on HADOOP-14098: --- I made efforts to keep the MetadataStore part of S3Guard

[jira] [Commented] (HADOOP-14965) s3a input stream "normal" fadvise mode to be adaptive

2017-10-19 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16211405#comment-16211405 ] Aaron Fabbri commented on HADOOP-14965: --- Seems like a good idea to make it adaptive. Assuming the

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-18 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210344#comment-16210344 ] Aaron Fabbri commented on HADOOP-13786: --- In terms of testing, one issue I'm hitting on OS X is the

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208754#comment-16208754 ] Aaron Fabbri commented on HADOOP-13786: --- Feeling another wave coming on.. Documentation is

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-17 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208734#comment-16208734 ] Aaron Fabbri commented on HADOOP-13786: --- Going to start posting some comments on v39 megapatch. I

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207046#comment-16207046 ] Aaron Fabbri commented on HADOOP-13786: --- I'm still reviewing and testing this stuff. Looks

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202674#comment-16202674 ] Aaron Fabbri commented on HADOOP-13786: --- Wow great work. I will re-review and do more testing on

[jira] [Commented] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202612#comment-16202612 ] Aaron Fabbri commented on HADOOP-13974: --- Yes, [~ste...@apache.org] happy to focus on getting the

[jira] [Updated] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13974: -- Attachment: HADOOP-13974.003.patch v3 patch. - Adds "are you sure" prompt any time `-abort`

[jira] [Updated] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-12 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13974: -- Status: Patch Available (was: Open) > S3a CLI to support list/purge of pending multipart

[jira] [Updated] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13974: -- Attachment: HADOOP-13974.002.patch v2 patch: - Adds "-expect" option as suggested above by

[jira] [Commented] (HADOOP-13786) Add S3Guard committer for zero-rename commits to S3 endpoints

2017-10-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201197#comment-16201197 ] Aaron Fabbri commented on HADOOP-13786: --- Thanks [~ste...@apache.org]. I'll bug any Spark folks

[jira] [Comment Edited] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201032#comment-16201032 ] Aaron Fabbri edited comment on HADOOP-13974 at 10/11/17 9:51 PM: -

[jira] [Updated] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-11 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13974: -- Attachment: HADOOP-13974.001.patch Attaching v1 patch. Some caveats I'd like feedback on: -

[jira] [Commented] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195441#comment-16195441 ] Aaron Fabbri commented on HADOOP-13974: --- Thanks [~ste...@apache.org]. I was thinking something

[jira] [Created] (HADOOP-14936) S3Guard: remove "experimental" from documentation

2017-10-06 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-14936: - Summary: S3Guard: remove "experimental" from documentation Key: HADOOP-14936 URL: https://issues.apache.org/jira/browse/HADOOP-14936 Project: Hadoop Common

[jira] [Resolved] (HADOOP-14746) Cut S3AOutputStream

2017-10-04 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri resolved HADOOP-14746. --- Resolution: Duplicate Fix Version/s: 3.0.0-beta1 Resolving this.. it was covered by

[jira] [Commented] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190577#comment-16190577 ] Aaron Fabbri commented on HADOOP-13974: --- I'll try to throw something together this week. > S3a

[jira] [Assigned] (HADOOP-13974) S3a CLI to support list/purge of pending multipart commits

2017-10-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13974: - Assignee: Aaron Fabbri > S3a CLI to support list/purge of pending multipart commits >

[jira] [Commented] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2017-10-03 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190506#comment-16190506 ] Aaron Fabbri commented on HADOOP-14927: --- [~ste...@apache.org] looking at the code for

[jira] [Created] (HADOOP-14927) ITestS3GuardTool failures in testDestroyNoBucket()

2017-10-03 Thread Aaron Fabbri (JIRA)
Aaron Fabbri created HADOOP-14927: - Summary: ITestS3GuardTool failures in testDestroyNoBucket() Key: HADOOP-14927 URL: https://issues.apache.org/jira/browse/HADOOP-14927 Project: Hadoop Common

[jira] [Commented] (HADOOP-14918) remove the Local Dynamo DB test option

2017-10-02 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16189188#comment-16189188 ] Aaron Fabbri commented on HADOOP-14918: --- The local dynamo lib was useful during development I

[jira] [Commented] (HADOOP-14890) Move up to AWS SDK 1.11.199

2017-09-26 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181272#comment-16181272 ] Aaron Fabbri commented on HADOOP-14890: --- +1 on branch-2 patch, LGTM. > Move up to AWS SDK

[jira] [Commented] (HADOOP-14890) Move up to AWS SDK 1.11.199

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179998#comment-16179998 ] Aaron Fabbri commented on HADOOP-14890: --- Committed to trunk and branch-3.0. Scale, unit, and

[jira] [Updated] (HADOOP-14890) Move up to AWS SDK 1.11.199

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14890: -- Fix Version/s: (was: 2.8.0) 3.0.0 > Move up to AWS SDK 1.11.199 >

[jira] [Updated] (HADOOP-14890) Move up to AWS SDK 1.11.199

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14890: -- Release Note: (was: Committed to trunk and branch-3.0. Scale, unit, and integration tests

[jira] [Updated] (HADOOP-14890) Move up to AWS SDK 1.11.199

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14890: -- Resolution: Fixed Release Note: Committed to trunk and branch-3.0. Scale, unit, and

[jira] [Commented] (HADOOP-14890) Move up to AWS SDK 1.11.199

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179944#comment-16179944 ] Aaron Fabbri commented on HADOOP-14890: --- Testing this now (unit, integration, scale). If it looks

[jira] [Updated] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14220: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Enhance S3GuardTool with

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179924#comment-16179924 ] Aaron Fabbri commented on HADOOP-14220: --- Committed to trunk and branch-3.0. Thanks for the

[jira] [Updated] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14220: -- Attachment: HADOOP-14220-016.patch Took v15 patch and tested in us-west-2; all clear.

[jira] [Updated] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-25 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14220: -- Status: Patch Available (was: Open) > Enhance S3GuardTool with bucket-info and set-capacity

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-22 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177400#comment-16177400 ] Aaron Fabbri commented on HADOOP-14220: --- Getting around to this again [~ste...@apache.org].. Did

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168622#comment-16168622 ] Aaron Fabbri commented on HADOOP-14220: --- Testing v13 patch ran into three failures in the new

[jira] [Updated] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14220: -- Status: Patch Available (was: Open) > Enhance S3GuardTool with bucket-info and set-capacity

[jira] [Updated] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14220: -- Attachment: HADOOP-14220-013.patch Attaching v13 patch as mentioned above. > Enhance

[jira] [Updated] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14220: -- Status: Open (was: Patch Available) > Enhance S3GuardTool with bucket-info and set-capacity

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168547#comment-16168547 ] Aaron Fabbri commented on HADOOP-14220: --- Looking at v12 patch. {noformat} + + + + +

[jira] [Comment Edited] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168258#comment-16168258 ] Aaron Fabbri edited comment on HADOOP-14220 at 9/15/17 5:53 PM: Thanks

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168258#comment-16168258 ] Aaron Fabbri commented on HADOOP-14220: --- Thanks for an update. I will try to test and review this

[jira] [Commented] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167102#comment-16167102 ] Aaron Fabbri commented on HADOOP-14738: --- Committed to branch-3.0 as well. > Remove S3N and

[jira] [Commented] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167055#comment-16167055 ] Aaron Fabbri commented on HADOOP-14738: --- Thanks [~andrew.wang]. I'm re-running tests and will

[jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure

2017-09-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167002#comment-16167002 ] Aaron Fabbri commented on HADOOP-14553: --- Thanks [~ste...@apache.org]. I re-ran all the

[jira] [Updated] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14738: -- Resolution: Fixed Fix Version/s: 3.0.0-beta1 Status: Resolved (was: Patch

[jira] [Commented] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-14 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16166597#comment-16166597 ] Aaron Fabbri commented on HADOOP-14738: --- Yep I'd relaunched the jenkins run and the minicluster

[jira] [Comment Edited] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165668#comment-16165668 ] Aaron Fabbri edited comment on HADOOP-14738 at 9/14/17 3:38 AM: javac

[jira] [Commented] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165668#comment-16165668 ] Aaron Fabbri commented on HADOOP-14738: --- javac warning is existing FAST_UPLOAD constant

[jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure

2017-09-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165663#comment-16165663 ] Aaron Fabbri commented on HADOOP-14553: --- Just noticed your comment above which answers one of my

[jira] [Commented] (HADOOP-14553) Add (parallelized) integration tests to hadoop-azure

2017-09-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165549#comment-16165549 ] Aaron Fabbri commented on HADOOP-14553: --- Coming back to test and review this again

[jira] [Updated] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14738: -- Status: Patch Available (was: Open) > Remove S3N and obsolete bits of S3A; rework docs >

[jira] [Updated] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-13 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14738: -- Attachment: HADOOP-14738-006.patch Attaching v6 patch.. Just the v5 patch rebased (and

[jira] [Comment Edited] (HADOOP-14851) LambdaTestUtils.eventually() doesn't spin on Assertion failures

2017-09-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159703#comment-16159703 ] Aaron Fabbri edited comment on HADOOP-14851 at 9/9/17 3:29 AM: --- Committed

[jira] [Updated] (HADOOP-14851) LambdaTestUtils.eventually() doesn't spin on Assertion failures

2017-09-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14851: -- Resolution: Fixed Fix Version/s: 3.0.0-beta1 Status: Resolved (was: Patch

[jira] [Updated] (HADOOP-14851) LambdaTestUtils.eventually() doesn't spin on Assertion failures

2017-09-08 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-14851: -- Attachment: HADOOP-14851-003.patch v3 patch: Added a period at the end of javadoc sentence to

[jira] [Commented] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157950#comment-16157950 ] Aaron Fabbri commented on HADOOP-14738: --- First off, removing a bunch of older code brings me joy.

[jira] [Commented] (HADOOP-14738) Remove S3N and obsolete bits of S3A; rework docs

2017-09-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157846#comment-16157846 ] Aaron Fabbri commented on HADOOP-14738: --- Looking at the latest patch now.. > Remove S3N and

[jira] [Updated] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13421: -- Attachment: HADOOP-13421.004.patch v4 patch. Same as v3, but adds new config option to

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-07 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157332#comment-16157332 ] Aaron Fabbri commented on HADOOP-14220: --- Good stuff, thanks [~ste...@apache.org]. +1 on v6 patch.

[jira] [Assigned] (HADOOP-14738) Deprecate S3N in hadoop 3.0/2,9, target removal in Hadoop 3.1; rework docs

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-14738: - Assignee: Aaron Fabbri (was: Steve Loughran) > Deprecate S3N in hadoop 3.0/2,9,

[jira] [Updated] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13421: -- Attachment: HADOOP-13421.003.patch Attaching v3 patch. Changes from v2: - Use an integer

[jira] [Comment Edited] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156001#comment-16156001 ] Aaron Fabbri edited comment on HADOOP-14220 at 9/6/17 8:38 PM: --- Great

[jira] [Comment Edited] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156001#comment-16156001 ] Aaron Fabbri edited comment on HADOOP-14220 at 9/6/17 8:37 PM: --- Great

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156001#comment-16156001 ] Aaron Fabbri commented on HADOOP-14220: --- Great stuff [~ste...@apache.org]. Thanks for doing this.

[jira] [Commented] (HADOOP-14220) Enhance S3GuardTool with bucket-info and set-capacity commands, tests

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155895#comment-16155895 ] Aaron Fabbri commented on HADOOP-14220: --- Thanks for the mention [~ste...@apache.org] I will review

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155746#comment-16155746 ] Aaron Fabbri commented on HADOOP-13421: --- {quote} better fs.s3a.list.version="1" line us up for a

[jira] [Comment Edited] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154691#comment-16154691 ] Aaron Fabbri edited comment on HADOOP-13421 at 9/6/17 1:51 AM: --- Attaching

[jira] [Updated] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13421: -- Attachment: HADOOP-13421.002.patch Attaching v2 patch. I ended up using a separate class

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154687#comment-16154687 ] Aaron Fabbri commented on HADOOP-13421: --- [~ste...@apache.org] thanks, I was thinking the same

[jira] [Commented] (HADOOP-14738) Deprecate S3N in hadoop 3.0/2,9, target removal in Hadoop 3.1; rework docs

2017-09-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154294#comment-16154294 ] Aaron Fabbri commented on HADOOP-14738: --- This looks great. +1 (non-binding) The docs are really

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151302#comment-16151302 ] Aaron Fabbri commented on HADOOP-13421: --- Whoever added the forced list response paging to

[jira] [Comment Edited] (HADOOP-14821) Executing the command 'hdfs -Dhadoop.security.credential.provider.path=file1.jceks,file2.jceks' fails if permission is denied to some files

2017-09-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150952#comment-16150952 ] Aaron Fabbri edited comment on HADOOP-14821 at 9/1/17 6:12 PM: --- Edit: I

[jira] [Commented] (HADOOP-14821) Executing the command 'hdfs -Dhadoop.security.credential.provider.path=file1.jceks,file2.jceks' fails if permission is denied to some files

2017-09-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150952#comment-16150952 ] Aaron Fabbri commented on HADOOP-14821: --- It looks like a permissions issue on your .jceks file: it

[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149826#comment-16149826 ] Aaron Fabbri commented on HADOOP-13998: --- +1 on v5 patch (non-binding, see upstream VOTE). Didn't

[jira] [Updated] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13421: -- Status: Patch Available (was: Open) > Switch to v2 of the S3 List Objects API in S3A >

[jira] [Updated] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri updated HADOOP-13421: -- Attachment: HADOOP-13421-HADOOP-13345.001.patch Attaching v1 patch, which is based on the

[jira] [Comment Edited] (HADOOP-13998) Merge initial S3guard release into trunk

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149638#comment-16149638 ] Aaron Fabbri edited comment on HADOOP-13998 at 8/31/17 9:32 PM: Applying

[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149638#comment-16149638 ] Aaron Fabbri commented on HADOOP-13998: --- Applying and testing patch 005 now. For Yetus results

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149268#comment-16149268 ] Aaron Fabbri commented on HADOOP-13421: --- Agreed. > Switch to v2 of the S3 List Objects API in S3A

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148373#comment-16148373 ] Aaron Fabbri commented on HADOOP-13421: --- Got this working, with v1 compatibility config knob (off

[jira] [Assigned] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Fabbri reassigned HADOOP-13421: - Assignee: Aaron Fabbri > Switch to v2 of the S3 List Objects API in S3A >

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146398#comment-16146398 ] Aaron Fabbri commented on HADOOP-13421: --- I'd be happy to work on this.. If anyone else has a patch

[jira] [Commented] (HADOOP-14777) S3Guard premerge changes: java 7 build & test tuning

2017-08-21 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135906#comment-16135906 ] Aaron Fabbri commented on HADOOP-14777: --- qq [~ste...@apache.org] did you do the lambda -> callable

[jira] [Commented] (HADOOP-14787) AliyunOSS: Implement the `createNonRecursive` operator

2017-08-21 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135605#comment-16135605 ] Aaron Fabbri commented on HADOOP-14787: --- Looks good to me (+1, non binding). I did not test this,

[jira] [Commented] (HADOOP-14777) S3Guard premerge changes: java 7 build & test tuning

2017-08-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129554#comment-16129554 ] Aaron Fabbri commented on HADOOP-14777: --- +1 on v1 patch. Tested in us-west-2 in three modes, no

[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk

2017-08-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129181#comment-16129181 ] Aaron Fabbri commented on HADOOP-13998: --- FYI -Ds3guard (local) tests all passed once I removed

[jira] [Commented] (HADOOP-14777) S3Guard premerge changes: java 7 build & test tuning

2017-08-16 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129101#comment-16129101 ] Aaron Fabbri commented on HADOOP-14777: --- {quote} One more thought: should we change the policy at

[jira] [Commented] (HADOOP-13998) Merge initial S3guard release into trunk

2017-08-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128169#comment-16128169 ] Aaron Fabbri commented on HADOOP-13998: --- v4 applied cleanly. S3A tests w/o S3guard all passed.

[jira] [Commented] (HADOOP-14738) Deprecate S3N in hadoop 3.0/2,9, target removal in Hadoop 3.1

2017-08-15 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127599#comment-16127599 ] Aaron Fabbri commented on HADOOP-14738: --- I'm happy to put up a patch for this once we have

<    1   2   3   4   5   6   7   8   9   10   >