[jira] [Commented] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954540#comment-15954540 ] Mingliang Liu commented on HADOOP-13926: Tested against us-west-1: {code} $ mvn

[jira] [Updated] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Attachment: HADOOP-13926-HADOOP-13345.005.patch Thanks [~rajesh.balamohan]. The v5 patch

[jira] [Commented] (HADOOP-14226) S3Guard: DynamoDBMetadataStore::move() should populate ancestor directories of destination paths

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954470#comment-15954470 ] Mingliang Liu commented on HADOOP-14226: The v2 patch looks good to me. It's much clearer.

[jira] [Updated] (HADOOP-14255) S3A to delete unnecessary fake directory objects in mkdirs()

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14255: --- Attachment: HADOOP-14255.001.patch [~ste...@apache.org], thanks for reviewing. {quote}

[jira] [Assigned] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-13926: -- Assignee: Mingliang Liu (was: Steve Loughran) > S3Guard:

[jira] [Updated] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Attachment: HADOOP-13926-HADOOP-13345.004.patch [~ste...@apache.org] I fixed the two minor

[jira] [Updated] (HADOOP-14226) S3Guard: DynamoDBMetadataStore::move() should populate ancestor directories of destination paths

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Summary: S3Guard: DynamoDBMetadataStore::move() should populate ancestor directories of

[jira] [Updated] (HADOOP-14268) Fix markdown itemization in hadoop-aws documents

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14268: --- Component/s: fs/s3 > Fix markdown itemization in hadoop-aws documents >

[jira] [Updated] (HADOOP-14268) Fix markdown itemization in hadoop-aws documents

2017-04-03 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14268: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Updated] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Attachment: HADOOP-13926-HADOOP-13345.003.patch > S3Guard:

[jira] [Updated] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-04-01 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Attachment: HADOOP-13926-HADOOP-13345.002.patch v2 patch for review: it implements what

[jira] [Updated] (HADOOP-14267) Make DistCpOptions class immutable

2017-03-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14267: --- Resolution: Fixed Hadoop Flags: Incompatible change,Reviewed (was: Incompatible

[jira] [Moved] (HADOOP-14267) Make DistCpOptions class immutable

2017-03-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu moved HDFS-10533 to HADOOP-14267: --- Target Version/s: 3.0.0-alpha3 (was: 3.0.0-alpha3)

[jira] [Comment Edited] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-03-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951839#comment-15951839 ] Mingliang Liu edited comment on HADOOP-13926 at 4/1/17 12:56 AM: -

[jira] [Commented] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-03-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951839#comment-15951839 ] Mingliang Liu commented on HADOOP-13926: Thanks, [~fabbri]! {quote} For unblocking merge to

[jira] [Created] (HADOOP-14266) S3Guard: S3AFileSystem::listFiles() to employ MetadataStore

2017-03-31 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14266: -- Summary: S3Guard: S3AFileSystem::listFiles() to employ MetadataStore Key: HADOOP-14266 URL: https://issues.apache.org/jira/browse/HADOOP-14266 Project: Hadoop

[jira] [Updated] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore

2017-03-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Summary: S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore (was:

[jira] [Commented] (HADOOP-14262) rpcTimeOut is not set up correctly in Client thus client doesn't time out

2017-03-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951460#comment-15951460 ] Mingliang Liu commented on HADOOP-14262: Client.getTimeOut() is calling Client.getRpcTimeout(),

[jira] [Commented] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15950305#comment-15950305 ] Mingliang Liu commented on HADOOP-14236: Thank you very much for your insightful comments and

[jira] [Updated] (HADOOP-13926) S3Guard: Improve listLocatedStatus and listFiles

2017-03-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Target Version/s: HADOOP-13345 (was: 2.9.0) Status: Patch Available (was:

[jira] [Updated] (HADOOP-13926) S3Guard: Improve listLocatedStatus and listFiles

2017-03-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Attachment: HADOOP-13926-HADOOP-13345.001.patch I rebase and updated [~rajesh.balamohan]'s

[jira] [Updated] (HADOOP-14255) S3A to delete unnecessary fake directory objects in mkdirs()

2017-03-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14255: --- Attachment: HADOOP-14255.000.patch Tested against us-west-1 region. {code} $ mvn

[jira] [Updated] (HADOOP-14255) S3A to delete unnecessary fake directory objects in mkdirs()

2017-03-30 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14255: --- Status: Patch Available (was: Open) > S3A to delete unnecessary fake directory objects in

[jira] [Commented] (HADOOP-14226) S3Guard: DynamoDBMetadata::move() should populate ancestor directories of destination paths

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948347#comment-15948347 ] Mingliang Liu commented on HADOOP-14226: Tested against us-west-1: {code} $ mvn

[jira] [Updated] (HADOOP-14255) S3A to delete unnecessary fake directory objects in mkdirs()

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14255: --- Description: In S3AFileSystem, as an optimization, we delete unnecessary fake directory

[jira] [Comment Edited] (HADOOP-14226) S3Guard: DynamoDBMetadata::move() should populate ancestor directories of destination paths

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948220#comment-15948220 ] Mingliang Liu edited comment on HADOOP-14226 at 3/30/17 1:24 AM: - The v1

[jira] [Updated] (HADOOP-14226) S3Guard: DynamoDBMetadata::move() should populate ancestor directories of destination paths

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Attachment: HADOOP-14226-HADOOP-13345.001.patch The v1 patch is to follow [~fabbri]'s

[jira] [Updated] (HADOOP-14226) S3Guard: DynamoDBMetadata::move() should populate ancestor directories of destination paths

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Description: UPDATE: Instead of changing the test, in this JIRA we make sure for each path

[jira] [Updated] (HADOOP-14226) S3Guard: DynamoDBMetadata::move() should populate ancestor directories of destination paths

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Priority: Major (was: Minor) > S3Guard: DynamoDBMetadata::move() should populate ancestor

[jira] [Updated] (HADOOP-14226) S3Guard: DynamoDBMetadata::move() should populate ancestor directories of destination paths

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Summary: S3Guard: DynamoDBMetadata::move() should populate ancestor directories of

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Comment Edited] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948009#comment-15948009 ] Mingliang Liu edited comment on HADOOP-14236 at 3/29/17 10:43 PM: --

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Attachment: HADOOP-14236-HADOOP-13345.004.patch > S3Guard: S3AFileSystem::rename() should

[jira] [Comment Edited] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948009#comment-15948009 ] Mingliang Liu edited comment on HADOOP-14236 at 3/29/17 10:32 PM: --

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-29 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Attachment: HADOOP-14236-HADOOP-13345.003.patch [~fabbri], thanks for your comment and

[jira] [Created] (HADOOP-14255) S3A to delete unnecessary fake directory objects in mkdirs()

2017-03-29 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14255: -- Summary: S3A to delete unnecessary fake directory objects in mkdirs() Key: HADOOP-14255 URL: https://issues.apache.org/jira/browse/HADOOP-14255 Project: Hadoop

[jira] [Commented] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946145#comment-15946145 ] Mingliang Liu commented on HADOOP-14226: [~fabbri] your point is very valid. I'll post a new

[jira] [Commented] (HADOOP-14084) Shell.joinThread swallows InterruptedException

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946116#comment-15946116 ] Mingliang Liu commented on HADOOP-14084: I think if the {{joinThread()}} is used only in

[jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider in trunk; Deprecate in branch-2

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14248: --- Summary: Retire SharedInstanceProfileCredentialsProvider in trunk; Deprecate in branch-2

[jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14248: --- Attachment: HADOOP-14248.001.patch > Retire SharedInstanceProfileCredentialsProvider after

[jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14248: --- Attachment: HADOOP-14248-branch-2.001.patch Thanks [~ste...@apache.org] for your comments.

[jira] [Updated] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14227: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HADOOP-13345

[jira] [Commented] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945804#comment-15945804 ] Mingliang Liu commented on HADOOP-14226: [~fabbri] do you find time to review? Thanks, >

[jira] [Commented] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-28 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945800#comment-15945800 ] Mingliang Liu commented on HADOOP-14247: So [~ste...@apache.org] are you +1 on the patch? >

[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944435#comment-15944435 ] Mingliang Liu commented on HADOOP-14248: Tested against us-west-1 region on laptop (using secret

[jira] [Commented] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944356#comment-15944356 ] Mingliang Liu commented on HADOOP-14247: Tested S3A in us-west-1 region. Tested

[jira] [Updated] (HADOOP-14237) S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14237: --- Issue Type: Sub-task (was: Bug) Parent: HADOOP-13204 > S3A Support Shared Instance

[jira] [Updated] (HADOOP-14237) S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14237: --- Target Version/s: 2.8.1, 3.0.0-alpha3 Status: Patch Available (was: Open) >

[jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14248: --- Hadoop Flags: Incompatible change Release Note: SharedInstanceProfileCredentialsProvider

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Description: After running integration test {{ITestS3AFileSystemContract}}, I found the

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Description: After running integration test {{ITestS3AFileSystemContract}}, I found the

[jira] [Commented] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944306#comment-15944306 ] Mingliang Liu commented on HADOOP-14248: [~ste...@apache.org] and [~cnauroth], can you have a

[jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14248: --- Status: Patch Available (was: Open) > Retire SharedInstanceProfileCredentialsProvider

[jira] [Updated] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14248: --- Attachment: HADOOP-14248.000.patch > Retire SharedInstanceProfileCredentialsProvider after

[jira] [Created] (HADOOP-14248) Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade

2017-03-27 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14248: -- Summary: Retire SharedInstanceProfileCredentialsProvider after AWS SDK upgrade Key: HADOOP-14248 URL: https://issues.apache.org/jira/browse/HADOOP-14248 Project:

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Component/s: test > FileContextMainOperationsBaseTest should clean up test root path >

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Description: {{fileContextTestHelper}} will create random test root directory for each test

[jira] [Commented] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944245#comment-15944245 ] Mingliang Liu commented on HADOOP-14247: [~ste...@apache.org] can you have a quick look? >

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Status: Patch Available (was: Open) > FileContextMainOperationsBaseTest should clean up

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Attachment: HADOOP-14247.000.patch > FileContextMainOperationsBaseTest should clean up test

[jira] [Created] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14247: -- Summary: FileContextMainOperationsBaseTest should clean up test root path Key: HADOOP-14247 URL: https://issues.apache.org/jira/browse/HADOOP-14247 Project:

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Priority: Minor (was: Major) > FileContextMainOperationsBaseTest should clean up test root

[jira] [Updated] (HADOOP-14247) FileContextMainOperationsBaseTest should clean up test root path

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14247: --- Target Version/s: 2.8.1, 3.0.0-alpha3 (was: 2.8.1) > FileContextMainOperationsBaseTest

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Attachment: HADOOP-14236-HADOOP-13345.002.patch > S3Guard: S3AFileSystem::rename() should

[jira] [Updated] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14227: --- Issue Type: Sub-task (was: Bug) Parent: HADOOP-13345 > S3Guard:

[jira] [Commented] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944143#comment-15944143 ] Mingliang Liu commented on HADOOP-14236: Tested against us-west-1. {code} $ mvn

[jira] [Assigned] (HADOOP-14237) S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-14237: -- Assignee: Kazuyuki Tanimura > S3A Support Shared Instance Profile Credentials Across

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Attachment: HADOOP-14236-HADOOP-13345.001.patch > S3Guard: S3AFileSystem::rename() should

[jira] [Updated] (HADOOP-13926) S3Guard: Improve listLocatedStatus and listFiles

2017-03-27 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13926: --- Priority: Major (was: Minor) > S3Guard: Improve listLocatedStatus and listFiles >

[jira] [Commented] (HADOOP-14237) S3A Support Shared Instance Profile Credentials Across All Instances

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941514#comment-15941514 ] Mingliang Liu commented on HADOOP-14237: In latest aws-java-sdk, it's a singleton instance so we

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Status: Patch Available (was: Open) > S3Guard: S3AFileSystem::rename() should move

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Attachment: HADOOP-14236-HADOOP-13345.000.patch V0 patch is to present what I'm thinking.

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Description: After running integration test {{ITestS3AFileSystemContract}}, I found the

[jira] [Updated] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14236: --- Priority: Critical (was: Major) > S3Guard: S3AFileSystem::rename() should move non-listed

[jira] [Created] (HADOOP-14236) S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store

2017-03-24 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14236: -- Summary: S3Guard: S3AFileSystem::rename() should move non-listed sub-directory entries in metadata store Key: HADOOP-14236 URL:

[jira] [Commented] (HADOOP-14230) TestAdlFileSystemContractLive fails to clean up

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940906#comment-15940906 ] Mingliang Liu commented on HADOOP-14230: +1 For {{if

[jira] [Updated] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14227: --- Priority: Minor (was: Major) Setting this JIRA priority as Minor. > S3Guard:

[jira] [Updated] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Priority: Minor (was: Major) Setting this JIRA priority as Minor. > S3Guard:

[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Release Note: For running integration tests, the command "mvn verify -Dscale -Ds3guard

[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HADOOP-13345

[jira] [Updated] (HADOOP-13966) Add ability to start DDB local server in every test

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-13966: --- Release Note: For running integration tests, the command "mvn verify -Dscale -Ds3guard

[jira] [Commented] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940805#comment-15940805 ] Mingliang Liu commented on HADOOP-14227: Sorry I forgot to mention. Tested against us-west-1.

[jira] [Comment Edited] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-24 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939462#comment-15939462 ] Mingliang Liu edited comment on HADOOP-14226 at 3/24/17 5:45 PM: -

[jira] [Updated] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14227: --- Attachment: HADOOP-14227-HADOOP-13345.000.patch > S3Guard: ITestS3AConcurrentOps is not

[jira] [Updated] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14227: --- Status: Patch Available (was: Open) > S3Guard: ITestS3AConcurrentOps is not cleaning up

[jira] [Created] (HADOOP-14227) S3Guard: ITestS3AConcurrentOps is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14227: -- Summary: S3Guard: ITestS3AConcurrentOps is not cleaning up test data Key: HADOOP-14227 URL: https://issues.apache.org/jira/browse/HADOOP-14227 Project: Hadoop

[jira] [Commented] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939462#comment-15939462 ] Mingliang Liu commented on HADOOP-14226: I manually checked the test and the data was cleaned up

[jira] [Updated] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Status: Patch Available (was: Open) Ping [~ste...@apache.org] and [~rajesh.balamohan].

[jira] [Updated] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14226: --- Attachment: HADOOP-14226-HADOOP-13345.000.patch > S3Guard: ITestDynamoDBMetadataStoreScale

[jira] [Created] (HADOOP-14226) S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data

2017-03-23 Thread Mingliang Liu (JIRA)
Mingliang Liu created HADOOP-14226: -- Summary: S3Guard: ITestDynamoDBMetadataStoreScale is not cleaning up test data Key: HADOOP-14226 URL: https://issues.apache.org/jira/browse/HADOOP-14226 Project:

[jira] [Commented] (HADOOP-13377) AliyunOSS: improvements for stabilization

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939238#comment-15939238 ] Mingliang Liu commented on HADOOP-13377: I moved two recent changes of fs/oss module as subtasks

[jira] [Updated] (HADOOP-14194) Aliyun OSS should not use empty endpoint as default

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14194: --- Issue Type: Sub-task (was: Bug) Parent: HADOOP-13377 > Aliyun OSS should not use

[jira] [Updated] (HADOOP-14192) Aliyun OSS FileSystem contract test should implement getTestBaseDir()

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14192: --- Issue Type: Sub-task (was: Bug) Parent: HADOOP-13377 > Aliyun OSS FileSystem

[jira] [Updated] (HADOOP-14214) DomainSocketWatcher::add()/delete() should not self interrupt while looping await()

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14214: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Updated] (HADOOP-14135) Remove URI parameter in AWSCredentialProvider constructors

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14135: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3

[jira] [Updated] (HADOOP-14135) Remove URI parameter in AWSCredentialProvider constructors

2017-03-23 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14135: --- Target Version/s: 2.8.1, 3.0.0-alpha3 (was: 2.9.0, 3.0.0-alpha3) > Remove URI parameter in

[jira] [Updated] (HADOOP-14214) DomainSocketWatcher::add()/delete() should not self interrupt while looping await()

2017-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14214: --- Attachment: HADOOP-14214.000.patch > DomainSocketWatcher::add()/delete() should not self

[jira] [Updated] (HADOOP-14214) DomainSocketWatcher::add()/delete() should not self interrupt while looping await()

2017-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14214: --- Status: Patch Available (was: Open) > DomainSocketWatcher::add()/delete() should not self

[jira] [Updated] (HADOOP-14214) DomainSocketWatcher::add()/delete() should not self interrupt while looping await()

2017-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HADOOP-14214: --- Component/s: hdfs-client > DomainSocketWatcher::add()/delete() should not self interrupt

[jira] [Commented] (HADOOP-13966) Add ability to start DDB local server in every test

2017-03-22 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937461#comment-15937461 ] Mingliang Liu commented on HADOOP-13966: {code} $ mvn verify

<    2   3   4   5   6   7   8   9   10   11   >