[jira] [Created] (HADOOP-13972) Access multiple Azure Data Lake stores with different SPIs

2017-01-10 Thread John Zhuge (JIRA)
John Zhuge created HADOOP-13972: --- Summary: Access multiple Azure Data Lake stores with different SPIs Key: HADOOP-13972 URL: https://issues.apache.org/jira/browse/HADOOP-13972 Project: Hadoop Common

[jira] [Updated] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HADOOP-13650: --- Attachment: (was: HADOOP-13650-HADOOP-13345.007.patch) > S3Guard: Provide command line

[jira] [Updated] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HADOOP-13650: --- Attachment: HADOOP-13650-HADOOP-13345.007.patch Re-upload the rebased one. > S3Guard:

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817532#comment-15817532 ] Hadoop QA commented on HADOOP-13650: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Updated] (HADOOP-13956) Read ADLS credentials from Credential Provider

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13956: Release Note: The hadoop-azure-datalake file system now supports configuration of the Azure Data

[jira] [Updated] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-10 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HADOOP-13650: --- Attachment: HADOOP-13650-HADOOP-13345.007.patch Hi, [~ste...@apache.org] Thanks for

[jira] [Updated] (HADOOP-13956) Read ADLS credentials from Credential Provider

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13956: Description: Read ADLS credentials using Hadoop CredentialProvider API. See

[jira] [Updated] (HADOOP-13956) Read ADLS credentials from Credential Provider

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13956: Summary: Read ADLS credentials from Credential Provider (was: Use CredentialProvider API to

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817394#comment-15817394 ] Duo Zhang commented on HADOOP-13433: The failed UT seems unrelated and can pass locally. > Race in

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817379#comment-15817379 ] Hadoop QA commented on HADOOP-13433: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817321#comment-15817321 ] Duo Zhang commented on HADOOP-13433: Oh, TestRaceWhenRelogin. The code I put here is used to verify

[jira] [Commented] (HADOOP-13962) Update ADLS SDK to 2.1.4

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817302#comment-15817302 ] Hadoop QA commented on HADOOP-13962: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Comment Edited] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817295#comment-15817295 ] He Xiaoqiao edited comment on HADOOP-13433 at 1/11/17 5:59 AM: ---

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817295#comment-15817295 ] He Xiaoqiao commented on HADOOP-13433: -- [~Apache9],Thank for your comments. when i patch v2 to

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817249#comment-15817249 ] Duo Zhang commented on HADOOP-13433: What's the problem? In our company we are still running 2.4.x

[jira] [Updated] (HADOOP-13962) Update ADLS SDK to 2.1.4

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13962: Attachment: HADOOP-13962.001.patch Patch 001 * Update ADLS SDK to version 2.1.4 Testing done *

[jira] [Updated] (HADOOP-13962) Update ADLS SDK to 2.1.4

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13962: Fix Version/s: 3.0.0-alpha2 Status: Patch Available (was: Open) > Update ADLS SDK to

[jira] [Commented] (HADOOP-13962) Update ADLS SDK to 2.1.4

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817239#comment-15817239 ] John Zhuge commented on HADOOP-13962: - Microsoft confirms HADOOP-13927 is a backend bug and its fix

[jira] [Updated] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HADOOP-13433: --- Attachment: HADOOP-13433-v4.patch Address the review comments. And use lambda expression in UTs as

[jira] [Commented] (HADOOP-13927) ADLS TestAdlContractRootDirLive.testRmNonEmptyRootDirNonRecursive failed

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817229#comment-15817229 ] John Zhuge commented on HADOOP-13927: - Microsoft confirmed it is a backend bug. Its fix will not

[jira] [Updated] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HADOOP-13433: --- Affects Version/s: 2.8.0 2.7.3 2.6.5

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817057#comment-15817057 ] He Xiaoqiao commented on HADOOP-13433: -- Thanks [~Apache9] for you contribution. it's very useful

[jira] [Commented] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816823#comment-15816823 ] Hadoop QA commented on HADOOP-13961: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13926) S3Guard: Improve listLocatedStatus

2017-01-10 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816583#comment-15816583 ] Rajesh Balamohan commented on HADOOP-13926: --- Agreed. That would need change in s3guard to

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816491#comment-15816491 ] Mingliang Liu commented on HADOOP-13345: I re-visited the retry logic and think it's not a

[jira] [Commented] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816490#comment-15816490 ] Andrew Wang commented on HADOOP-13961: -- Hit submit patch and changed assignee to Sangjin, can

[jira] [Updated] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13961: - Assignee: Sangjin Lee (was: Andrew Wang) > mvn install fails on trunk >

[jira] [Updated] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13961: - Status: Patch Available (was: In Progress) > mvn install fails on trunk >

[jira] [Assigned] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reassigned HADOOP-13961: Assignee: Andrew Wang > mvn install fails on trunk > -- > >

[jira] [Updated] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HADOOP-13961: - Assignee: (was: John Zhuge) > mvn install fails on trunk > -- > >

[jira] [Commented] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816444#comment-15816444 ] Xiaoyu Yao commented on HADOOP-13903: - Thanks [~tmgstev] for adding the traces. Can you fix the

[jira] [Comment Edited] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816284#comment-15816284 ] John Zhuge edited comment on HADOOP-13961 at 1/10/17 10:49 PM: --- +1

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816423#comment-15816423 ] Mingliang Liu commented on HADOOP-13589: We can address this in a separate JIRA. > S3Guard:

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816416#comment-15816416 ] Mingliang Liu commented on HADOOP-13589: Yes I think it makes sense to enable integration test

[jira] [Created] (HADOOP-13971) Fix memory leak in FileSystem.Cache.Key class

2017-01-10 Thread Wei-Chiu Chuang (JIRA)
Wei-Chiu Chuang created HADOOP-13971: Summary: Fix memory leak in FileSystem.Cache.Key class Key: HADOOP-13971 URL: https://issues.apache.org/jira/browse/HADOOP-13971 Project: Hadoop Common

[jira] [Commented] (HADOOP-13114) DistCp should have option to compress data on write

2017-01-10 Thread Ravi Prakash (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816356#comment-15816356 ] Ravi Prakash commented on HADOOP-13114: --- Thanks Koji! I was under the impression that even binary

[jira] [Comment Edited] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816284#comment-15816284 ] John Zhuge edited comment on HADOOP-13961 at 1/10/17 9:46 PM: -- +1

[jira] [Commented] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816284#comment-15816284 ] John Zhuge commented on HADOOP-13961: - +1 (non-binding) on Sanglin's patch 002. Testing #1 existing

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816273#comment-15816273 ] Hadoop QA commented on HADOOP-13336: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816224#comment-15816224 ] Mingliang Liu commented on HADOOP-13876: Thanks [~ste...@apache.org] for clarifying the case. I

[jira] [Assigned] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu reassigned HADOOP-13876: -- Assignee: Mingliang Liu > S3Guard: better support for multi-bucket access including

[jira] [Commented] (HADOOP-13496) Include file lengths in Mismatch in length error for distcp

2017-01-10 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816139#comment-15816139 ] Ted Yu commented on HADOOP-13496: - The patch applies cleanly on latest trunk. bq. which versions does

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816135#comment-15816135 ] Mingliang Liu commented on HADOOP-13336: {quote} the complaint about an unused import is wrong:

[jira] [Commented] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Tristan Stevens (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816051#comment-15816051 ] Tristan Stevens commented on HADOOP-13903: -- Thanks [~asuresh]. The patches we have apply as

[jira] [Updated] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13336: Status: Patch Available (was: Open) > S3A to support per-bucket configuration >

[jira] [Updated] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13336: Attachment: HADOOP-13336-010.patch and trunk version of the same patch > S3A to support

[jira] [Updated] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13336: Attachment: HADOOP-13336-HADOOP-13345-010.patch checkstyle complained about some of the

[jira] [Updated] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13336: Status: Open (was: Patch Available) > S3A to support per-bucket configuration >

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815904#comment-15815904 ] Chris Nauroth commented on HADOOP-13589: I just realized that this doesn't cover the contract

[jira] [Updated] (HADOOP-13970) garbage data read from the beginning of a tar file

2017-01-10 Thread Steve Yang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yang updated HADOOP-13970: Description: Hadoop from CDH 5.7.1 on Spark using databricks

[jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A

2017-01-10 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815897#comment-15815897 ] Aaron Fabbri commented on HADOOP-13345: --- +1 merging in trunk.. Works for me [~ste...@apache.org]

[jira] [Updated] (HADOOP-13970) garbage data read from the beginning of a tar file

2017-01-10 Thread Steve Yang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yang updated HADOOP-13970: Attachment: taxi_simplified.tar bug test file to load into Spark data frame > garbage data read

[jira] [Created] (HADOOP-13970) garbage data read from the beginning of a tar file

2017-01-10 Thread Steve Yang (JIRA)
Steve Yang created HADOOP-13970: --- Summary: garbage data read from the beginning of a tar file Key: HADOOP-13970 URL: https://issues.apache.org/jira/browse/HADOOP-13970 Project: Hadoop Common

[jira] [Commented] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815869#comment-15815869 ] Steve Loughran commented on HADOOP-13936: - presumably DDB Can just be updated after every

[jira] [Commented] (HADOOP-13968) S3a FS to support ".temp_pending_put" path for pending put operations

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815840#comment-15815840 ] Steve Loughran commented on HADOOP-13968: - +when creating a path {{.temp_pending_put/dest.txt}},

[jira] [Created] (HADOOP-13969) S3A to support commit(path) operation, which commits all pending put commits in a path

2017-01-10 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-13969: --- Summary: S3A to support commit(path) operation, which commits all pending put commits in a path Key: HADOOP-13969 URL: https://issues.apache.org/jira/browse/HADOOP-13969

[jira] [Created] (HADOOP-13968) S3a FS to support ".temp_pending_put" path for pending put operations

2017-01-10 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-13968: --- Summary: S3a FS to support ".temp_pending_put" path for pending put operations Key: HADOOP-13968 URL: https://issues.apache.org/jira/browse/HADOOP-13968

[jira] [Created] (HADOOP-13967) S3ABlockOutputStream to support plugin point for different multipart strategies

2017-01-10 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-13967: --- Summary: S3ABlockOutputStream to support plugin point for different multipart strategies Key: HADOOP-13967 URL: https://issues.apache.org/jira/browse/HADOOP-13967

[jira] [Updated] (HADOOP-13786) Add S3Guard committer for zero-rename commits to consistent S3 endpoints

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13786: Issue Type: New Feature (was: Bug) > Add S3Guard committer for zero-rename commits to

[jira] [Updated] (HADOOP-13953) Make FTPFileSystem's data connection mode and transfer mode configurable

2017-01-10 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HADOOP-13953: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Updated] (HADOOP-13531) S3A output streams to share a single LocalDirAllocator for round-robin drive use

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13531: Parent Issue: HADOOP-11694 (was: HADOOP-13204) > S3A output streams to share a single

[jira] [Updated] (HADOOP-12387) Improve S3AFastOutputStream memory management

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-12387: Parent Issue: HADOOP-11694 (was: HADOOP-13204) > Improve S3AFastOutputStream memory

[jira] [Updated] (HADOOP-13281) S3A to track multipart upload count, size, duration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13281: Fix Version/s: (was: 2.9.0) 2.8.0 > S3A to track multipart upload

[jira] [Updated] (HADOOP-13281) S3A to track multipart upload count, size, duration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13281: Parent Issue: HADOOP-11694 (was: HADOOP-13204) > S3A to track multipart upload count,

[jira] [Commented] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815782#comment-15815782 ] Arun Suresh commented on HADOOP-13903: -- +1 for the latest patch. Will commit this to trunk tomorrow

[jira] [Comment Edited] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815782#comment-15815782 ] Arun Suresh edited comment on HADOOP-13903 at 1/10/17 6:34 PM: --- +1 for the

[jira] [Updated] (HADOOP-13857) S3AUtils.translateException to map (wrapped) InterruptedExceptions to InterruptedIOEs

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13857: Parent Issue: HADOOP-11694 (was: HADOOP-13204) > S3AUtils.translateException to map

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Larry McCay (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815719#comment-15815719 ] Larry McCay commented on HADOOP-13336: -- That sounds like right to me, [~ste...@apache.org]. > S3A

[jira] [Commented] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815669#comment-15815669 ] Hadoop QA commented on HADOOP-13903: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815658#comment-15815658 ] Steve Loughran commented on HADOOP-13336: - OK, thanks for the vote. Can I be the one to

[jira] [Updated] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Tristan Stevens (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tristan Stevens updated HADOOP-13903: - Attachment: HADOOP-13903-7.patch > KMS does not provide any useful debug information >

[jira] [Updated] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Tristan Stevens (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tristan Stevens updated HADOOP-13903: - Attachment: HADOOP-13903-branch2.7-7.patch > KMS does not provide any useful debug

[jira] [Commented] (HADOOP-13903) KMS does not provide any useful debug information

2017-01-10 Thread Tristan Stevens (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815524#comment-15815524 ] Tristan Stevens commented on HADOOP-13903: -- Last patches were bad - replacing with String was

[jira] [Updated] (HADOOP-13597) Switch KMS from Tomcat to Jetty

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HADOOP-13597: Release Note: The following environment variables are deprecated. Set the corresponding

[jira] [Created] (HADOOP-13966) add ability to start DDB local server in every test

2017-01-10 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-13966: --- Summary: add ability to start DDB local server in every test Key: HADOOP-13966 URL: https://issues.apache.org/jira/browse/HADOOP-13966 Project: Hadoop Common

[jira] [Updated] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sangjin Lee updated HADOOP-13961: - Attachment: HADOOP-13961.002.patch Would something like this work? There is literally no

[jira] [Updated] (HADOOP-12161) Add getStoragePolicy API to the FileSystem interface

2017-01-10 Thread Sean Busbey (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-12161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HADOOP-12161: - Component/s: fs > Add getStoragePolicy API to the FileSystem interface >

[jira] [Commented] (HADOOP-13961) mvn install fails on trunk

2017-01-10 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815297#comment-15815297 ] John Zhuge commented on HADOOP-13961: - Maybe a cleaner solution is to move {{MiniKMS}} from test to

[jira] [Commented] (HADOOP-13114) DistCp should have option to compress data on write

2017-01-10 Thread Nathan Roberts (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815294#comment-15815294 ] Nathan Roberts commented on HADOOP-13114: - Sorry for jumping in late. I tend to agree this seems

[jira] [Commented] (HADOOP-13650) S3Guard: Provide command line tools to manipulate metadata store.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815166#comment-15815166 ] Steve Loughran commented on HADOOP-13650: - With the filesystem driven customisation, we should

[jira] [Commented] (HADOOP-13114) DistCp should have option to compress data on write

2017-01-10 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815124#comment-15815124 ] Koji Noguchi commented on HADOOP-13114: --- bq. I guess it'd be useful for any files which are

[jira] [Commented] (HADOOP-13876) S3Guard: better support for multi-bucket access including read-only

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815025#comment-15815025 ] Steve Loughran commented on HADOOP-13876: - I like the docs, but as stated: we don't need this

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814915#comment-15814915 ] Steve Loughran commented on HADOOP-13433: - OK, real review Review core patch looks good. *

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814896#comment-15814896 ] Steve Loughran commented on HADOOP-13433: - looks like Hadoop already has minikdc as a test-time

[jira] [Commented] (HADOOP-13965) Groups should be consistent in using group mapping class

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814852#comment-15814852 ] Hadoop QA commented on HADOOP-13965: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814844#comment-15814844 ] Hadoop QA commented on HADOOP-13336: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Duo Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814826#comment-15814826 ] Duo Zhang commented on HADOOP-13433: I do not think we can put all the tests in minikdc as we need

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814820#comment-15814820 ] Hadoop QA commented on HADOOP-13589: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13336) S3A to support per-bucket configuration

2017-01-10 Thread Larry McCay (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814817#comment-15814817 ] Larry McCay commented on HADOOP-13336: -- LGTM +1 > S3A to support per-bucket configuration >

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814781#comment-15814781 ] Steve Loughran commented on HADOOP-13433: - BTW, we need an affects-version marker on the issue

[jira] [Commented] (HADOOP-13433) Race in UGI.reloginFromKeytab

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814779#comment-15814779 ] Steve Loughran commented on HADOOP-13433: - It might be better to move a test into minikdc for

[jira] [Updated] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13589: Attachment: HADOOP-13589-HADOOP-13345-002.patch patch 002, generated properly > S3Guard:

[jira] [Updated] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13589: Status: Patch Available (was: Open) > S3Guard: Allow execution of all S3A integration

[jira] [Updated] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13589: Status: Open (was: Patch Available) > S3Guard: Allow execution of all S3A integration

[jira] [Updated] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13589: Affects Version/s: HADOOP-13345 Status: Patch Available (was: In Progress)

[jira] [Updated] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13589: Attachment: HADOOP-13589-HADOOP-13345-002.patch patch 002: non-auth by default. Not

[jira] [Commented] (HADOOP-13589) S3Guard: Allow execution of all S3A integration tests with S3Guard enabled.

2017-01-10 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15814762#comment-15814762 ] Steve Loughran commented on HADOOP-13589: - I see. So the 'dynamo' thing is moot. Ideally I'd

[jira] [Updated] (HADOOP-13965) Groups should be consistent in using group mapping class

2017-01-10 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-13965: --- Attachment: HADOOP-13965.001.patch > Groups should be consistent in using group mapping class >

[jira] [Updated] (HADOOP-13965) Groups should be consistent in using group mapping class

2017-01-10 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-13965: --- Status: Patch Available (was: Open) Attach a simple patch to fix. > Groups should be consistent

[jira] [Updated] (HADOOP-13965) Groups should be consistent in using group mapping class

2017-01-10 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HADOOP-13965: --- Description: {code:title=Groups.java} public Groups(Configuration conf, final Timer timer) {

  1   2   >