[jira] [Commented] (HADOOP-15696) KMS performance regression due to too many open file descriptors after Jetty migration

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597076#comment-16597076 ] Xiao Chen commented on HADOOP-15696: Thanks for the work here [~jojochuang] and

[jira] [Comment Edited] (HADOOP-15698) KMS log4j is not initialized properly at startup

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597067#comment-16597067 ] Xiao Chen edited comment on HADOOP-15698 at 8/30/18 5:11 AM: - Committed to

[jira] [Updated] (HADOOP-15698) KMS log4j is not initialized properly at startup

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HADOOP-15698: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.2 3.0.4

[jira] [Updated] (HADOOP-15698) KMS log4j is not initialized properly at startup

2018-08-29 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HADOOP-15698: --- Summary: KMS log4j is not initialized properly at startup (was: KMS startup logs don't show) >

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597030#comment-16597030 ] Surendra Singh Lilhore commented on HADOOP-15684: - [~trjianjianjiao] : I didn't see any

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596968#comment-16596968 ] Da Zhou commented on HADOOP-15663: -- Attaching HADOOP-15663-HADOOP-15407-005.patch. There is a conflict

[jira] [Updated] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15663: - Attachment: HADOOP-15663-HADOOP-15407-005.patch > ABFS: Simplify configuration >

[jira] [Comment Edited] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596843#comment-16596843 ] Da Zhou edited comment on HADOOP-15663 at 8/30/18 1:11 AM: --- Thanks for the

[jira] [Commented] (HADOOP-15407) Support Windows Azure Storage - Blob file system in Hadoop

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596884#comment-16596884 ] Da Zhou commented on HADOOP-15407: -- Yes, [~sunilg], it will be ready. We are now working on JIRAS

[jira] [Commented] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596856#comment-16596856 ] Daniel Templeton commented on HADOOP-15705: --- Filed HADOOP-15706 for the other typo you found.

[jira] [Created] (HADOOP-15706) Typo in compatibility doc: SHOUD -> SHOULD

2018-08-29 Thread Daniel Templeton (JIRA)
Daniel Templeton created HADOOP-15706: - Summary: Typo in compatibility doc: SHOUD -> SHOULD Key: HADOOP-15706 URL: https://issues.apache.org/jira/browse/HADOOP-15706 Project: Hadoop Common

[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.2

[jira] [Commented] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2018-08-29 Thread Steve Jacobs (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596852#comment-16596852 ] Steve Jacobs commented on HADOOP-15628: --- I think this is fixed in 3.1, at least from my look at

[jira] [Comment Edited] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596843#comment-16596843 ] Da Zhou edited comment on HADOOP-15663 at 8/29/18 8:56 PM: --- Thanks for the

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596843#comment-16596843 ] Da Zhou commented on HADOOP-15663: -- Thanks for the feedback. Attaching patch:

[jira] [Updated] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15663: - Attachment: HADOOP-15663-HADOOP-15407-004.patch > ABFS: Simplify configuration >

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Thomas Marquardt (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596835#comment-16596835 ] Thomas Marquardt commented on HADOOP-15663: --- LGTM too.  Some comments:

[jira] [Commented] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596828#comment-16596828 ] Wei-Chiu Chuang commented on HADOOP-15705: -- +1 thanks for clarification [~templedf]   BTW,

[jira] [Commented] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596817#comment-16596817 ] Aaron Fabbri commented on HADOOP-14630: --- +1 on the patch as soon as you get a green result from

[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Status: Patch Available (was: Open) > Typo in the definition of "stable" in the

[jira] [Updated] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated HADOOP-15705: -- Attachment: HADOOP-15705.001.patch > Typo in the definition of "stable" in the

[jira] [Created] (HADOOP-15705) Typo in the definition of "stable" in the interface classification

2018-08-29 Thread Daniel Templeton (JIRA)
Daniel Templeton created HADOOP-15705: - Summary: Typo in the definition of "stable" in the interface classification Key: HADOOP-15705 URL: https://issues.apache.org/jira/browse/HADOOP-15705

[jira] [Commented] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596784#comment-16596784 ] Aaron Fabbri commented on HADOOP-15107: --- +1 on the v4 patch. Code all looks good. {noformat} +

[jira] [Commented] (HADOOP-10948) SwiftNativeFileSystem's directory is incompatible with Swift and Horizon

2018-08-29 Thread Shyamsundar Ramanathan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596780#comment-16596780 ] Shyamsundar Ramanathan commented on HADOOP-10948: - * We applied the patch and have

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread JIRA
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596704#comment-16596704 ] Íñigo Goiri commented on HADOOP-15684: -- I'll follow up in HDFS-13805 for the

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread Rong Tang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596696#comment-16596696 ] Rong Tang commented on HADOOP-15684: [~elgoiri]    For test case TestJournalNodeSync,  it also

[jira] [Commented] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596683#comment-16596683 ] Aaron Fabbri commented on HADOOP-15107: --- I don't want to rob others of the joys of learning the

[jira] [Commented] (HADOOP-15658) Memory leak in S3AOutputStream

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596667#comment-16596667 ] Steve Loughran commented on HADOOP-15658: - I've just moved this to being a 3.3 feature, but if

[jira] [Resolved] (HADOOP-15022) s3guard IT tests increase R/W capacity of the test table by 1

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15022. - Resolution: Duplicate > s3guard IT tests increase R/W capacity of the test table by 1

[jira] [Updated] (HADOOP-15658) Memory leak in S3AOutputStream

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15658: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15620 > Memory leak in

[jira] [Commented] (HADOOP-15226) Über-JIRA: S3Guard Phase III: Hadoop 3.2 features

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596665#comment-16596665 ] Steve Loughran commented on HADOOP-15226: - For people watching this: just moved all the

[jira] [Updated] (HADOOP-15667) FileSystemMultipartUploader should verify that UploadHandle has non-0 length

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15667: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15620 >

[jira] [Updated] (HADOOP-15667) FileSystemMultipartUploader should verify that UploadHandle has non-0 length

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15667: Affects Version/s: 3.2.0 > FileSystemMultipartUploader should verify that UploadHandle

[jira] [Updated] (HADOOP-15667) FileSystemMultipartUploader should verify that UploadHandle has non-0 length

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15667: Component/s: fs/s3 > FileSystemMultipartUploader should verify that UploadHandle has

[jira] [Updated] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15426: Attachment: HADOOP-15426-007.patch > Make S3guard client resilient to DDB throttle

[jira] [Updated] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15426: Status: Patch Available (was: Open) > Make S3guard client resilient to DDB throttle

[jira] [Updated] (HADOOP-15426) Make S3guard client resilient to DDB throttle events and network failures

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15426: Status: Open (was: Patch Available) > Make S3guard client resilient to DDB throttle

[jira] [Updated] (HADOOP-14159) Add some Java-8 friendly way to work with RemoteIterable, especially listings

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14159: Priority: Minor (was: Major) > Add some Java-8 friendly way to work with

[jira] [Updated] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14630: Status: Patch Available (was: Open) patch 004; rebased for trunk > Contract Tests to

[jira] [Updated] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14630: Attachment: HADOOP-14630-004.patch > Contract Tests to verify create, mkdirs and rename

[jira] [Updated] (HADOOP-14630) Contract Tests to verify create, mkdirs and rename under a file is forbidden

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14630: Target Version/s: (was: 3.2.0) > Contract Tests to verify create, mkdirs and rename

[jira] [Updated] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15628: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15220 > S3A Filesystem does not

[jira] [Commented] (HADOOP-13278) S3AFileSystem mkdirs does not need to validate parent path components

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596636#comment-16596636 ] Steve Loughran commented on HADOOP-13278: - Moving to branch-3.3 As noted, we do want to check

[jira] [Resolved] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14483. - Resolution: Won't Fix > increase default value of fs.s3a.multipart.size to 128M >

[jira] [Updated] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14483: Parent Issue: HADOOP-15220 (was: HADOOP-15620) > increase default value of

[jira] [Commented] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596621#comment-16596621 ] Steve Loughran commented on HADOOP-14483: - I'm going to close as WONTFIX. There are some subtle

[jira] [Updated] (HADOOP-14483) increase default value of fs.s3a.multipart.size to 128M

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14483: Target Version/s: (was: 3.3.0) > increase default value of fs.s3a.multipart.size to

[jira] [Commented] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596618#comment-16596618 ] Steve Loughran commented on HADOOP-15107: - I want this to go in to Hadoop 3.2; there's no

[jira] [Updated] (HADOOP-15107) Stabilize/tune S3A committers; review correctness & docs

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15107: Priority: Blocker (was: Major) > Stabilize/tune S3A committers; review correctness &

[jira] [Updated] (HADOOP-14833) Remove s3a user:secret authentication

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14833: Target Version/s: (was: 3.2.0) > Remove s3a user:secret authentication >

[jira] [Updated] (HADOOP-14468) S3Guard: make short-circuit getFileStatus() configurable

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14468: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > S3Guard: make short-circuit

[jira] [Updated] (HADOOP-15183) S3Guard store becomes inconsistent after partial failure of rename

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15183: Target Version/s: (was: 3.2.0) > S3Guard store becomes inconsistent after partial

[jira] [Updated] (HADOOP-15183) S3Guard store becomes inconsistent after partial failure of rename

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15183: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > S3Guard store becomes inconsistent

[jira] [Updated] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13936: Target Version/s: (was: 3.2.0) > S3Guard: DynamoDB can go out of sync with

[jira] [Updated] (HADOOP-13936) S3Guard: DynamoDB can go out of sync with S3AFileSystem::delete operation

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-13936: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > S3Guard: DynamoDB can go out of sync

[jira] [Updated] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15664: - Status: Patch Available (was: Open) > ABFS: Reduce test run time via parallelization and grouping >

[jira] [Updated] (HADOOP-14158) Possible for modified configuration to leak into metadatastore in S3GuardTool

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14158: Parent Issue: HADOOP-15619 (was: HADOOP-15226) > Possible for modified configuration to

[jira] [Commented] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596607#comment-16596607 ] Da Zhou commented on HADOOP-15664: -- Attaching HADOOP-15664-HADOOP-15407-002.patch: - Fixed import

[jira] [Updated] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Da Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Da Zhou updated HADOOP-15664: - Attachment: HADOOP-15664-HADOOP-15407-002.patch > ABFS: Reduce test run time via parallelization and

[jira] [Commented] (HADOOP-15628) S3A Filesystem does not check return from AmazonS3Client deleteObjects

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596602#comment-16596602 ] Steve Loughran commented on HADOOP-15628: - [~steveatbat]: if you can do a patch for this ASAP

[jira] [Updated] (HADOOP-15244) s3guard uploads command to add a way to complete outstanding uploads

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15244: Parent Issue: HADOOP-15220 (was: HADOOP-15620) > s3guard uploads command to add a way

[jira] [Updated] (HADOOP-15244) s3guard uploads command to add a way to complete outstanding uploads

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15244: Target Version/s: (was: 3.3.0) > s3guard uploads command to add a way to complete

[jira] [Updated] (HADOOP-14833) Remove s3a user:secret authentication

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14833: Parent: HADOOP-15620 (was: HADOOP-15220) > Remove s3a user:secret authentication >

[jira] [Updated] (HADOOP-15409) S3AFileSystem.verifyBucketExists to move to s3.doesBucketExistV2

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15409: Parent: HADOOP-15620 (was: HADOOP-15220) > S3AFileSystem.verifyBucketExists to move to

[jira] [Updated] (HADOOP-15651) NPE in S3AInputStream.read() in ITestS3AInconsistency.testOpenFailOnRead

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15651: Parent: HADOOP-15620 (was: HADOOP-15220) > NPE in S3AInputStream.read() in

[jira] [Updated] (HADOOP-14833) Remove s3a user:secret authentication

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-14833: Priority: Major (was: Blocker) > Remove s3a user:secret authentication >

[jira] [Commented] (HADOOP-15684) triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

2018-08-29 Thread JIRA
[ https://issues.apache.org/jira/browse/HADOOP-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596546#comment-16596546 ] Íñigo Goiri commented on HADOOP-15684: -- We would need a review from somebody familiar with HA and

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Sunil Govindan (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596532#comment-16596532 ] Sunil Govindan commented on HADOOP-15430: - Thanks [~ste...@apache.org]. > hadoop fs -mkdir -p

[jira] [Commented] (HADOOP-15703) ABFS - Implement client-side throttling

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596527#comment-16596527 ] Steve Loughran commented on HADOOP-15703: - + it'd be good to get these metrics into the ABFS

[jira] [Assigned] (HADOOP-15703) ABFS - Implement client-side throttling

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-15703: --- Assignee: Sneha Varma > ABFS - Implement client-side throttling >

[jira] [Updated] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15430: Priority: Minor (was: Blocker) > hadoop fs -mkdir -p path-ending-with-slash/ fails with

[jira] [Updated] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15430: Status: Open (was: Patch Available) > hadoop fs -mkdir -p path-ending-with-slash/ fails

[jira] [Updated] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15430: Target Version/s: (was: 3.2.0) > hadoop fs -mkdir -p path-ending-with-slash/ fails

[jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596520#comment-16596520 ] Steve Loughran commented on HADOOP-15430: - Lets downgrade this one; I can live without it. I do

[jira] [Commented] (HADOOP-15602) Support SASL Rpc request handling in separate Handlers

2018-08-29 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596518#comment-16596518 ] Vinayakumar B commented on HADOOP-15602: Hi [~daryn] , Wondering whether you got chance to

[jira] [Commented] (HADOOP-15703) ABFS - Implement client-side throttling

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596519#comment-16596519 ] Steve Loughran commented on HADOOP-15703: - Looks good, excluding what I'm about to say about

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596501#comment-16596501 ] Vishwajeet Dusane commented on HADOOP-15700: The scenario i am trying to address is ABFS

[jira] [Commented] (HADOOP-15676) Cleanup TestSSLHttpServer

2018-08-29 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596489#comment-16596489 ] Szilard Nemeth commented on HADOOP-15676: - Hey [~zsiegl]! Thanks for your comments. Fixed the

[jira] [Commented] (HADOOP-15676) Cleanup TestSSLHttpServer

2018-08-29 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596488#comment-16596488 ] Szilard Nemeth commented on HADOOP-15676: - Hey [~zsiegl]! Thanks for your comments. Fixed the

[jira] [Updated] (HADOOP-15676) Cleanup TestSSLHttpServer

2018-08-29 Thread Szilard Nemeth (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated HADOOP-15676: Attachment: HADOOP-15676.002.patch > Cleanup TestSSLHttpServer >

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Thomas Marquardt (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596468#comment-16596468 ] Thomas Marquardt commented on HADOOP-15700: --- [~vishwajeet.dusane], who is going to build this

[jira] [Commented] (HADOOP-15664) ABFS: Reduce test run time via parallelization and grouping

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596445#comment-16596445 ] Steve Loughran commented on HADOOP-15664: - This is a really great speedup! Everyone will

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

2018-08-29 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596443#comment-16596443 ] Vishwajeet Dusane commented on HADOOP-15669: Thank you for clarification

[jira] [Commented] (HADOOP-15663) ABFS: Simplify configuration

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596442#comment-16596442 ] Steve Loughran commented on HADOOP-15663: - Looks good # the testing doc is going to need

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Vishwajeet Dusane (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596439#comment-16596439 ] Vishwajeet Dusane commented on HADOOP-15700: Precisely the scenario i was trying to address

[jira] [Commented] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596437#comment-16596437 ] Steve Loughran commented on HADOOP-15702: - sounds good; all to easy to write test which work

[jira] [Updated] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15702: Component/s: fs/azure > ABFS: Increase timeout of ITestAbfsReadWriteAndSeek >

[jira] [Updated] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15702: Component/s: test > ABFS: Increase timeout of ITestAbfsReadWriteAndSeek >

[jira] [Updated] (HADOOP-15702) ABFS: Increase timeout of ITestAbfsReadWriteAndSeek

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15702: Affects Version/s: HADOOP-15407 > ABFS: Increase timeout of ITestAbfsReadWriteAndSeek >

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596434#comment-16596434 ] Steve Loughran commented on HADOOP-15669: - (BTW: The other stores generally offer a config

[jira] [Commented] (HADOOP-15669) ABFS: Improve HTTPS Performance

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596433#comment-16596433 ] Steve Loughran commented on HADOOP-15669: - bq.

[jira] [Updated] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15700: Issue Type: Sub-task (was: Bug) Parent: HADOOP-15407 > ABFS: Failure in

[jira] [Commented] (HADOOP-15700) ABFS: Failure in OpenSSLProvider should fall back to JSSE

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596430#comment-16596430 ] Steve Loughran commented on HADOOP-15700: - there's never any guarantee that a java-8 compiled

[jira] [Commented] (HADOOP-15697) memory leak in distcp run method

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596428#comment-16596428 ] Steve Loughran commented on HADOOP-15697: - Can you tag this JIRA with the hadoop version you

[jira] [Updated] (HADOOP-15697) memory leak in distcp run method

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-15697: Component/s: tools/distcp > memory leak in distcp run method >

[jira] [Commented] (HADOOP-15695) In wasbFS user group always set via core-site

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596424#comment-16596424 ] Steve Loughran commented on HADOOP-15695: - +[~tmarquardt] > In wasbFS user group always set

[jira] [Commented] (HADOOP-15680) ITestNativeAzureFileSystemConcurrencyLive times out

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596417#comment-16596417 ] Steve Loughran commented on HADOOP-15680: - looks good. +1 >

[jira] [Commented] (HADOOP-15635) s3guard set-capacity command to fail fast if bucket is unguarded

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596397#comment-16596397 ] Steve Loughran commented on HADOOP-15635: - well, {{S3AFileSystem.hasMetadataStore()}} returns

[jira] [Assigned] (HADOOP-14734) add option to tag DDB table(s) created

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-14734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-14734: --- Assignee: Abraham Fine (was: Gabor Bota) > add option to tag DDB table(s)

[jira] [Commented] (HADOOP-15688) ABFS: InputStream wrapped in FSDataInputStream twice

2018-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596378#comment-16596378 ] Steve Loughran commented on HADOOP-15688: - we should make supporting byte buffer readable

  1   2   >