[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399030896 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399026980 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399027614 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399027244 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399027766 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399029873 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399028369 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399029674 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399030377 ## File path:

[GitHub] [hadoop] snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream

2020-03-26 Thread GitBox
snvijaya commented on a change in pull request #1907: HADOOP-16854 Fix to prevent OutOfMemoryException and Make the threadpool and bytebuffer pool common across all AbfsOutputStream instances URL: https://github.com/apache/hadoop/pull/1907#discussion_r399027883 ## File path:

[GitHub] [hadoop] hadoop-yetus commented on issue #1820: HADOOP-16830. Add public IOStatistics API + S3A implementation

2020-03-26 Thread GitBox
hadoop-yetus commented on issue #1820: HADOOP-16830. Add public IOStatistics API + S3A implementation URL: https://github.com/apache/hadoop/pull/1820#issuecomment-604718975 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[jira] [Commented] (HADOOP-16090) S3A Client to add explicit support for versioned stores

2020-03-26 Thread Dmitri Chmelev (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068087#comment-17068087 ] Dmitri Chmelev commented on HADOOP-16090: - Depending on your use case, you can mitigate this

[jira] [Commented] (HADOOP-16090) S3A Client to add explicit support for versioned stores

2020-03-26 Thread Dmitri Chmelev (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068083#comment-17068083 ] Dmitri Chmelev commented on HADOOP-16090: - This issue is a function of the number of writes,

[jira] [Commented] (HADOOP-16090) S3A Client to add explicit support for versioned stores

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068044#comment-17068044 ] Steve Loughran commented on HADOOP-16090: - [~rsatish1110] the parse errors are from the v1 list

[jira] [Commented] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068043#comment-17068043 ] Steve Loughran commented on HADOOP-16942: - (FWIW, this issue surfaces because on versioned

[jira] [Updated] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16942: Priority: Major (was: Minor) > S3A creating folder level delete markers >

[jira] [Resolved] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-16942. - Fix Version/s: HADOOP-13230 Resolution: Duplicate Duplicate of HADOOP-13230

[jira] [Updated] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16942: Fix Version/s: (was: HADOOP-13230) > S3A creating folder level delete markers >

[GitHub] [hadoop] hadoop-yetus commented on issue #1916: MAPREDUCE-7267 Merge paths with multi threads during commit job in FileOutputCommitter

2020-03-26 Thread GitBox
hadoop-yetus commented on issue #1916: MAPREDUCE-7267 Merge paths with multi threads during commit job in FileOutputCommitter URL: https://github.com/apache/hadoop/pull/1916#issuecomment-604672664 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime |

[jira] [Commented] (HADOOP-15743) Jetty and SSL tunings to stabilize KMS performance

2020-03-26 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17068001#comment-17068001 ] Jonathan Hung commented on HADOOP-15743: Yeah, we've been hitting this issue on an HDFS cluster

[jira] [Commented] (HADOOP-15743) Jetty and SSL tunings to stabilize KMS performance

2020-03-26 Thread Kihwal Lee (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067988#comment-17067988 ] Kihwal Lee commented on HADOOP-15743: - Loosely related:

[jira] [Commented] (HADOOP-16939) fs.s3a.authoritative.path should support multiple FS URIs

2020-03-26 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067969#comment-17067969 ] Hudson commented on HADOOP-16939: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18097 (See

[jira] [Updated] (HADOOP-16939) fs.s3a.authoritative.path should support multiple FS URIs

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16939: Parent Issue: HADOOP-15620 (was: HADOOP-16829) > fs.s3a.authoritative.path should

[GitHub] [hadoop] mackrorysd merged pull request #1914: HADOOP-16939. fs.s3a.authoritative.path should support multiple FS URIs

2020-03-26 Thread GitBox
mackrorysd merged pull request #1914: HADOOP-16939. fs.s3a.authoritative.path should support multiple FS URIs URL: https://github.com/apache/hadoop/pull/1914 This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-15743) Jetty and SSL tunings to stabilize KMS performance

2020-03-26 Thread Jonathan Hung (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067936#comment-17067936 ] Jonathan Hung commented on HADOOP-15743: [~daryn] where did you find the config

[GitHub] [hadoop] RogPodge closed pull request #1399: YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover

2020-03-26 Thread GitBox
RogPodge closed pull request #1399: YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover URL: https://github.com/apache/hadoop/pull/1399 This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] turboFei commented on issue #1916: [MAPREDUCE-7267] Merge paths with multi threads during commit job in FileOutputCommitter

2020-03-26 Thread GitBox
turboFei commented on issue #1916: [MAPREDUCE-7267] Merge paths with multi threads during commit job in FileOutputCommitter URL: https://github.com/apache/hadoop/pull/1916#issuecomment-604581478 cc @steveloughran This is an

[GitHub] [hadoop] turboFei opened a new pull request #1916: [MAPREDUCE-7267] Merge paths with multi threads during commit job in FileOutputCommitter

2020-03-26 Thread GitBox
turboFei opened a new pull request #1916: [MAPREDUCE-7267] Merge paths with multi threads during commit job in FileOutputCommitter URL: https://github.com/apache/hadoop/pull/1916 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the

[GitHub] [hadoop] steveloughran commented on issue #1914: HADOOP-16939. fs.s3a.authoritative.path should support multiple FS URIs

2020-03-26 Thread GitBox
steveloughran commented on issue #1914: HADOOP-16939. fs.s3a.authoritative.path should support multiple FS URIs URL: https://github.com/apache/hadoop/pull/1914#issuecomment-604562394 through yetus first time -I am pleasantly surprised. Tested: S3 london

[jira] [Commented] (HADOOP-16818) ABFS: Combine append+flush calls for blockblob & appendblob

2020-03-26 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067793#comment-17067793 ] Hudson commented on HADOOP-16818: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18096 (See

[jira] [Created] (HADOOP-16943) Allow IPv6 in hadoop_config.sh

2020-03-26 Thread Luca Toscano (Jira)
Luca Toscano created HADOOP-16943: - Summary: Allow IPv6 in hadoop_config.sh Key: HADOOP-16943 URL: https://issues.apache.org/jira/browse/HADOOP-16943 Project: Hadoop Common Issue Type:

[jira] [Updated] (HADOOP-16943) Allow IPv6 in hadoop_config.sh

2020-03-26 Thread Luca Toscano (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Toscano updated HADOOP-16943: -- Description: Hi! I am currently using CDH 5.x and moving to BigTop 1.4 / 1.5 (so jumping

[GitHub] [hadoop] goiri merged pull request #1915: YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover

2020-03-26 Thread GitBox
goiri merged pull request #1915: YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover URL: https://github.com/apache/hadoop/pull/1915 This is an automated message from the Apache Git Service. To

[jira] [Reopened] (HADOOP-16818) ABFS: Combine append+flush calls for blockblob & appendblob

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-16818: - > ABFS: Combine append+flush calls for blockblob & appendblob >

[jira] [Assigned] (HADOOP-16937) Multiple Tests failure in hadoop-azure componnt

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-16937: --- Assignee: Ishani > Multiple Tests failure in hadoop-azure componnt >

[jira] [Updated] (HADOOP-16937) Multiple Tests failure in hadoop-azure componnt

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16937: Affects Version/s: (was: 3.2.1) 3.3.0 > Multiple Tests

[jira] [Updated] (HADOOP-16937) Multiple Tests failure in hadoop-azure componnt

2020-03-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-16937: Component/s: test > Multiple Tests failure in hadoop-azure componnt >

[jira] [Updated] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated HADOOP-16942: --- Description: Using S3A URL scheme while writing out data from Spark to S3 is creating many

[jira] [Commented] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067701#comment-17067701 ] vijayant soni commented on HADOOP-16942: As per

[jira] [Comment Edited] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067692#comment-17067692 ] vijayant soni edited comment on HADOOP-16942 at 3/26/20, 1:57 PM: --

[jira] [Commented] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067692#comment-17067692 ] vijayant soni commented on HADOOP-16942: Error when there are too many delete markers.

[jira] [Updated] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated HADOOP-16942: --- Description: Using S3A URL scheme while writing out data from Spark to S3 is creating many

[GitHub] [hadoop] hadoop-yetus commented on issue #1640: HADOOP-16637. Fix findbugs warnings in hadoop-cos.

2020-03-26 Thread GitBox
hadoop-yetus commented on issue #1640: HADOOP-16637. Fix findbugs warnings in hadoop-cos. URL: https://github.com/apache/hadoop/pull/1640#issuecomment-604433726 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[jira] [Updated] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vijayant soni updated HADOOP-16942: --- Description: Using S3A URL scheme while writing out data from Spark to S3 is creating many

[jira] [Created] (HADOOP-16942) S3A creating folder level delete markers

2020-03-26 Thread vijayant soni (Jira)
vijayant soni created HADOOP-16942: -- Summary: S3A creating folder level delete markers Key: HADOOP-16942 URL: https://issues.apache.org/jira/browse/HADOOP-16942 Project: Hadoop Common Issue

[GitHub] [hadoop] esahekmat commented on a change in pull request #1904: HADOOP-16931. Reduce the execution time of the TestRolloverSignerSecretProvider

2020-03-26 Thread GitBox
esahekmat commented on a change in pull request #1904: HADOOP-16931. Reduce the execution time of the TestRolloverSignerSecretProvider URL: https://github.com/apache/hadoop/pull/1904#discussion_r396005668 ## File path:

[jira] [Resolved] (HADOOP-16936) Failure in Abfs tests

2020-03-26 Thread Mehakmeet Singh (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehakmeet Singh resolved HADOOP-16936. -- Resolution: Duplicate > Failure in Abfs tests > - > >

[jira] [Commented] (HADOOP-16936) Failure in Abfs tests

2020-03-26 Thread Mehakmeet Singh (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067441#comment-17067441 ] Mehakmeet Singh commented on HADOOP-16936: -- duplicate of HADOOP-16937 > Failure in Abfs tests