[jira] [Commented] (HADOOP-16850) Support getting thread info from thread group for JvmMetrics to improve the performance

2020-02-12 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035994#comment-17035994 ] Tao Yang commented on HADOOP-16850: --- Thanks [~aajisaka] for your suggestion. There are dozens of

[jira] [Updated] (HADOOP-16850) Support getting thread info from thread group for JvmMetrics to improve the performance

2020-02-12 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Yang updated HADOOP-16850: -- Attachment: HADOOP-16850.002.patch > Support getting thread info from thread group for JvmMetrics to

[jira] [Commented] (HADOOP-16850) Support getting thread info from thread group for JvmMetrics to improve the performance

2020-02-12 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035868#comment-17035868 ] Akira Ajisaka commented on HADOOP-16850: Thank [~Tao Yang] for the reply. I think that is ok.

[jira] [Commented] (HADOOP-16850) Support getting thread info from thread group for JvmMetrics to improve the performance

2020-02-12 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035857#comment-17035857 ] Tao Yang commented on HADOOP-16850: --- Thanks [~aajisaka] for the review. {quote}Would you add a Hadoop

[jira] [Commented] (HADOOP-16850) Support getting thread info from thread group for JvmMetrics to improve the performance

2020-02-12 Thread Akira Ajisaka (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035847#comment-17035847 ] Akira Ajisaka commented on HADOOP-16850: Thanks [~Tao Yang] for the report and providing the

[jira] [Commented] (HADOOP-16850) Support getting thread info from thread group for JvmMetrics to improve the performance

2020-02-12 Thread Tao Yang (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035835#comment-17035835 ] Tao Yang commented on HADOOP-16850: --- Hi, [~aajisaka], could you please take a look at this issue?

[GitHub] [hadoop] aajisaka commented on issue #1832: HDFS-13989. RBF: Add FSCK to the Router

2020-02-12 Thread GitBox
aajisaka commented on issue #1832: HDFS-13989. RBF: Add FSCK to the Router URL: https://github.com/apache/hadoop/pull/1832#issuecomment-585497638 Merged. Thanks @goiri for your reviews. This is an automated message from the

[GitHub] [hadoop] aajisaka merged pull request #1832: HDFS-13989. RBF: Add FSCK to the Router

2020-02-12 Thread GitBox
aajisaka merged pull request #1832: HDFS-13989. RBF: Add FSCK to the Router URL: https://github.com/apache/hadoop/pull/1832 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [hadoop] hadoop-yetus commented on issue #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1845#issuecomment-585428044 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] hadoop-yetus commented on issue #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1845#issuecomment-585421681 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] steveloughran commented on issue #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
steveloughran commented on issue #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1845#issuecomment-585398805 new exception breaks a few tests ``` INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:

[GitHub] [hadoop] steveloughran opened a new pull request #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
steveloughran opened a new pull request #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1845 see #1838 Adds a new exception UnknownStoreException to indicate "there's no store there" * raised in verify

[GitHub] [hadoop] hadoop-yetus commented on issue #1842: HADOOP-16730 : ABFS: Add Authorizer Interface

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#issuecomment-585379660 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] hadoop-yetus commented on issue #1844: HADOOP-16706. ITestClientUrlScheme fails for accounts which don't support HTTP

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1844: HADOOP-16706. ITestClientUrlScheme fails for accounts which don't support HTTP URL: https://github.com/apache/hadoop/pull/1844#issuecomment-585376509 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] steveloughran commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
steveloughran commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#issuecomment-585351633 Also once I fix that by adding a trailing /, the getFileStatus("/") fails to raise an FNFE, which is because

[GitHub] [hadoop] snvijaya commented on issue #1842: HADOOP-16730 : ABFS: Add Authorizer Interface

2020-02-12 Thread GitBox
snvijaya commented on issue #1842: HADOOP-16730 : ABFS: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#issuecomment-585350897 Test results for account without namespace enabled (East US 2): [INFO] Tests run: 52, Failures: 0, Errors: 0, Skipped: 0 [ERROR]

[jira] [Updated] (HADOOP-16730) ABFS: Support for Shared Access Signatures (SAS)

2020-02-12 Thread Sneha Vijayarajan (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan updated HADOOP-16730: --- Status: Patch Available (was: Open) > ABFS: Support for Shared Access Signatures

[GitHub] [hadoop] steveloughran commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
steveloughran commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#issuecomment-585343180 testing myself, setting validation to 0 for the entire test run to field test it better. one failure so far

[GitHub] [hadoop] liuml07 commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets

2020-02-12 Thread GitBox
liuml07 commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets URL: https://github.com/apache/hadoop/pull/1840#discussion_r378422262 ## File path:

[GitHub] [hadoop] liuml07 commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets

2020-02-12 Thread GitBox
liuml07 commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets URL: https://github.com/apache/hadoop/pull/1840#discussion_r378421141 ## File path:

[GitHub] [hadoop] steveloughran commented on a change in pull request #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
steveloughran commented on a change in pull request #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#discussion_r378419167 ## File path:

[GitHub] [hadoop] steveloughran opened a new pull request #1844: HADOOP-16706. ITestClientUrlScheme fails for accounts which don't support HTTP

2020-02-12 Thread GitBox
steveloughran opened a new pull request #1844: HADOOP-16706. ITestClientUrlScheme fails for accounts which don't support HTTP URL: https://github.com/apache/hadoop/pull/1844 Adds a new service code to recognise accounts without HTTP support; catches that and considers that a

[jira] [Work started] (HADOOP-16706) ITestClientUrlScheme fails for accounts which don't support HTTP

2020-02-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-16706 started by Steve Loughran. --- > ITestClientUrlScheme fails for accounts which don't support HTTP >

[jira] [Assigned] (HADOOP-16706) ITestClientUrlScheme fails for accounts which don't support HTTP

2020-02-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-16706: --- Assignee: Steve Loughran (was: Gabor Bota) > ITestClientUrlScheme fails for

[jira] [Created] (HADOOP-16859) ABFS: Add unbuffer support to AbfsInputStream

2020-02-12 Thread Sahil Takiar (Jira)
Sahil Takiar created HADOOP-16859: - Summary: ABFS: Add unbuffer support to AbfsInputStream Key: HADOOP-16859 URL: https://issues.apache.org/jira/browse/HADOOP-16859 Project: Hadoop Common

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1826: HADOOP-16823. Large DeleteObject requests are their own Thundering Herd

2020-02-12 Thread GitBox
hadoop-yetus removed a comment on issue #1826: HADOOP-16823. Large DeleteObject requests are their own Thundering Herd URL: https://github.com/apache/hadoop/pull/1826#issuecomment-584758279 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] hadoop-yetus commented on issue #1843: HADOOP-16794. encryption over rename/copy

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1843: HADOOP-16794. encryption over rename/copy URL: https://github.com/apache/hadoop/pull/1843#issuecomment-585232459 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment |

[jira] [Created] (HADOOP-16858) S3Guard fsck: Add option to prune orphaned entries

2020-02-12 Thread Gabor Bota (Jira)
Gabor Bota created HADOOP-16858: --- Summary: S3Guard fsck: Add option to prune orphaned entries Key: HADOOP-16858 URL: https://issues.apache.org/jira/browse/HADOOP-16858 Project: Hadoop Common

[GitHub] [hadoop] steveloughran commented on issue #1823: HADOOP-16794 S3 Encryption keys not propagating correctly during copy operation

2020-02-12 Thread GitBox
steveloughran commented on issue #1823: HADOOP-16794 S3 Encryption keys not propagating correctly during copy operation URL: https://github.com/apache/hadoop/pull/1823#issuecomment-585208625 see #1843 for a patch on top of this, to help propagate settings better; let's keep discussion in

[GitHub] [hadoop] steveloughran commented on issue #1843: HADOOP-16794. encryption over rename/copy

2020-02-12 Thread GitBox
steveloughran commented on issue #1843: HADOOP-16794. encryption over rename/copy URL: https://github.com/apache/hadoop/pull/1843#issuecomment-585207520 Testing: only tested this new test with * client set to SSE-KMS with explicit key * bucket in ireland with default encryption of

[GitHub] [hadoop] steveloughran opened a new pull request #1843: HADOOP-16794. encryption over rename/copy

2020-02-12 Thread GitBox
steveloughran opened a new pull request #1843: HADOOP-16794. encryption over rename/copy URL: https://github.com/apache/hadoop/pull/1843 Patch atop Mukund's #1823 patch returns to copying sse algorithm header, but then * extracts full KMS settings from src and sets on request

[GitHub] [hadoop] bgaborg commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets

2020-02-12 Thread GitBox
bgaborg commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets URL: https://github.com/apache/hadoop/pull/1840#discussion_r378226384 ## File path:

[GitHub] [hadoop] bgaborg commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets

2020-02-12 Thread GitBox
bgaborg commented on a change in pull request #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets URL: https://github.com/apache/hadoop/pull/1840#discussion_r378223217 ## File path:

[jira] [Commented] (HADOOP-16856) cmake is missing in the CentOS 8 section of BUILDING.txt

2020-02-12 Thread Hudson (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035331#comment-17035331 ] Hudson commented on HADOOP-16856: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17944 (See

[jira] [Assigned] (HADOOP-15964) Add S3A support for Async Scatter/Gather IO

2020-02-12 Thread Gabor Bota (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-15964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota reassigned HADOOP-15964: --- Assignee: Gabor Bota > Add S3A support for Async Scatter/Gather IO >

[jira] [Updated] (HADOOP-16856) cmake is missing in the CentOS 8 section of BUILDING.txt

2020-02-12 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16856: -- Fix Version/s: 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed

[GitHub] [hadoop] iwasakims merged pull request #1841: HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt.

2020-02-12 Thread GitBox
iwasakims merged pull request #1841: HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt. URL: https://github.com/apache/hadoop/pull/1841 This is an automated message from the Apache Git Service. To

[GitHub] [hadoop] hadoop-yetus commented on issue #1842: Hadoop 16730: Add Authorizer Interface

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1842: Hadoop 16730: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842#issuecomment-585164063 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:|

[GitHub] [hadoop] steveloughran commented on issue #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets

2020-02-12 Thread GitBox
steveloughran commented on issue #1840: HADOOP-16853. ITestS3GuardOutOfBandOperations failing on versioned S3 buckets URL: https://github.com/apache/hadoop/pull/1840#issuecomment-585147169 > `bytes = readOpenFileAPI(rawFS, testFilePath, length, null);` should still fail right? Do you

[GitHub] [hadoop] hadoop-yetus removed a comment on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
hadoop-yetus removed a comment on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#issuecomment-584831964 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] mukund-thakur removed a comment on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
mukund-thakur removed a comment on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#issuecomment-584804743 > if closing the `fs` value triggers failures in superclass cleanup, then you are sharing an FS instance

[GitHub] [hadoop] snvijaya opened a new pull request #1842: Hadoop 16730: Add Authorizer Interface

2020-02-12 Thread GitBox
snvijaya opened a new pull request #1842: Hadoop 16730: Add Authorizer Interface URL: https://github.com/apache/hadoop/pull/1842 Enabling Authorizer Interface as a plugin This is an automated message from the Apache Git

[GitHub] [hadoop] hadoop-yetus commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#issuecomment-585103560 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] hadoop-yetus commented on issue #1790: [HADOOP-16818] ABFS: Combine append+flush calls for blockblob & appendblob

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1790: [HADOOP-16818] ABFS: Combine append+flush calls for blockblob & appendblob URL: https://github.com/apache/hadoop/pull/1790#issuecomment-585102571 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] hadoop-yetus commented on issue #1841: HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt.

2020-02-12 Thread GitBox
hadoop-yetus commented on issue #1841: HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt. URL: https://github.com/apache/hadoop/pull/1841#issuecomment-585095438 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment |

[GitHub] [hadoop] mukund-thakur commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

2020-02-12 Thread GitBox
mukund-thakur commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1838#issuecomment-585086587 All review comments addressed. This is an

[jira] [Updated] (HADOOP-16856) cmake is missing in the CentOS 8 section of BUILDING.txt

2020-02-12 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-16856: -- Status: Patch Available (was: Open) > cmake is missing in the CentOS 8 section of

[GitHub] [hadoop] iwasakims opened a new pull request #1841: HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt.

2020-02-12 Thread GitBox
iwasakims opened a new pull request #1841: HADOOP-16856. cmake is missing in the CentOS 8 section of BUILDING.txt. URL: https://github.com/apache/hadoop/pull/1841 This is an automated message from the Apache Git Service. To

[jira] [Commented] (HADOOP-16856) cmake is missing in the CentOS 8 section of BUILDING.txt

2020-02-12 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035133#comment-17035133 ] Masatake Iwasaki commented on HADOOP-16856: --- [~aajisaka] you are right. I will submit a PR.

[jira] [Assigned] (HADOOP-16856) cmake is missing in the CentOS 8 section of BUILDING.txt

2020-02-12 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-16856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki reassigned HADOOP-16856: - Assignee: Masatake Iwasaki > cmake is missing in the CentOS 8 section of