[jira] [Commented] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179374#comment-17179374 ] Glen Geng commented on HDDS-4128: - FYI [~nanda]  [~timmylicheng] related to  

[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4128: Description: Since the Scheduler in SCMPipelineManager that used to destroyPipeline is removed,

[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4128: Description: RATIS ONE Pipeline is closed but not removed when a datanode goes stale.   Since the

[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4128: Description: (was: When a node goes stale the pipelines in that node have to be closed and removed from

[jira] [Updated] (HDDS-4128) RATIS ONE Pipeline is closed but not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Glen Geng updated HDDS-4128: Summary: RATIS ONE Pipeline is closed but not removed when a datanode goes stale (was: CLONE - Pipeline is

[jira] [Created] (HDDS-4128) CLONE - Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
Glen Geng created HDDS-4128: --- Summary: CLONE - Pipeline is not removed when a datanode goes stale Key: HDDS-4128 URL: https://issues.apache.org/jira/browse/HDDS-4128 Project: Hadoop Distributed Data Store

[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179352#comment-17179352 ] Glen Geng commented on HDDS-4125: - RATIS THREE pipeline, if stay in closed state, should be removed by

[jira] [Assigned] (HDDS-4115) CLI command to show current SCM leader and follower status

2020-08-17 Thread Rui Wang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Wang reassigned HDDS-4115: -- Assignee: Rui Wang > CLI command to show current SCM leader and follower status >

[jira] [Commented] (HDDS-4116) SCM CLI command towards certain IP

2020-08-17 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179307#comment-17179307 ] Li Cheng commented on HDDS-4116: [~adoroszlai] The goal of this task is to enable admin cli to send scm

[jira] [Updated] (HDDS-3837) Add isLeader check for SCM state updates

2020-08-17 Thread Li Cheng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Cheng updated HDDS-3837: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Add isLeader check for SCM state updates

[GitHub] [hadoop-ozone] bharatviswa504 commented on a change in pull request #1335: HDDS-4127. Components with web interface should depend on hdds-docs.

2020-08-17 Thread GitBox
bharatviswa504 commented on a change in pull request #1335: URL: https://github.com/apache/hadoop-ozone/pull/1335#discussion_r471813458 ## File path: hadoop-ozone/ozone-manager/pom.xml ## @@ -70,6 +71,12 @@ https://maven.apache.org/xsd/maven-4.0.0.xsd;> bcprov-jdk15on

[jira] [Updated] (HDDS-4119) Improve performance of the BufferPool management of Ozone client

2020-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4119: - Labels: pull-request-available (was: ) > Improve performance of the BufferPool management of

[GitHub] [hadoop-ozone] elek opened a new pull request #1336: HDDS-4119. Improve performance of the BufferPool management of Ozone client

2020-08-17 Thread GitBox
elek opened a new pull request #1336: URL: https://github.com/apache/hadoop-ozone/pull/1336 ## What changes were proposed in this pull request? Teragen reported to be slow with low number of mappers compared to HDFS. In my test (one pipeline, 3 yarn nodes) 10 g teragen with

[jira] [Assigned] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar reassigned HDDS-4125: - Assignee: Glen Geng (was: Nanda kumar) > Pipeline is not removed when a datanode goes stale >

[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Nanda kumar (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179138#comment-17179138 ] Nanda kumar commented on HDDS-4125: --- It is not just one node pipeline, we should also handle three node

[GitHub] [hadoop-ozone] nandakumar131 commented on pull request #1145: HDDS-3895. Implement container related operations in ContainerManagerImpl

2020-08-17 Thread GitBox
nandakumar131 commented on pull request #1145: URL: https://github.com/apache/hadoop-ozone/pull/1145#issuecomment-675004188 It is not just about one node pipeline, we have the same issue in three node pipeline as well. If all the three nodes goes stale (because of restart), we will mark

[jira] [Updated] (HDDS-4127) Components with web interface should depend on hdds-docs

2020-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4127: - Labels: pull-request-available (was: ) > Components with web interface should depend on

[GitHub] [hadoop-ozone] elek opened a new pull request #1335: HDDS-4127. Components with web interface should depend on hdds-docs.

2020-08-17 Thread GitBox
elek opened a new pull request #1335: URL: https://github.com/apache/hadoop-ozone/pull/1335 ## What changes were proposed in this pull request? @ChenSammi reported the problem that the ozone-0.6.0 branch couldn't be compiled after changing the version to 1.0.0. ``` [INFO]

[jira] [Updated] (HDDS-4127) Components with web interface should depend on hdds-docs

2020-08-17 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-4127: -- Summary: Components with web interface should depend on hdds-docs (was: Component with web interface

[jira] [Created] (HDDS-4127) Component with web interface should depend on hdds-docs

2020-08-17 Thread Marton Elek (Jira)
Marton Elek created HDDS-4127: - Summary: Component with web interface should depend on hdds-docs Key: HDDS-4127 URL: https://issues.apache.org/jira/browse/HDDS-4127 Project: Hadoop Distributed Data Store

[jira] [Comment Edited] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178919#comment-17178919 ] Glen Geng edited comment on HDDS-4125 at 8/17/20, 11:32 AM: Since the thread

[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178919#comment-17178919 ] Glen Geng commented on HDDS-4125: - Since the thread pool that remove RATIS ONE pipeline is removed, we

[jira] [Commented] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread Glen Geng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178917#comment-17178917 ] Glen Geng commented on HDDS-4125: - Hi Nanda, I encountered this issue as well, and provide a fix. Please

[jira] [Updated] (HDDS-4125) Pipeline is not removed when a datanode goes stale

2020-08-17 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-4125: - Labels: pull-request-available (was: ) > Pipeline is not removed when a datanode goes stale >

[GitHub] [hadoop-ozone] GlenGeng opened a new pull request #1334: HDDS-4125: Pipeline is not removed when a datanode goes stale

2020-08-17 Thread GitBox
GlenGeng opened a new pull request #1334: URL: https://github.com/apache/hadoop-ozone/pull/1334 ## What changes were proposed in this pull request? StaleNodeHandler need Immediately remove RATIS ONE Pipeline, since scrubPipeline() only handle RATIS THREE Pipeline. ## What is

[GitHub] [hadoop-ozone] timmylicheng commented on pull request #1145: HDDS-3895. Implement container related operations in ContainerManagerImpl

2020-08-17 Thread GitBox
timmylicheng commented on pull request #1145: URL: https://github.com/apache/hadoop-ozone/pull/1145#issuecomment-674816824 @nandakumar131 Ratis One pipeline is not instantly removed in StaleNodeHandler. We will come up with a fix.

[jira] [Created] (HDDS-4126) Freon key generator should support >2g files

2020-08-17 Thread Marton Elek (Jira)
Marton Elek created HDDS-4126: - Summary: Freon key generator should support >2g files Key: HDDS-4126 URL: https://issues.apache.org/jira/browse/HDDS-4126 Project: Hadoop Distributed Data Store

[GitHub] [hadoop-ozone] fapifta commented on pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.

2020-08-17 Thread GitBox
fapifta commented on pull request #1322: URL: https://github.com/apache/hadoop-ozone/pull/1322#issuecomment-674766274 Hi @avijayanhwx, thank you for starting this work, and posting an inital version of the core code for the upgrade support. I have a few general questions and

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.

2020-08-17 Thread GitBox
fapifta commented on a change in pull request #1322: URL: https://github.com/apache/hadoop-ozone/pull/1322#discussion_r471346148 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/upgrade/OMLayoutFeatureCatalog.java ## @@ -0,0 +1,71 @@ +/** + *

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.

2020-08-17 Thread GitBox
fapifta commented on a change in pull request #1322: URL: https://github.com/apache/hadoop-ozone/pull/1322#discussion_r471332451 ## File path: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMVersionManager.java ## @@ -0,0 +1,63 @@ +/** + *

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #1322: HDDS-3829. Introduce Layout Feature interface in Ozone.

2020-08-17 Thread GitBox
fapifta commented on a change in pull request #1322: URL: https://github.com/apache/hadoop-ozone/pull/1322#discussion_r471313928 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/s3/security/package-info.java ## @@ -19,4 +19,4 @@ /**