[jira] [Updated] (HDDS-1561) Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[ https://issues.apache.org/jira/browse/HDDS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-1561: -- Fix Version/s: 0.5.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove > - > > Key: HDDS-1561 > URL: https://issues.apache.org/jira/browse/HDDS-1561 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode, SCM >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Lokesh Jain >Priority: Blocker > Labels: pull-request-available > Fix For: 0.5.0 > > Attachments: HDDS-1561.001.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Right now, if a pipeline is destroyed by SCM, all the container on the > pipeline are marked as quasi closed when datanode received close container > command. SCM while processing these containers reports, marks these > containers as closed once majority of the nodes are available. > This is however not a sufficient condition in cases where the raft log > directory is missing or corrupted. As the containers will not have all the > applied transaction. > To solve this problem, we should QUASI_CLOSE the containers in datanode as > part of ratis groupRemove. If a container is in OPEN state in datanode > without any active pipeline, it will be marked as Unhealthy while processing > close container command. > cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-1561) Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[ https://issues.apache.org/jira/browse/HDDS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-1561: -- Status: Patch Available (was: Open) > Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove > - > > Key: HDDS-1561 > URL: https://issues.apache.org/jira/browse/HDDS-1561 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode, SCM >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Lokesh Jain >Priority: Blocker > Labels: pull-request-available > Attachments: HDDS-1561.001.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Right now, if a pipeline is destroyed by SCM, all the container on the > pipeline are marked as quasi closed when datanode received close container > command. SCM while processing these containers reports, marks these > containers as closed once majority of the nodes are available. > This is however not a sufficient condition in cases where the raft log > directory is missing or corrupted. As the containers will not have all the > applied transaction. > To solve this problem, we should QUASI_CLOSE the containers in datanode as > part of ratis groupRemove. If a container is in OPEN state in datanode > without any active pipeline, it will be marked as Unhealthy while processing > close container command. > cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-1561) Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[ https://issues.apache.org/jira/browse/HDDS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-1561: - Labels: pull-request-available (was: ) > Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove > - > > Key: HDDS-1561 > URL: https://issues.apache.org/jira/browse/HDDS-1561 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode, SCM >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Lokesh Jain >Priority: Blocker > Labels: pull-request-available > Attachments: HDDS-1561.001.patch > > > Right now, if a pipeline is destroyed by SCM, all the container on the > pipeline are marked as quasi closed when datanode received close container > command. SCM while processing these containers reports, marks these > containers as closed once majority of the nodes are available. > This is however not a sufficient condition in cases where the raft log > directory is missing or corrupted. As the containers will not have all the > applied transaction. > To solve this problem, we should QUASI_CLOSE the containers in datanode as > part of ratis groupRemove. If a container is in OPEN state in datanode > without any active pipeline, it will be marked as Unhealthy while processing > close container command. > cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-1561) Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[ https://issues.apache.org/jira/browse/HDDS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lokesh Jain updated HDDS-1561: -- Attachment: HDDS-1561.001.patch > Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove > - > > Key: HDDS-1561 > URL: https://issues.apache.org/jira/browse/HDDS-1561 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode, SCM >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Lokesh Jain >Priority: Blocker > Attachments: HDDS-1561.001.patch > > > Right now, if a pipeline is destroyed by SCM, all the container on the > pipeline are marked as quasi closed when datanode received close container > command. SCM while processing these containers reports, marks these > containers as closed once majority of the nodes are available. > This is however not a sufficient condition in cases where the raft log > directory is missing or corrupted. As the containers will not have all the > applied transaction. > To solve this problem, we should QUASI_CLOSE the containers in datanode as > part of ratis groupRemove. If a container is in OPEN state in datanode > without any active pipeline, it will be marked as Unhealthy while processing > close container command. > cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-1561) Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[ https://issues.apache.org/jira/browse/HDDS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-1561: -- Description: Right now, if a pipeline is destroyed by SCM, all the container on the pipeline are marked as quasi closed when datanode received close container command. SCM while processing these containers reports, marks these containers as closed once majority of the nodes are available. This is however not a sufficient condition in cases where the raft log directory is missing or corrupted. As the containers will not have all the applied transaction. To solve this problem, we should QUASI_CLOSE the containers in datanode as part of ratis groupRemove. If a container is in OPEN state in datanode without any active pipeline, it will be marked as Unhealthy while processing close container command. cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] was: Right now, if a pipeline is destroyed by SCM, all the container on the pipeline are marked as quasi closed. SCM while processing these containers reports, marks these containers as closed once majority of the nodes are available. This is however not a sufficient condition in cases where the raft log directory is missing or corrupted. As the containers will not have all the applied transaction. To solve this problem, a new container replica state needs to be added to differentiate this from quasi closed containers. cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] > Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove > - > > Key: HDDS-1561 > URL: https://issues.apache.org/jira/browse/HDDS-1561 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode, SCM >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Nanda kumar >Priority: Blocker > > Right now, if a pipeline is destroyed by SCM, all the container on the > pipeline are marked as quasi closed when datanode received close container > command. SCM while processing these containers reports, marks these > containers as closed once majority of the nodes are available. > This is however not a sufficient condition in cases where the raft log > directory is missing or corrupted. As the containers will not have all the > applied transaction. > To solve this problem, we should QUASI_CLOSE the containers in datanode as > part of ratis groupRemove. If a container is in OPEN state in datanode > without any active pipeline, it will be marked as Unhealthy while processing > close container command. > cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-1561) Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove
[ https://issues.apache.org/jira/browse/HDDS-1561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nanda kumar updated HDDS-1561: -- Summary: Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove (was: Add a new replica state to identify containers which have not been able to apply all the transactions.) > Mark OPEN containers as QUASI_CLOSED as part of Ratis groupRemove > - > > Key: HDDS-1561 > URL: https://issues.apache.org/jira/browse/HDDS-1561 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode, SCM >Affects Versions: 0.3.0 >Reporter: Mukul Kumar Singh >Assignee: Nanda kumar >Priority: Blocker > > Right now, if a pipeline is destroyed by SCM, all the container on the > pipeline are marked as quasi closed. SCM while processing these containers > reports, marks these containers as closed once majority of the nodes are > available. > This is however not a sufficient condition in cases where the raft log > directory is missing or corrupted. As the containers will not have all the > applied transaction. To solve this problem, > a new container replica state needs to be added to differentiate this from > quasi closed containers. > cc [~jnp], [~shashikant], [~sdeka], [~nandakumar131] -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
