[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[ https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=323153=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323153 ] ASF GitHub Bot logged work on HDDS-2198: Author: ASF GitHub Bot Created on: 04/Oct/19 02:51 Start Date: 04/Oct/19 02:51 Worklog Time Spent: 10m Work Description: nandakumar131 commented on pull request #1540: HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. URL: https://github.com/apache/hadoop/pull/1540 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 323153) Time Spent: 1h 20m (was: 1h 10m) > SCM should not consider containers in CLOSING state to come out of safemode > --- > > Key: HDDS-2198 > URL: https://issues.apache.org/jira/browse/HDDS-2198 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Reporter: Nilotpal Nandi >Assignee: Nanda kumar >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > There are cases where SCM can be stuck in safemode for ever if it considers > containers in CLOSING state for coming out of safemode > * If there are 5 containers in OPEN state inside SCM > * Out of 5, 3 containers are created in datanodes by the client. > * 2 containers are yet to be created in datanodes > * Due to some pipeline issue, pipeline close action is sent. > * All 5 container's state are changed from OPEN to CLOSING in SCM. > * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the > datanodes closes those containers. > * 2 of the containers are still in CLOSING state. > * SCM is restarted. > * SCM will never gets container reports for the containers which were in > CLOSING state as those containers were never created in datanodes. > * SCM will remain in safemode. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[ https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=323152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323152 ] ASF GitHub Bot logged work on HDDS-2198: Author: ASF GitHub Bot Created on: 04/Oct/19 02:50 Start Date: 04/Oct/19 02:50 Worklog Time Spent: 10m Work Description: nandakumar131 commented on issue #1540: HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. URL: https://github.com/apache/hadoop/pull/1540#issuecomment-538206445 Failures are not related to this change. I will merge this shortly. Thanks @bharatviswa504 for the review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 323152) Time Spent: 1h 10m (was: 1h) > SCM should not consider containers in CLOSING state to come out of safemode > --- > > Key: HDDS-2198 > URL: https://issues.apache.org/jira/browse/HDDS-2198 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Reporter: Nilotpal Nandi >Assignee: Nanda kumar >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > There are cases where SCM can be stuck in safemode for ever if it considers > containers in CLOSING state for coming out of safemode > * If there are 5 containers in OPEN state inside SCM > * Out of 5, 3 containers are created in datanodes by the client. > * 2 containers are yet to be created in datanodes > * Due to some pipeline issue, pipeline close action is sent. > * All 5 container's state are changed from OPEN to CLOSING in SCM. > * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the > datanodes closes those containers. > * 2 of the containers are still in CLOSING state. > * SCM is restarted. > * SCM will never gets container reports for the containers which were in > CLOSING state as those containers were never created in datanodes. > * SCM will remain in safemode. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[ https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=322185=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-322185 ] ASF GitHub Bot logged work on HDDS-2198: Author: ASF GitHub Bot Created on: 02/Oct/19 19:39 Start Date: 02/Oct/19 19:39 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1540: HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. URL: https://github.com/apache/hadoop/pull/1540#issuecomment-537647448 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 32 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 33 | hadoop-ozone in trunk failed. | | -1 | compile | 22 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 60 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 851 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in trunk failed. | | -1 | javadoc | 21 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 967 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 46 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 36 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 36 | hadoop-ozone in the patch failed. | | -1 | compile | 25 | hadoop-hdds in the patch failed. | | -1 | compile | 19 | hadoop-ozone in the patch failed. | | -1 | javac | 25 | hadoop-hdds in the patch failed. | | -1 | javac | 19 | hadoop-ozone in the patch failed. | | +1 | checkstyle | 57 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | shadedclient | 721 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in the patch failed. | | -1 | javadoc | 20 | hadoop-ozone in the patch failed. | | -1 | findbugs | 31 | hadoop-hdds in the patch failed. | | -1 | findbugs | 21 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 28 | hadoop-hdds in the patch failed. | | -1 | unit | 26 | hadoop-ozone in the patch failed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 2370 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1540 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e6961ad387b9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / e8ae632 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/2/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile |
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[
https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=322152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-322152
]
ASF GitHub Bot logged work on HDDS-2198:
Author: ASF GitHub Bot
Created on: 02/Oct/19 18:54
Start Date: 02/Oct/19 18:54
Worklog Time Spent: 10m
Work Description: nandakumar131 commented on pull request #1540:
HDDS-2198. SCM should not consider containers in CLOSING state to come out of
safemode.
URL: https://github.com/apache/hadoop/pull/1540#discussion_r330715763
##
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/ContainerSafeModeRule.java
##
@@ -63,19 +64,17 @@ public ContainerSafeModeRule(String ruleName, EventQueue
eventQueue,
" value should be >= 0.0 and <= 1.0");
containerMap = new ConcurrentHashMap<>();
-if(containers != null) {
- containers.forEach(c -> {
-// TODO: There can be containers in OPEN state which were never
-// created by the client. We are not considering these containers for
-// now. These containers can be handled by tracking pipelines.
-if (c != null && c.getState() != null &&
-!c.getState().equals(HddsProtos.LifeCycleState.OPEN)) {
- containerMap.put(c.getContainerID(), c);
-}
- });
- maxContainer = containerMap.size();
-}
-
+containers.forEach(container -> {
Review comment:
`ContainerManager.getContainers()` doesn't return null, it returns empty
list if there are no containers.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 322152)
Time Spent: 50m (was: 40m)
> SCM should not consider containers in CLOSING state to come out of safemode
> ---
>
> Key: HDDS-2198
> URL: https://issues.apache.org/jira/browse/HDDS-2198
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
>Reporter: Nilotpal Nandi
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> There are cases where SCM can be stuck in safemode for ever if it considers
> containers in CLOSING state for coming out of safemode
> * If there are 5 containers in OPEN state inside SCM
> * Out of 5, 3 containers are created in datanodes by the client.
> * 2 containers are yet to be created in datanodes
> * Due to some pipeline issue, pipeline close action is sent.
> * All 5 container's state are changed from OPEN to CLOSING in SCM.
> * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the
> datanodes closes those containers.
> * 2 of the containers are still in CLOSING state.
> * SCM is restarted.
> * SCM will never gets container reports for the containers which were in
> CLOSING state as those containers were never created in datanodes.
> * SCM will remain in safemode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[
https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=321465=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321465
]
ASF GitHub Bot logged work on HDDS-2198:
Author: ASF GitHub Bot
Created on: 01/Oct/19 19:25
Start Date: 01/Oct/19 19:25
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1540:
HDDS-2198. SCM should not consider containers in CLOSING state to come out of
safemode.
URL: https://github.com/apache/hadoop/pull/1540#discussion_r330232887
##
File path:
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/safemode/TestSCMSafeModeManager.java
##
@@ -85,7 +86,8 @@ public void testSafeModeState() throws Exception {
@Test
public void testSafeModeStateWithNullContainers() {
-new SCMSafeModeManager(config, null, null, queue);
+new SCMSafeModeManager(config, Collections.emptyList(),
Review comment:
Same as above.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 321465)
Time Spent: 40m (was: 0.5h)
> SCM should not consider containers in CLOSING state to come out of safemode
> ---
>
> Key: HDDS-2198
> URL: https://issues.apache.org/jira/browse/HDDS-2198
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
>Reporter: Nilotpal Nandi
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> There are cases where SCM can be stuck in safemode for ever if it considers
> containers in CLOSING state for coming out of safemode
> * If there are 5 containers in OPEN state inside SCM
> * Out of 5, 3 containers are created in datanodes by the client.
> * 2 containers are yet to be created in datanodes
> * Due to some pipeline issue, pipeline close action is sent.
> * All 5 container's state are changed from OPEN to CLOSING in SCM.
> * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the
> datanodes closes those containers.
> * 2 of the containers are still in CLOSING state.
> * SCM is restarted.
> * SCM will never gets container reports for the containers which were in
> CLOSING state as those containers were never created in datanodes.
> * SCM will remain in safemode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[
https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=321464=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321464
]
ASF GitHub Bot logged work on HDDS-2198:
Author: ASF GitHub Bot
Created on: 01/Oct/19 19:25
Start Date: 01/Oct/19 19:25
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1540:
HDDS-2198. SCM should not consider containers in CLOSING state to come out of
safemode.
URL: https://github.com/apache/hadoop/pull/1540#discussion_r330232789
##
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/safemode/ContainerSafeModeRule.java
##
@@ -63,19 +64,17 @@ public ContainerSafeModeRule(String ruleName, EventQueue
eventQueue,
" value should be >= 0.0 and <= 1.0");
containerMap = new ConcurrentHashMap<>();
-if(containers != null) {
- containers.forEach(c -> {
-// TODO: There can be containers in OPEN state which were never
-// created by the client. We are not considering these containers for
-// now. These containers can be handled by tracking pipelines.
-if (c != null && c.getState() != null &&
-!c.getState().equals(HddsProtos.LifeCycleState.OPEN)) {
- containerMap.put(c.getContainerID(), c);
-}
- });
- maxContainer = containerMap.size();
-}
-
+containers.forEach(container -> {
Review comment:
Why containers!= null condition is removed?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 321464)
Time Spent: 0.5h (was: 20m)
> SCM should not consider containers in CLOSING state to come out of safemode
> ---
>
> Key: HDDS-2198
> URL: https://issues.apache.org/jira/browse/HDDS-2198
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
>Reporter: Nilotpal Nandi
>Assignee: Nanda kumar
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> There are cases where SCM can be stuck in safemode for ever if it considers
> containers in CLOSING state for coming out of safemode
> * If there are 5 containers in OPEN state inside SCM
> * Out of 5, 3 containers are created in datanodes by the client.
> * 2 containers are yet to be created in datanodes
> * Due to some pipeline issue, pipeline close action is sent.
> * All 5 container's state are changed from OPEN to CLOSING in SCM.
> * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the
> datanodes closes those containers.
> * 2 of the containers are still in CLOSING state.
> * SCM is restarted.
> * SCM will never gets container reports for the containers which were in
> CLOSING state as those containers were never created in datanodes.
> * SCM will remain in safemode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[ https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=319768=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319768 ] ASF GitHub Bot logged work on HDDS-2198: Author: ASF GitHub Bot Created on: 27/Sep/19 19:31 Start Date: 27/Sep/19 19:31 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1540: HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. URL: https://github.com/apache/hadoop/pull/1540#issuecomment-536069314 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 50 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | -1 | mvninstall | 34 | hadoop-hdds in trunk failed. | | -1 | mvninstall | 35 | hadoop-ozone in trunk failed. | | -1 | compile | 24 | hadoop-hdds in trunk failed. | | -1 | compile | 15 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 66 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1062 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 24 | hadoop-hdds in trunk failed. | | -1 | javadoc | 20 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 1178 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 46 | hadoop-hdds in trunk failed. | | -1 | findbugs | 21 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 38 | hadoop-hdds in the patch failed. | | -1 | mvninstall | 31 | hadoop-ozone in the patch failed. | | -1 | compile | 26 | hadoop-hdds in the patch failed. | | -1 | compile | 18 | hadoop-ozone in the patch failed. | | -1 | javac | 26 | hadoop-hdds in the patch failed. | | -1 | javac | 18 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 29 | hadoop-hdds: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 877 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 22 | hadoop-hdds in the patch failed. | | -1 | javadoc | 19 | hadoop-ozone in the patch failed. | | -1 | findbugs | 33 | hadoop-hdds in the patch failed. | | -1 | findbugs | 19 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 26 | hadoop-hdds in the patch failed. | | -1 | unit | 23 | hadoop-ozone in the patch failed. | | +1 | asflicense | 33 | The patch does not generate ASF License warnings. | | | | 2753 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1540 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 4cf6e1cb6b3a 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / ce58c05 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-compile-hadoop-hdds.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-findbugs-hadoop-hdds.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/patch-mvninstall-hadoop-hdds.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1540/1/artifact/out/patch-mvninstall-hadoop-ozone.txt | |
[jira] [Work logged] (HDDS-2198) SCM should not consider containers in CLOSING state to come out of safemode
[ https://issues.apache.org/jira/browse/HDDS-2198?focusedWorklogId=319744=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319744 ] ASF GitHub Bot logged work on HDDS-2198: Author: ASF GitHub Bot Created on: 27/Sep/19 18:44 Start Date: 27/Sep/19 18:44 Worklog Time Spent: 10m Work Description: nandakumar131 commented on pull request #1540: HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. URL: https://github.com/apache/hadoop/pull/1540 There are cases where SCM can be stuck in safemode for ever if it considers containers in CLOSING state for coming out of safemode * If there are 5 containers in OPEN state inside SCM * Out of 5, 3 containers are created in datanodes by the client. * 2 containers are yet to be created in datanodes * Due to some pipeline issue, pipeline close action is sent. * All 5 container's state are changed from OPEN to CLOSING in SCM. * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the datanodes closes those containers. * 2 of the containers are still in CLOSING state. * SCM is restarted. * SCM will never gets container reports for the containers which were in CLOSING state as those containers were never created in datanodes. * SCM will remain in safemode. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 319744) Remaining Estimate: 0h Time Spent: 10m > SCM should not consider containers in CLOSING state to come out of safemode > --- > > Key: HDDS-2198 > URL: https://issues.apache.org/jira/browse/HDDS-2198 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Reporter: Nilotpal Nandi >Assignee: Nanda kumar >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There are cases where SCM can be stuck in safemode for ever if it considers > containers in CLOSING state for coming out of safemode > * If there are 5 containers in OPEN state inside SCM > * Out of 5, 3 containers are created in datanodes by the client. > * 2 containers are yet to be created in datanodes > * Due to some pipeline issue, pipeline close action is sent. > * All 5 container's state are changed from OPEN to CLOSING in SCM. > * Eventually , 3 container's state moves from CLOSING to CLOSED in SCM as the > datanodes closes those containers. > * 2 of the containers are still in CLOSING state. > * SCM is restarted. > * SCM will never gets container reports for the containers which were in > CLOSING state as those containers were never created in datanodes. > * SCM will remain in safemode. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
