Nilotpal Nandi created HDDS-1289:
------------------------------------
Summary: get Key failed on SCM restart
Key: HDDS-1289
URL: https://issues.apache.org/jira/browse/HDDS-1289
Project: Hadoop Distributed Data Store
Issue Type: Bug
Reporter: Nilotpal Nandi
Attachments: hadoop-hdfs-scm-ctr-e139-1542663976389-86524-01-000003.log
Seeing ContainerNotFoundException in scm log when get key operation tried after
scm restart.
scm.log:
[^hadoop-hdfs-scm-ctr-e139-1542663976389-86524-01-000003.log]
{noformat}
2019-03-13 17:00:54,348 ERROR container.ContainerReportHandler
(ContainerReportHandler.java:processContainerReplicas(173)) - Received
container report for an unknown container 22 from datanode
80f046cb-6fe2-4a05-bb67-9bf46f48723b{ip: 172.27.69.155, host:
ctr-e139-1542663976389-86524-01-000005.hwx.site} {}
org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #22 at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543)
at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230)
at
org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565)
at
org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393)
at
org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51)
at
org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2019-03-13 17:00:54,349 ERROR
container.ContainerReportHandler
(ContainerReportHandler.java:processContainerReplicas(173)) - Received
container report for an unknown container 23 from datanode
80f046cb-6fe2-4a05-bb67-9bf46f48723b{ip: 172.27.69.155, host:
ctr-e139-1542663976389-86524-01-000005.hwx.site} {}
org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #23 at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543)
at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230)
at
org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565)
at
org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393)
at
org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51)
at
org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) 2019-03-13 17:01:24,230 ERROR
container.ContainerReportHandler
(ContainerReportHandler.java:processContainerReplicas(173)) - Received
container report for an unknown container 22 from datanode
076fd0d8-ab5f-4fbe-ad10-b71a1ccb19bf{ip: 172.27.39.88, host:
ctr-e139-1542663976389-86524-01-000004.hwx.site} {}
org.apache.hadoop.hdds.scm.container.ContainerNotFoundException: #22 at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.checkIfContainerExist(ContainerStateMap.java:543)
at
org.apache.hadoop.hdds.scm.container.states.ContainerStateMap.updateContainerReplica(ContainerStateMap.java:230)
at
org.apache.hadoop.hdds.scm.container.ContainerStateManager.updateContainerReplica(ContainerStateManager.java:565)
at
org.apache.hadoop.hdds.scm.container.SCMContainerManager.updateContainerReplica(SCMContainerManager.java:393)
at
org.apache.hadoop.hdds.scm.container.ReportHandlerHelper.processContainerReplica(ReportHandlerHelper.java:74)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.processContainerReplicas(ContainerReportHandler.java:159)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:110)
at
org.apache.hadoop.hdds.scm.container.ContainerReportHandler.onMessage(ContainerReportHandler.java:51)
at
org.apache.hadoop.hdds.server.events.SingleThreadExecutor.lambda$onMessage$1(SingleThreadExecutor.java:85)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
