[
https://issues.apache.org/jira/browse/HDDS-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nanda kumar updated HDDS-1126:
------------------------------
Attachment: HDDS-1126.000.patch
> datanode is trying to qausi-close a container which is already closed
> ---------------------------------------------------------------------
>
> Key: HDDS-1126
> URL: https://issues.apache.org/jira/browse/HDDS-1126
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Reporter: Nilotpal Nandi
> Assignee: Nanda kumar
> Priority: Major
> Attachments: HDDS-1126.000.patch
>
>
> steps taken :
> --------------------
> # created 12 datanodes cluster and running workload on all the nodes
> # running failure injection/restart on 1 datanode at a time periodically and
> randomly.
>
> Error seen in ozone.log :
> --------------------------
>
> {noformat}
> 2019-02-18 06:06:32,780 [Datanode State Machine Thread - 0] DEBUG
> (DatanodeStateMachine.java:176) - Executing cycle Number : 30
> 2019-02-18 06:06:32,784 [Command processor thread] DEBUG
> (CloseContainerCommandHandler.java:71) - Processing Close Container command.
> 2019-02-18 06:06:32,785 [Datanode State Machine Thread - 0] DEBUG
> (DatanodeStateMachine.java:176) - Executing cycle Number : 31
> 2019-02-18 06:06:32,785 [Command processor thread] ERROR
> (CloseContainerCommandHandler.java:118) - Can't close container #37
> org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException:
> Cannot quasi close container #37 while in CLOSED state.
> at
> org.apache.hadoop.ozone.container.keyvalue.KeyValueHandler.quasiCloseContainer(KeyValueHandler.java:903)
> at
> org.apache.hadoop.ozone.container.ozoneimpl.ContainerController.quasiCloseContainer(ContainerController.java:93)
> at
> org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CloseContainerCommandHandler.handle(CloseContainerCommandHandler.java:110)
> at
> org.apache.hadoop.ozone.container.common.statemachine.commandhandler.CommandDispatcher.handle(CommandDispatcher.java:93)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$initCommandHandlerThread$1(DatanodeStateMachine.java:413)
> at java.lang.Thread.run(Thread.java:748)
> 2019-02-18 06:06:32,785 [Command processor thread] DEBUG
> (CloseContainerCommandHandler.java:71) - Processing Close Container command.
> 2019-02-18 06:06:32,788 [Command processor thread] DEBUG
> (CloseContainerCommandHandler.java:71) - Processing Close Container command.
> 2019-02-18 06:06:32,788 [Datanode State Machine Thread - 0] DEBUG
> (DatanodeStateMachine.java:176) - Executing cycle Number : 32
> 2019-02-18 06:06:34,430 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:36,608 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:38,876 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:41,084 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:43,297 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:45,469 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:47,684 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:49,958 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:52,124 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:54,344 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:56,499 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:06:58,764 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:07:00,969 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:07:02,788 [Datanode State Machine Thread - 0] DEBUG
> (DatanodeStateMachine.java:176) - Executing cycle Number : 33
> 2019-02-18 06:07:03,240 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
> 2019-02-18 06:07:05,486 [main] DEBUG (OzoneClientFactory.java:287) - Using
> org.apache.hadoop.ozone.client.rpc.RpcClient as client protocol.
>
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
