Namit Maheshwari created HDDS-689: ------------------------------------- Summary: Datanode shuts down on restart Key: HDDS-689 URL: https://issues.apache.org/jira/browse/HDDS-689 Project: Hadoop Distributed Data Store Issue Type: Bug Reporter: Namit Maheshwari
Restart all the 8 DNs in the cluster. 2 of them shuts down as below {code:java} 2018-10-18 01:10:57,102 ERROR impl.StateMachineUpdater (ExitUtils.java:terminate(86)) - Terminating with exit status 2: StateMachineUpdater-69d15283-4e2e-4c30-a028-f2bad0f83cc1: the StateMachineUpdater hits Throwable java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.CompletableFuture$AsyncSupply@7ee4907b rejected from java.util.concurrent.ThreadPoolExecutor@702371d7[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:668) at java.util.concurrent.CompletableFuture.asyncSupplyStage(CompletableFuture.java:1604) at java.util.concurrent.CompletableFuture.supplyAsync(CompletableFuture.java:1830) at org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine.applyTransaction(ContainerStateMachine.java:433) at org.apache.ratis.server.impl.RaftServerImpl.applyLogToStateMachine(RaftServerImpl.java:1093) at org.apache.ratis.server.impl.StateMachineUpdater.run(StateMachineUpdater.java:148) at java.lang.Thread.run(Thread.java:748) 2018-10-18 01:10:57,107 WARN fs.CachingGetSpaceUsed (CachingGetSpaceUsed.java:run(183)) - Thread Interrupted waiting to refresh disk information: sleep interrupted 2018-10-18 01:10:57,108 INFO datanode.DataNode (LogAdapter.java:info(51)) - SHUTDOWN_MSG: {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org