刘珍 created IOTDB-4204:
-------------------------
Summary: RatisRequestFailedException: Ratis request failed
Key: IOTDB-4204
URL: https://issues.apache.org/jira/browse/IOTDB-4204
Project: Apache IoTDB
Issue Type: Bug
Components: mpp-cluster
Reporter: 刘珍
master_0822_700b135，schema/data region均设置RatisConsensus
3副本3C9D，故障节点启动后，写入数据报错（部分写入失败）：
2022-08-22 15:32:09,652 [pool-17-IoTDB-InternalServiceRPC-Client-24] ERROR
o.a.i.d.s.t.i.DataNodeInternalRPCServiceImpl:248 - Something wrong happened
while calling consensus layer's write API.
org.apache.iotdb.consensus.exception.RatisRequestFailedException: Ratis request
failed
at
org.apache.iotdb.consensus.ratis.RatisConsensus.write(RatisConsensus.java:200)
at
org.apache.iotdb.db.service.thrift.impl.DataNodeInternalRPCServiceImpl.sendPlanNode(DataNodeInternalRPCServiceImpl.java:227)
at
org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$sendPlanNode.getResult(IDataNodeRPCService.java:2101)
at
org.apache.iotdb.mpp.rpc.thrift.IDataNodeRPCService$Processor$sendPlanNode.getResult(IDataNodeRPCService.java:2081)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.ratis.protocol.exceptions.RaftRetryFailureException:
Failed
RaftClientRequest:client-CBD43621169C->172.20.70.18_40010@group-000100000008,
cid=466, seq=0, RW, org.apache.iotdb.consensus.
ratis.RequestMessage@474cae0e for 2 attempts with
org.apache.iotdb.consensus.ratis.RatisClient$RatisRetryPolicy@138967d4
at
org.apache.ratis.client.impl.RaftClientImpl.noMoreRetries(RaftClientImpl.java:295)
at
org.apache.ratis.client.impl.BlockingImpl.sendRequestWithRetry(BlockingImpl.java:117)
at org.apache.ratis.client.impl.BlockingImpl.send(BlockingImpl.java:84)
at org.apache.ratis.client.impl.BlockingImpl.send(BlockingImpl.java:58)
at
org.apache.iotdb.consensus.ratis.RatisConsensus.write(RatisConsensus.java:194)
... 9 common frames omitted
Caused by: java.io.IOException:
org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException: UNAVAILABLE: io
exception
at org.apache.ratis.grpc.GrpcUtil.unwrapException(GrpcUtil.java:92)
at org.apache.ratis.grpc.GrpcUtil.unwrapIOException(GrpcUtil.java:152)
at
org.apache.ratis.grpc.client.GrpcClientRpc$1.onError(GrpcClientRpc.java:156)
at
org.apache.ratis.thirdparty.io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:479)
at
org.apache.ratis.thirdparty.io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at
org.apache.ratis.thirdparty.io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at
org.apache.ratis.grpc.metrics.intercept.client.MetricClientCallListener.onClose(MetricClientCallListener.java:47)
at
org.apache.ratis.thirdparty.io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463)
at
org.apache.ratis.thirdparty.io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427)
at
org.apache.ratis.thirdparty.io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at
org.apache.ratis.thirdparty.io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at
org.apache.ratis.thirdparty.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
org.apache.ratis.thirdparty.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
... 3 common frames omitted
Caused by: org.apache.ratis.thirdparty.io.grpc.StatusRuntimeException:
UNAVAILABLE: io exception
at
org.apache.ratis.thirdparty.io.grpc.Status.asRuntimeException(Status.java:535)
... 16 common frames omitted
复现流程：
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
