[
https://issues.apache.org/jira/browse/IOTDB-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17556646#comment-17556646
]
任宇华 commented on IOTDB-3536:
----------------------------
Previously, flush requests were sent synchronously, and each send will lead to
blocking. Therefore, the failure of datanode connection may be due to too long
waiting for syncdatanodeclientpool
> [ flush ] ConfigNode : org.apache.thrift.TException: Error in calling method
> flush
> ----------------------------------------------------------------------------------
>
> Key: IOTDB-3536
> URL: https://issues.apache.org/jira/browse/IOTDB-3536
> Project: Apache IoTDB
> Issue Type: Bug
> Affects Versions: 0.14.0-SNAPSHOT
> Reporter: 刘珍
> Assignee: 任宇华
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-06-20-09-58-41-791.png,
> image-2022-06-20-09-59-46-382.png, image-2022-06-20-09-59-54-181.png
>
>
> master_0617_1840cd1
> 3节点3副本，写入数据，执行flush ：
> !image-2022-06-20-09-58-41-791.png!
> ConfigNode的日志 ：
> 2022-06-17 17:47:05,929 [pool-2-IoTDB-ConfigNodeRPC-Client-74] ERROR
> o.a.i.c.c.SyncDataNodeClientPool:135 - flush on DataNode
> TEndPoint(ip:192.168.10.68, port:9003) failed
> org.apache.thrift.TException: Error in calling method flush
> at
> org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.flush(<generated>)
> at
> org.apache.iotdb.confignode.client.SyncDataNodeClientPool.flush(SyncDataNodeClientPool.java:130)
> at
> org.apache.iotdb.confignode.service.thrift.ConfigNodeRPCServiceProcessor.flush(ConfigNodeRPCServiceProcessor.java:440)
> at
> org.apache.iotdb.confignode.rpc.thrift.ConfigIService$Processor$flush.getResult(ConfigIService.java:2361)
> at
> org.apache.iotdb.confignode.rpc.thrift.ConfigIService$Processor$flush.getResult(ConfigIService.java:2341)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.TException: Error in calling method recv_flush
> at
> org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.recv_flush(<generated>)
> at
> org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.flush(InternalService.java:591)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.CGLIB$flush$61(<generated>)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79$$FastClassByCGLIB$$d4c42083.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
> ... 11 common frames omitted
> Caused by: org.apache.thrift.TException: Error in calling method receiveBase
> at
> org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:94)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.receiveBase(<generated>)
> at
> org.apache.iotdb.mpp.rpc.thrift.InternalService$Client.recv_flush(InternalService.java:604)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.CGLIB$recv_flush$56(<generated>)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79$$FastClassByCGLIB$$d4c42083.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
> ... 17 common frames omitted
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.SocketTimeoutException: Read timed out
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:178)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109)
> at
> org.apache.iotdb.rpc.TElasticFramedTransport.readFrame(TElasticFramedTransport.java:112)
> at
> org.apache.iotdb.rpc.TElasticFramedTransport.read(TElasticFramedTransport.java:107)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:109)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:463)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:361)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:244)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79.CGLIB$receiveBase$64(<generated>)
> at
> org.apache.iotdb.commons.client.sync.SyncDataNodeInternalServiceClient$$EnhancerByCGLIB$$9d7b0c79$$FastClassByCGLIB$$d4c42083.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> org.apache.iotdb.commons.client.sync.SyncThriftClientWithErrorHandler.intercept(SyncThriftClientWithErrorHandler.java:55)
> ... 23 common frames omitted
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:176)
> ... 35 common frames omitted
> !image-2022-06-20-09-59-46-382.png!
> datanode的日志：
> !image-2022-06-20-09-59-54-181.png!
> 测试环境：
> 1. 192.168.10.62/66/68
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> schema_replication_factor=3
> data_replication_factor=3
> avg_series_point_number_threshold=1000
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
