刘珍 created IOTDB-3584:
-------------------------
Summary: Write data for a long time (19
hours),IP_40010@group-000100000004 is in LEADER state but not ready yet
Key: IOTDB-3584
URL: https://issues.apache.org/jira/browse/IOTDB-3584
Project: Apache IoTDB
Issue Type: Bug
Components: mpp-cluster
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: Song Ziyang
Attachments: image-2022-06-21-15-50-54-369.png
master_0620_3c88dfb
benchmark写入19小时,bm连接的datanode :
2022-06-21 11:46:05,486 [192.168.130.3_40010@group-000100000006-FollowerState]
INFO o.a.r.s.i.FollowerState:108 -
192.168.130.3_40010@group-000100000006-FollowerState: Skipping leader election
since it stepped down recently (elapsed = 14682ms < waitTime = 30s)
2022-06-21 11:46:05,692 [null-request--thread1] INFO
o.a.r.g.s.GrpcClientProtocolService$UnorderedRequestStreamObserver:284 - Failed
RaftClientRequest:client-6B295D30FFBC->192.168.130.3_40010@group-000100000004,
cid=201437, seq=0, RW, Message:000d00000018726f6f74...(size=41220),
reply=RaftClientReply:client-6B295D30FFBC->192.168.130.3_40010@group-000100000004,
cid=201437, FAILED
org.apache.ratis.protocol.exceptions.LeaderNotReadyException:
192.168.130.3_40010@group-000100000004{color:#DE350B}* is in LEADER state but
not ready yet., *{color}logIndex=0, commits[192.168.130.3_40010:c400003,
192.168.130.4_40010:c400003, 192.168.130.5_40010:c400003]
{color:#DE350B}*metric监控的insertTablet延迟陡升*{color}
!image-2022-06-21-15-50-54-369.png!
FollowerState线程一直销毁,创建,销毁,且其cpu占用高。
复现流程
1. 192.168.130.3/4/5 16C32G
data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
schema_region_consensus_protocol_class=org.apache.iotdb.consensus.standalone.StandAloneConsensus
schema_replication_factor=1
data_replication_factor=3
数据库路径:
/data3/cluster_test/master_0620_3c88dfb
2. benchmark 配置见附件
benchmark在192.168.130.2 /home/benchmark/bm_0527_e7b23aa
--
This message was sent by Atlassian Jira
(v8.20.7#820007)