刘珍 created IOTDB-3965:
-------------------------

             Summary: [ratis md5 ] ERROR 
o.a.i.c.r.FileInfoWithDelayedMd5Computing:64 - compute file digest for 
.tsfile.resource.md5.tmp failed due to {}
                 Key: IOTDB-3965
                 URL: https://issues.apache.org/jira/browse/IOTDB-3965
             Project: Apache IoTDB
          Issue Type: Bug
          Components: mpp-cluster
    Affects Versions: 0.14.0-SNAPSHOT
            Reporter: 刘珍
            Assignee: Song Ziyang


RatisConsensus 3副本3C5D
缩容1个datanode(包含2个dataregion),缩容节点的datanode 报错,add new peer 失败:

2022-07-26 15:05:47,271 [ForkJoinPool.commonPool-worker-13] ERROR 
o.a.i.c.r.FileInfoWithDelayedMd5Computing:64 - compute file digest for 
/data2/cluster_test/shrink_0726_ratis/datanode/sbin/../data/consensus/data_region/47474747-4747-4747-4747-000100000004/sm/1_400002/sequence/root.test.g0_0/4/0/1658816468388-9-0-0.tsfile.md5.tmp
 failed due to {}
java.io.FileNotFoundException: 
/data2/cluster_test/shrink_0726_ratis/datanode/sbin/../data/consensus/data_region/47474747-4747-4747-4747-000100000004/sm/1_400002/sequence/root.test.g0_0/4/0/1658816468388-9-0-0.tsfile.md5.tmp
 (No such file or directory)
        at java.io.FileInputStream.open0(Native Method)
        at java.io.FileInputStream.open(FileInputStream.java:195)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at 
org.apache.ratis.util.MD5FileUtil.computeMd5ForFile(MD5FileUtil.java:125)
        at 
org.apache.iotdb.consensus.ratis.FileInfoWithDelayedMd5Computing.lambda$new$0(FileInfoWithDelayedMd5Computing.java:61)
        at 
java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
        at 
java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at 
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703)
        at 
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:172)

2022-07-26 14:54:48,514 [pool-16-IoTDB-Region-Migrate-Pool-1] ERROR 
o.a.i.d.s.RegionMigrateService$RegionMigrateTask:348 - add new peer 
TEndPoint(ip:192.168.130.4, port:40010) for region DataRegion[5] failed, resp: 
ConsensusGenericResponse{success=false} 
exception=org.apache.iotdb.consensus.exception.RatisRequestFailedException: 
Ratis request failed

复现流程:
1. 192.168.130.1/2/3/4/5

schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
data_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
schema_replication_factor=3
data_replication_factor=3

confignode:
MAX_HEAP_SIZE="4G"

datanode:
MAX_HEAP_SIZE="16G"
max_waiting_time_when_insert_blocked=3600000
query_timeout_threshold=36000000

2. benchmark 在ip2

配置文件见附件

3. 缩容ip5 ,包含2个dataregion
./sbin/remove-datanode.sh  192.168.130.5:6667

ip5有1个dataregion做了1次snapshot
有1个dataregion leader 做了1次snapshot,ip5同步慢,还没有发生snapshot

查看ip5的log。



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to