Song Ziyang created IOTDB-4688:
----------------------------------
Summary: Streaming MD5 replace blocking MD5
Key: IOTDB-4688
URL: https://issues.apache.org/jira/browse/IOTDB-4688
Project: Apache IoTDB
Issue Type: Sub-task
Reporter: Song Ziyang
Currently in InstallSnapshot, statemachine has to pre-compute each file's md5
digest and provide it in {{{}FileInfo{}}}. When snapshot file is large, this
computing is time-consuming and may cause leader timeout.
In this PR I propose to move the computing task along within the
InstallSnapshot stream process. That is, for each SnapshotChunk, we can read
the chunk & compute its moving MD5 digest in the same time. This can enable us
to read file & compute file MD5 in one sequential scan and thus save IO costs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
