Hi all, We did a lot benchmark tests recently and discovered some potential improvements.
1. Delay snapshot MD5 computing to InstallSnapshot stream process Leader’s LogAppender while-loop checks latest snapshot info to decide wether to send a snapshot to a follower. The SnapshotInfo includes every snapshot file with its MD5 digest. Therefore, StateMachine is required to compute MD5 each time it takes a snapshot. However, for database workload, snapshot files may contain GBs of data, which makes MD5 computing a very consuming task. Since MD5 is only used when leader InstallSnapshot to a follower, it is better to compute MD5 along with InstallSnapshot stream process. Currently, InstallSnapshot stream process will break snapshot file into fixed-size chunks and send them to followers one by one. Is it possible to calculate MD5 when reading each chunk? This implementation can avoid precomputing MD5 and minimize the IO cost. Corresponding jira: https://issues.apache.org/jira/browse/RATIS-1597 <https://issues.apache.org/jira/browse/RATIS-1597> 2. Check RetryCache before applyLog RetryCache is used to reduce redundant retry requests before AppendLog. To support exactly-once semantic, it is necessary also to check RetryCache before ApplyLog. Consider this scenario: RetryCache is updated when the request is successfully committed to RaftLog, before replying to client. If the same 2 requests arrive at same time, the req#1 pass the RetryCache check and successfully appended to RaftLog, but before updating RetryCache and reply, the req#2 arrives. Req#2 will also pass RetryCache check and start to be appended to log. Corresponding jira: https://issues.apache.org/jira/browse/RATIS-1598 <https://issues.apache.org/jira/browse/RATIS-1598> Any ideas on these 2 issues? Best Wishes, William Song
