[
https://issues.apache.org/jira/browse/IOTDB-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090199#comment-17090199
]
Tan xinyu commented on IOTDB-606:
---------------------------------
Hi, I'd like to mention a few of my thoughts.
As you can see the
[code|https://github.com/LebronAl/incubator-iotdb/blob/b36564c06889e6dba55f202ef671e560db2bf584/cluster/src/main/java/org/apache/iotdb/cluster/server/member/MetaGroupMember.java#L1094]
here.Although the snapshot of the metadata group currently transfers
storageGroup, TTL, and so on. But it still needs to replay all logs one by one,
which makes little sense to take a snapshot of the metadata group. Because
supporting batch log replication is the same as this implementation.
I think the fact that there is a need for full replay is due to three types of
logs : ADD_NODE_LOG,REMOVE_NODE_LOG,CLOSE_FILE_LOG. In fact, we should not need
to replay the first two types of logs as long as you send the partition table
at the same time. In addition, the last type of log does not appear to be in
the metadata group. So in your design there seems to be no need to replay all
logs.
These are my thoughts after a brief reading of the code, which may be wrong.
But I really want the implementation of this issue to be able to remove the
implementation that replays all the logs.
Thanks for contributing~
> [Distributed] Replace raw logs in MetaSnapshot
> ----------------------------------------------
>
> Key: IOTDB-606
> URL: https://issues.apache.org/jira/browse/IOTDB-606
> Project: Apache IoTDB
> Issue Type: Improvement
> Reporter: Tian Jiang
> Priority: Major
> Labels: cluster, metadata, snapshot
>
> The current MetaSnapshot is using the simplest way, storing the raw committed
> logs. It would be more efficient to replace the logs with compact structures
> like the partition table and other objects that will be affected by meta logs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
