Tian Jiang created IOTDB-562:
--------------------------------
Summary: Apache IoTDB raft log persistence in the distributed
version
Key: IOTDB-562
URL: https://issues.apache.org/jira/browse/IOTDB-562
Project: Apache IoTDB
Issue Type: Improvement
Components: Core/Cluster
Reporter: Tian Jiang
Assignee: Kaifeng Xue
IoTDB is a highly efficient time series database, which supports high-speed
query process, including aggregation query.
Currently, IoTDB has supported shared-nothing cluster which using raft
mechanism and raft logs to communicate among all nodes. So raft logs are very
important in communication, consistency keeping, and fail-over.
However, the current logs are just stored in memory which means raft logs will
lost when the nodes down and then recover. Secondly, the raft logs may overlap
with current WAL which means we may do some unnecessary log writing works.
So there are two improvements about raft logs need to be done:
1. Store the raft logs in a durable material such as a disk. You need to design
a serializable form of logs and then put them to disk.
2. Find a way of using raft logs in the IoTDB recovery process. That's means we
just write raft logs rather than both raft logs and WAL. This will avoid some
unnecessary log writing works and improve insertion performance.
This proposal is mainly for improving raft logs in clustered IOTDB. Besides, if
we can let the summary info be more useful, it could be better.
Notice that the premise is that the raft logs writing process should not be
slow down too much. That means the serializable form should be high efficiency
enough.
You should know:
• IoTDB cluster structure
• IoTDB WAL
• IoTDB insertion process
• Raft
• Java
difficulty: Major
mentors:
[email protected]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
