Tian Jiang created IOTDB-1324:
---------------------------------
Summary: [Distributed] Quorum selection when appending logs
Key: IOTDB-1324
URL: https://issues.apache.org/jira/browse/IOTDB-1324
Project: Apache IoTDB
Issue Type: Improvement
Components: Core/Cluster
Reporter: Tian Jiang
Raft logs contain raw data, which is not compressed, thus sending them may
increase network bandwidth usage significantly. In the current implementation,
a log is sent to *_all_* followers and the leader will wait for a positive
response from any quorum before proceeding to commit. By choosing a quorum and
only send logs to nodes in the selected quorum and only using snapshots (data
files) to make other nodes catch up, network bandwidth can be saved as data
files are highly compressed.
This is similar to the learner mechanism, but we do not have learners
explicitly: the leader simply ignores nodes that are not in the selected quorum
during log replication and uses snapshots to make them catch up when the
snapshot is updated; as such nodes are still followers, they can be added into
the quorum without any explicit state change.
The problems to be solved are 1. The leader should be able to know when the
snapshot is updated, so it also knows went to make the non-quorum followers
catch up; 2. Queries to the non-quorum nodes should be redirected if
consistency is required, as the leader updates the nodes more lazily, the
latency may be too long if the follower waits for the synchronization; 3. The
leader should modify the quorum smartly in response to failures in the quorum,
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
