[
https://issues.apache.org/jira/browse/IGNITE-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-15705:
-----------------------------------------
Reviewer: Alexey Scherbakov
> Investigate raft client timeouts
> --------------------------------
>
> Key: IGNITE-15705
> URL: https://issues.apache.org/jira/browse/IGNITE-15705
> Project: Ignite
> Issue Type: Task
> Reporter: Kirill Gusakov
> Assignee: Mirza Aliev
> Priority: Critical
> Labels: ignite-3
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> h3. Problem
> Raft client timeout should be large enough for the operation to be performed
> even if it falls on several consecutive rounds of choosing a new leader of
> the raft group. Most of jraft timeouts are based on electionTimeoutMs.
> {code:java}
> // A follower would become a candidate if it doesn't receive any message
> // from the leader in |election_timeout_ms| milliseconds
> // Default: 1000 (1s)
> private int electionTimeoutMs = 1000; // follower to candidate timeout
> {code}
> For example both voteTime and electionTime use exact value of
> getElectionTimeoutMs (1000 ms):
> {code:java}
> String name = "JRaft-VoteTimer-" + suffix;
> this.voteTimer = new RepeatedTimer(name,
> options.getElectionTimeoutMs(), timerFactory.getVoteTimer(name)) {...};
> name = "JRaft-ElectionTimer-" + suffix;
> electionTimer = new RepeatedTimer(name,
> options.getElectionTimeoutMs(), timerFactory.getElectionTimer(name)) {...};
> {code}
> Going back to client timeout, seems that it should be greater than
> reasonableAmountOfElecionRounds(electionTime + networkTimeoutToRetrieveAcks).
> So seems that we should check the value of “networkTimeoutToRetrieveAcks” and
> set client timeout to the corresponding value.
> Not sure whether it’s a good idea but let’s also consider raft client timeout
> to be derivative of leader election timeout not only semantically but also
> within code:
> {code:java}
> private static final int TIMEOUT = 10 * leaderElectionTimeout;{code}
> UPD:
> It was decided to implement election timeout autoadjusting mechanism. The
> main idea is that in a stable cluster election timeout should be relatively
> small, but when
> something is preventing elections from completion, like an unstable network
> or long GC pauses, we don't want to have a lot of elections, so election
> timeout is adjusted.
> Hence, the upper bound of the election timeout adjusting is the value, which
> is enough to elect a leader or handle problems that prevent a successful
> leader election.
> Leader election timeout is set to an initial value after a successful
> election of a leader.
> In our case, the upper bound of the timeout adjusting is more than timeout of
> a membership protocol to remove failed node from the cluster. So, we may
> assume
> that 11s could be enough as far as 11s is greater than suspicion timeout
> ((log_2(1000) * 500ms * 1)) for the 1000 nodes cluster with ping interval
> equals 500ms.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
