[
https://issues.apache.org/jira/browse/SLING-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15123485#comment-15123485
]
Ian Boston commented on SLING-5435:
-----------------------------------
I wrote the initial description, so need to clarify.
The aim of the issue is to make it possible to reduce heartbeat timeouts and
improve the speed and reliability of a leader election, from the current level
to a level seen in other applications that perform leader elections (eg
MongoDB, etcd, etc). Most of these applications are using some variant of RAFT,
but any widely tested and proven algorithm would stand the best chance of
achieving this. Most, if not all, perform the leader election via network
communication between the candidates. None, afaik rely on a centralised store
to perform leader election, as that would itself need to be elected (as MongoDB
does)
IIUC (or rather I was told) that the reason a leader election could not use an
algorithm like RAFT over network for leader election as opposed to the current
via repository discovery mechanism was because it was too fast for some
"processes". Those "processes" needed the repository to arrive at a consistent
state before they could accept a new leader.
If those "processes" don't exist, then it sounds like there is nothing to stop
a faster leader election implementation that is not slowed down by the latency
required to ensure a repository reaches a consistent state.
If that's really the case, then this issue can be closed and replaced by a new
issue titled something like "Implement leader election using RAFT over the
network" in the same way that systems like etcd perform elections.
I have put the word processes in " as I don't know if they are single threads,
or conceptual tasks that represent a sequence of operations.
> Decouple processes that depend on cluster leader elections from the cluster
> leader elections.
> ---------------------------------------------------------------------------------------------
>
> Key: SLING-5435
> URL: https://issues.apache.org/jira/browse/SLING-5435
> Project: Sling
> Issue Type: Improvement
> Components: General
> Reporter: Ian Boston
>
> Currently there are many processes in Sling that must complete before a Sling
> Discovery cluster leader election is declared complete. These processes
> include things like transferring all Jobs from the old leader to the new
> leader and waiting for the data to appear visible on the new leader. This
> introduces an additional overhead to the leader election process which
> introduces a higher than desirable timeout for elections and heartbeat. This
> higher than desirable timeout precludes the use of more efficient election
> and distributed consensus algorithms as implemented in Etcd, Zookeeper or
> implementations of RAFT.
> If the election could be declared complete leaving individual components to
> manage their own post election operations (ie decoupling those processes from
> the election), then faster election or alternative Discovery implementations
> such as the one implemented on etcd could be used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
