[
https://issues.apache.org/jira/browse/RATIS-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated RATIS-2273:
-------------------------------
Description:
RATIS-1298 discussed the possibility and designs of having a Ratis listener
which listens from non-leader (e.g. follower or another listeners). The idea is
that since Ratis listener will never be a voting-member, it is safe to just
replicate the Raft logs of any follower up to the commitIndex.
This can be useful in the case of HDDS-12307 for asynchronous replication from
non-leader node since it doesn't require the most up-to-date state.
Few modified ideas from RATIS-1298 (credits to the contributors that suggested
the original ideas):
* Pull-based approach: Listener pull log from follower
** Can be round-robin
** Or we can pick a single follower to subscribe from. In case of follower
failure, the listener will pick another follower.
** This requires some kind of subscription API to pull committed logs from a
single Raft server
** Pros & Cons
*** Cons
**** We need to write code about learner pull log, we can not reuse
GrpcLogAppender.
**** Also write a failover mechanism to detect follower crash / slowness and
pick another follower
*** Pros
**** The voting-members do not need to be aware of the listener
* Push-based approach: Follower push log to learner. When learner start, it
register to one follower, then follower use GrpcLogAppender to append log to
learner, if follower crash, learner need to register to other follower.
** Pros & Cons
*** Cons: We need to consider the logic of register , especially when follower
crash. And also need to consider follower become leader, we need to allocate
listener to other followers.
*** Pros: We can use GrpcLogAppender to append log.
I'm leaning on the pull-based approach.
was:
RATIS-1298 discussed the possibility and designs of having a Ratis listener
which listens from non-leader (e.g. follower or another listeners). The idea is
that since Ratis listener will never be a voting-member, it is safe to just
replicate the Raft logs of any follower up to the commitIndex.
This can be useful in the case of HDDS-12307 for asynchronous replication from
non-leader node since it doesn't require the most up-to-date state.
> Support Ratis listener sync from non-leader
> -------------------------------------------
>
> Key: RATIS-2273
> URL: https://issues.apache.org/jira/browse/RATIS-2273
> Project: Ratis
> Issue Type: Improvement
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> RATIS-1298 discussed the possibility and designs of having a Ratis listener
> which listens from non-leader (e.g. follower or another listeners). The idea
> is that since Ratis listener will never be a voting-member, it is safe to
> just replicate the Raft logs of any follower up to the commitIndex.
> This can be useful in the case of HDDS-12307 for asynchronous replication
> from non-leader node since it doesn't require the most up-to-date state.
> Few modified ideas from RATIS-1298 (credits to the contributors that
> suggested the original ideas):
> * Pull-based approach: Listener pull log from follower
> ** Can be round-robin
> ** Or we can pick a single follower to subscribe from. In case of follower
> failure, the listener will pick another follower.
> ** This requires some kind of subscription API to pull committed logs from a
> single Raft server
> ** Pros & Cons
> *** Cons
> **** We need to write code about learner pull log, we can not reuse
> GrpcLogAppender.
> **** Also write a failover mechanism to detect follower crash / slowness and
> pick another follower
> *** Pros
> **** The voting-members do not need to be aware of the listener
> * Push-based approach: Follower push log to learner. When learner start, it
> register to one follower, then follower use GrpcLogAppender to append log to
> learner, if follower crash, learner need to register to other follower.
> ** Pros & Cons
> *** Cons: We need to consider the logic of register , especially when
> follower crash. And also need to consider follower become leader, we need to
> allocate listener to other followers.
> *** Pros: We can use GrpcLogAppender to append log.
> I'm leaning on the pull-based approach.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
