[
https://issues.apache.org/jira/browse/SOLR-14306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17051873#comment-17051873
]
Jan Høydahl commented on SOLR-14306:
------------------------------------
Seems Kafka has had the same discussions (see
https://cwiki.apache.org/confluence/display/KAFKA/KIP-273+-+Kafka+to+support+using+ETCD+beside+Zookeeper)
but I think they ended up with
https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum
instead, I.e handling state and coordination in Kafka instead of external
system.
Would be interesting to evaluate Apache Ratis
(http://ratis.incubator.apache.org/) as an embedded zk replacement!
> Refactor coordination code into separate module and evaluate using Curator
> --------------------------------------------------------------------------
>
> Key: SOLR-14306
> URL: https://issues.apache.org/jira/browse/SOLR-14306
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Reporter: Tomas Eduardo Fernandez Lobbe
> Priority: Major
>
> This Jira issue is to discuss two changes that unfortunately are difficult to
> address separately
> # Separate all ZooKeeper coordination logic into it’s own module, that can
> be tested in isolation
> # Evaluate using Apache Curator for coordination instead of our own logic.
> I drafted a
> [SIP|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148640472],
> but this is very much WIP, I’d like to hear opinions before I spend too much
> time on something people hates.
> From the initial draft of the SIP:
> {quote}The main goal of this change is to allow better testing of the
> different ZooKeeper interactions related to coordination (leader election,
> queues, etc).
There are already some abstractions in place for lower level
> operations (set-data, get-data, etc, see DistribStateManager), so the idea is
> to have a new, related abstraction named CoordinationManager, where we could
> have some higher level coordination-related classes, like LeaderRunner
> (Overseer), LeaderLatch (for shard leaders), etc.
Curator comes into place
> because, in order to refactor the existing code into these new abstractions,
> we’d have to rework much of it, so we could instead consider using Curator, a
> library that was mentioned in the past many times. While I don’t think this
> is required, It would make this transition and our code simpler (from what I
> could see, however, input from people with more Curator experience would be
> greatly appreciated).
> While it would be out of the scope of this change, If the
> abstractions/interfaces are correctly designed, this could lead to, in the
> future, be able to use something other than ZooKeeper for coordination,
> either etcd or maybe even some in-memory replacement for tests.
> {quote}
> There are still many open questions, and many questions I still don’t know
> we’ll have, but please, let me know if you have any early feedback, specially
> if you’ve worked with Curator in the past.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
