[
https://issues.apache.org/jira/browse/SOLR-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936267#comment-15936267
]
Shalin Shekhar Mangar commented on SOLR-9735:
---------------------------------------------
Please access the [first iteration of the design
here|https://docs.google.com/document/d/1QX6AyOkceQs-PlQUW_TE4jb9N2ZgL7GFL7Ao7yl7fBM/edit?usp=sharing].
Anyone with the link can comment on the google document without signing-in.
It outlines the basic concepts and the vocabulary we will use to discuss and
implement this feature. It also has examples of various new APIs that will be
written to support this feature.
A good part of the design and terminology was conceived by the open source team
at Lucidworks during our in-person meetings at San Francisco in February. Noble
Paul and I are the primary authors of the document but it includes valuable
ideas and feedback from all members of the open source team viz. Andrzej,
Cassandra, Dat, Erick, Ishan, Hoss, Steve and Trey. This contribution is being
sponsored by Lucidworks Inc.
This is very much a living document and will continue to be refined as we think
through various use-cases and edge cases. Please feel free to share your
feedback and comments here.
I have already created a features/autoscaling branch for this work on the
public git repo and we've begun prototyping key pieces of the design for which
we'll open sub-issues.
> Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud
> ------------------------------------------------------------------
>
> Key: SOLR-9735
> URL: https://issues.apache.org/jira/browse/SOLR-9735
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Anshum Gupta
> Assignee: Shalin Shekhar Mangar
> Original Estimate: 1,344h
> Remaining Estimate: 1,344h
>
> As SolrCloud is now used at fairly large scale, most users end up writing
> their own cluster management tools. We should have a framework for cluster
> management in Solr.
> In a discussion with [~noble.paul], we outlined the following steps w.r.t.
> the approach to having this implemented:
> * *Basic API* calls for cluster management e.g. utilize added nodes, remove a
> node etc. These calls would need explicit invocation by the users to begin
> with. It would also specify the {{strategy}} to use. For instance I can have
> a strategy called {{optimizeCoreCount}} which would target to have an even
> no:of cores in each node . The strategy could optionally take parameters as
> well
> * *Metrics* and stats tracking e.g. qps, etc. These would be required for any
> advanced cluster management tasks e.g. *maintain a qps of 'x'* by
> *auto-adding a replica* (using a recipe) etc. We would need
> collection/shard/node level views of metrics for this.
> * *Recipes*: combination of multiple sequential/parallel API calls based on
> rules. This would be complicated specially as most of these would be long
> running series of tasks which would either have to be rolled back or resumed
> in case of a failure.
> * *Event based triggers* that would not require explicit cluster management
> calls for end users.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
