[
https://issues.apache.org/jira/browse/SOLR-14613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170222#comment-17170222
]
Ilan Ginzburg commented on SOLR-14613:
--------------------------------------
I do have the unpleasant impression since I started work on this Jira that
every step I make is mirrored in a slightly different way elsewhere.
I'd be +sincerely happy+ to stop working on this and do other things. I'd hate
to invest more time and see the work being duplicated.
I understand the motivation behind SOLR-14680 (Provide simple interfaces to our
concrete SolrCloud classes) even though I don't agree with the approach as I
feel there's a fundamental difference between defining internal interfaces and
making these interfaces visible outside Solr which I feel is a mistake.
In this Jira there are two main aspects:
1. Visibility over cluster state to make placement decisions, with:
-- 1.1 The stuff that usually goes in {{ClusterState}}: nodes, collections,
shards, replicas,
-- 1.2 More information needed for placement computation: system properties,
metrics, various stats such as number of cores, disk type etc.
2. Plugins getting requests for placement computation and returning placement
decisions.
1.1 apparently went to SOLR-14680. My plan was to continue working for now with
the abstractions I originally created in [PR
1684|https://github.com/apache/lucene-solr/pull/1684] because I feel they are
easier to use and more efficient to implement than those in SOLR-14680 and see
later if there's a need to switch (which should be trivially easy once
SOLR-14680 is mature enough).
For 1.2, I tried to introduce a strongly typed approach with an efficient fetch
strategy allowing to both fetch only what's required and do so using a single
round trip to each remote node.
For item 2, I went with {{Request}} and {{WorkOrder}}, again with a strongly
typed approach. That's where I was expecting to concentrate my efforts now,
inventory the Collection API commands and make sure all placement decisions
(and the relevant data) is available in the plugin framework so that it's
possible to wire all Collection API placement decisions into plugin calls, and
make sure plugins have all the data they need to make decisions.
Then would have come the questions of how to replace "triggers" (scheduled and
condition based job execution). I expected to tackle this once placement is
solved, since placement is the most urgent.
Should I continue work on these subjects and if so, which part(s)?
TBH I'm tempted to move to other topics and leave replacing Autoscaling to
others for now.
I don't want to fight.
[~noble.paul] [~ichattopadhyaya] [~ab] as the most invested in Autoscaling.
> Provide a clean API for pluggable replica assignment implementations
> --------------------------------------------------------------------
>
> Key: SOLR-14613
> URL: https://issues.apache.org/jira/browse/SOLR-14613
> Project: Solr
> Issue Type: Improvement
> Components: AutoScaling
> Reporter: Andrzej Bialecki
> Priority: Major
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> As described in SIP-8 the current autoscaling Policy implementation has
> several limitations that make it difficult to use for very large clusters and
> very large collections. SIP-8 also mentions the possible migration path by
> providing alternative implementations of the placement strategies that are
> less complex but more efficient in these very large environments.
> We should review the existing APIs that the current autoscaling engine uses
> ({{SolrCloudManager}} , {{AssignStrategy}} , {{Suggester}} and related
> interfaces) to see if they provide a sufficient and minimal API for plugging
> in alternative autoscaling placement strategies, and if necessary refactor
> the existing APIs.
> Since these APIs are internal it should be possible to do this without
> breaking back-compat.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
