[
https://issues.apache.org/jira/browse/SOLR-16487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622377#comment-17622377
]
Justin Sweeney commented on SOLR-16487:
---------------------------------------
I have a PR coming for these improvements shortly.
> Improve efficiency of pull replicas
> -----------------------------------
>
> Key: SOLR-16487
> URL: https://issues.apache.org/jira/browse/SOLR-16487
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: replication (java)
> Affects Versions: 9.0, 8.11.2
> Reporter: Justin Sweeney
> Priority: Minor
>
> Currently there are a few inefficiencies related to pull replicas that can
> cause increased resource usage on a Solr cluster, particularly if there are a
> large number of cores.
> The following are the inefficiencies identified:
> # The
> [RecoveryStrategy.replicate()|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/cloud/RecoveryStrategy.java#L219]
> method makes a call to commit to on the leader. This happens whenever a
> replica is reloaded. For PULL replicas in particular this isn't necessary
> since we can just pull down whatever the latest data is and rely on other
> mechanisms to be consistently committing the leader. (As an aside, it seems
> like forcing a commit on the leader might never be necessary, but for this
> I've limited it to focusing on PULL replicas).
> # In a case where the leader has no data yet (index version is 0), then a
> non-leader replica will consistently delete and recreate its core due to this
> case in IndexFetcher:
> [https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/handler/IndexFetcher.java#L549.]
> This can cause unnecessary CPU usage until the leader has data indexed to it.
> # The polling for replication is fairly simply, but can lead to polling too
> often. As an example if you had the following config for commits:
> {code:java}
> <autoCommit>
> <maxTime>15000</maxTime>
> <openSearcher>false</openSearcher>
> </autoCommit>
> <autoSoftCommit>
> <maxTime>60000</maxTime>
> </autoSoftCommit> {code}
> The current logic would setup polling to be half of the autoCommit time, so
> poll every 7.5 seconds. However since a new searcher isn't opened, there will
> only be changes reflected every 60 seconds on the leader. We can make this
> logic a bit smarter knowing that the replication handler won't reflect
> changes until a new searcher is opened.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
