[
https://issues.apache.org/jira/browse/SOLR-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998138#comment-15998138
]
Shalin Shekhar Mangar commented on SOLR-10524:
----------------------------------------------
Yes, I like this. Same performance, much smaller changes and no chance of
something going wrong in the cluster because of processing re-ordered messages.
+1 to commit.
There are optimizations we can do on the read side using multi-get. Lets open
another issue to explore that as well.
As a side note, there is a bug in the nsToMs method in testOverseer -- it
actually assumes the nanoseconds as milliseconds and the converts them to nano
seconds! I'll fix it separately.
> Explore in-memory partitioning for processing Overseer queue messages
> ---------------------------------------------------------------------
>
> Key: SOLR-10524
> URL: https://issues.apache.org/jira/browse/SOLR-10524
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Erick Erickson
> Attachments: SOLR-10524.patch, SOLR-10524.patch, SOLR-10524.patch,
> SOLR-10524.patch
>
>
> There are several JIRAs (I'll link in a second) about trying to be more
> efficient about processing overseer messages as the overseer can become a
> bottleneck, especially with very large numbers of replicas in a cluster. One
> of the approaches mentioned near the end of SOLR-5872 (15-Mar) was to "read
> large no:of items say 10000. put them into in memory buckets and feed them
> into overseer....".
> This JIRA is to break out that part of the discussion as it might be an easy
> win whereas "eliminating the Overseer queue" would be quite an undertaking.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
