[
https://issues.apache.org/jira/browse/SOLR-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17449023#comment-17449023
]
Atri Sharma edited comment on SOLR-15715 at 11/25/21, 8:39 AM:
---------------------------------------------------------------
{quote}{{Cost of bringing up a new aggregation node and bringing down a
misbehaving aggregation node is much lower than misbehaving nodes with data in
it.}}
{quote}
{{{}{}}}{{{}That is exactly what I want to understand – how do you scale these
nodes up? Are these autoscaled? What is the implication of scaling these nodes
on the metadata storage (i.e. ZK state) and the overseer?{}}}
{{You are concentrating a load (that was previously distributed) on a few
select nodes, so the probability of needing to scale these nodes is high. That
can eventually mean a larger cluster overall, since you aren't really removing
any data nodes?}}
{quote}{{At FullStory, this solution has been running in a production Solr
cluster with hundreds of nodes for several months now with immense benefits in
reducing load on the data nodes.}}
{quote}
While I am glad to hear that, I would still like to see a simulated (and
reproducible) benchmark that targets the above mentioned scenario and
demonstrates the said feature's handling of the same.
was (Author: atri):
{quote}Cost of bringing up a new aggregation node and bringing down a
misbehaving aggregation node is much lower than misbehaving nodes with data in
it.
That is exactly what I want to understand – how do you scale these nodes up?
Are these autoscaled? What is the implication of scaling these nodes on the
metadata storage (i.e. ZK state) and the overseer?
You are concentrating a load (that was previously distributed) on a few select
nodes, so the probability of needing to scale these nodes is high. That can
eventually mean a larger cluster overall, since you aren't really removing any
data nodes?
At FullStory, this solution has been running in a production Solr cluster with
hundreds of nodes for several months now with immense benefits in reducing load
on the data nodes.
{quote}
While I am glad to hear that, I would still like to see a simulated (and
reproducible) benchmark that targets the above mentioned scenario and
demonstrates the said feature's handling of the same.
> Dedicated query aggregator nodes in the solr cluster.
> ------------------------------------------------------
>
> Key: SOLR-15715
> URL: https://issues.apache.org/jira/browse/SOLR-15715
> Project: Solr
> Issue Type: New Feature
> Components: SearchComponents - other
> Affects Versions: 8.10.1
> Reporter: Hitesh Khamesra
> Priority: Major
> Attachments: coordinator-poc.jpg, coordinator-poc.pdf,
> regular-node.jpg, regular-node.pdf
>
>
> We have a large collection with 1000s of shards in the solr cluster. We have
> observed that distributed solr query takes many resources(thread, memory,
> etc.) on the solr data node(node which contains indexes). Thus we need
> dedicated query nodes to execute distributed queries on large solr
> collection. That would reduce the memory/cpu pressure from solr data nodes.
> Elastis search has similar functionality
> [here|https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#coordinating-node]
>
> [~noble.paul] [~ichattopadhyaya]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
