[
https://issues.apache.org/jira/browse/SOLR-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911237#comment-16911237
]
Dr Oleg Savrasov edited comment on SOLR-6930 at 8/20/19 11:35 AM:
------------------------------------------------------------------
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{{{{<solr>}}}}
{{ {{ <metrics>}}}}
{{ {{ <reporter name="unInvertedField" group="node"
class="org.apache.solr.metrics.breaker.MemoryCircuitBreaker">}}}}
{{ {{ <str name="limit">20%</str>}}}}
{{ </reporter> }}
{{ {{ </metrics>}}}}
{{ {{</solr>}}}}
was (Author: osavrasov):
Our customers very often experience OOM because of caching too large
UnInvertedField items. This happens when they try to count facets or sort by
not docvalues field. We don't want to fully disable UnInvertedFields usage
because in some cases they work better than docvalues. For example, if the
field is sparse and defined just in a few documents out of the millions, it
seems better to create and cache relatively small UnInvertedField rather than
traverse full docvalues structure. So to prevent OOM we need a kind of circuit
breaker which would count memory utilized by UnInvertedField items and forbid
caching them, when some configured threshold is exceeded. As you can see, with
such an approach we are not trying to predict memory consumption, we're just
restricting UnInvertedField cache by consumed memory.
Since such a breakers try to control memory usage, ideally they should be part
of some Global Resource Manager, see
https://issues.apache.org/jira/browse/SOLR-13578. But we don't have such a
manager so far, so for the time being we've tried to rely on existing Metric
framework. Memory consumption is controlled by custom MetricReporter, so to
enable circuit breaker it's necessary to configure appropriate unInvertedField
metric reporter in solr.xml, for example:
{{<solr>}}
{{ <metrics>}}
{{ <reporter name="unInvertedField" group="node"
class="org.apache.solr.metrics.breaker.MemoryCircuitBreaker">}}
{{ <str name="limit">20%</str>}}
{{ </reporter> }}
{{ </metrics>}}
{{</solr>}}
> Provide "Circuit Breakers" For Expensive Solr Queries
> -----------------------------------------------------
>
> Key: SOLR-6930
> URL: https://issues.apache.org/jira/browse/SOLR-6930
> Project: Solr
> Issue Type: Improvement
> Components: search
> Reporter: Mike Drob
> Priority: Major
> Attachments: SOLR-6930.patch
>
>
> Ref:
> http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html
> ES currently allows operators to configure "circuit breakers" to preemptively
> fail queries that are estimated too large rather than allowing an OOM
> Exception to happen. We might be able to do the same thing.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
