[
https://issues.apache.org/jira/browse/BLUR-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13694621#comment-13694621
]
Aaron McCurry commented on BLUR-160:
------------------------------------
Yeah, this is a good one. So in a system that I run in production we are
seeing under high load some bad queries. These are either terrible wildcard or
fuzzy or both (we are using 0.1.4 which uses lucene 3.6 which makes the problem
worse than in lucene 4.0). The front end application that sends these queries
to Blur does it's best to filter out bad queries but some get through and some
are valid queries. And most of the time Blur can get through them but
sometimes there is a perfect storm of bad queries at the same time. This has
causes us to has some slow nodes because of GC (and even OOM) that in turn
cause ZK timeouts and the nodes dies. BLur then takes action and brings the
shards that are down on another shard server. The controller reissues the
queries to this new shard server, and it too goes down. Usually it stops there
because the query times out.
So this task is here to try and detect an almost out of memory heap and do
something about it before it's too late. I have created a new class that
detects the situation and will take action. Now we need to have it clear out
the jvm of queries (maybe more types of requests). The class is called
GCWatcher and it has static methods to interact with. Let me know what you
think.
Aaron
> Create a back pressure feature in Blur when GC is too high.
> -----------------------------------------------------------
>
> Key: BLUR-160
> URL: https://issues.apache.org/jira/browse/BLUR-160
> Project: Apache Blur
> Issue Type: Improvement
> Components: Blur
> Affects Versions: 0.2.0
> Reporter: Aaron McCurry
> Priority: Blocker
> Fix For: 0.2.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
