On 3/18/16 8:56 AM, Emir Arnautovic wrote:
Problem starts with autowarmCount="5000" - that executes 5000 queries
when new searcher is created and as queries are executed, document cache
is filled. If you have large queryResultWindowSize and queries return
big number of documents, that will eat up memory before new search is
executed. It probably takes some time as well.
This is also combined with filter cache. How big is your index?
Index is not very large.
numDocs:
85933
maxDoc:
161115
deletedDocs:
75182
Size
1.08 GB
I have run a query to return all documents with all fields. I could not
reproduce OOM. I understand that I need to reduce cache sizes but
wondering what conditions could have caused OOM so I can keep a watch.
Thanks
Thanks,
Emir
On 18.03.2016 15:43, Rallavagu wrote:
Thanks for the recommendations Shawn. Those are the lines I am
thinking as well. I am reviewing application also.
Going with the note on cache invalidation for every two minutes due to
soft commit, wonder how would it go OOM in simply two minutes or is it
likely that a thread is holding the searcher due to long running query
that might be potentially causing OOM? Was trying to reproduce but
could not so far.
Here is the filter cache config
<filterCache class="solr.FastLRUCache" size="5000" initialSize="5000"
autowarmCount="1000"/>
Query Results cache
<queryResultCache class="solr.LRUCache" size="20000"
initialSize="20000" autowarmCount="5000"/>
On 3/18/16 7:31 AM, Shawn Heisey wrote:
On 3/18/2016 8:22 AM, Rallavagu wrote:
So, each soft commit would create a new searcher that would invalidate
the old cache?
Here is the configuration for Document Cache
<documentCache class="solr.LRUCache" size="100000"
initialSize="100000" autowarmCount="0"/>
<enableLazyFieldLoading>true</enableLazyFieldLoading>
In an earlier message, you indicated you're running into OOM. I think
we can see why with this cache definition.
There are exactly two ways to deal with OOM. One is to increase the
heap size. The other is to reduce the amount of memory that the program
requires by changing something -- that might be the code, the config, or
how you're using it.
Start by reducing that cache size to 4096 or 1024.
https://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap
If yuo've also got a very large filterCache, reduce that size too. The
filterCache typically eats up a LOT of memory, because each entry in the
cache is very large.
Thanks,
Shawn
