On 5/25/24 07:09, Imran Chaudhry wrote:
Just a follow-up that since limiting my Solr QuerySet rows to 1000 like
this:
qs.get_results(rows=1000)
...I have not had a single OOM killer issue. I consider this resolved.
Reducing rows can reduce the amount of heap memory that Solr needs
because Solr builds the entire response in memory before sending it to
the query source. You might also want to use the "fl" parameter to
control which fields will be in the response.
side-note:
I tuned the heap and Java mem by observing the Solr dashboard and observing
the "JVM-Memory" grows with various queries in my web-app.
For reference I kept it constant like this:
# Increase Java Heap as needed to support your indexing / query needs
SOLR_HEAP="256m"
SOLR_JAVA_MEM="-Xms256m -Xmx256m"
Only one of these is actually needed. I would personally use SOLR_HEAP.
The effective result is the same.
Note that "256m" is half of the default, which is 512m. So you haven't
increased it, you have actually decreased it. But if the OOM was
happening at the OS level and not the Java level, reducing the heap
might be the correct way to go. It would also mean that most of the
host memory is being allocated to other programs and so the host memory
might simply be too small for the workload it runs. As this system
grows, you will probably need to increase the host memory beyond 4gb.
Thanks,
Shawn
