Hi
I've inherited a Solr config and am doing some sanity checks before making
some updates, I'm concerned about the memory settings.
System has 1 index in 2 shards split across 2 Ubuntu 64 bit nodes, each node
has 32 CPU cores and 132GB RAM, we index around 500k files a day spread out
over the day in batches every 10 minutes, a portion of these are updates to
existing content, maybe 5-10%. Currently MergeFactor is set to 2 and commit
settings are:
<autoCommit>
<maxTime>60000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>900000</maxTime>
</autoSoftCommit>
Currently each node has around 25M docs in with an index size of 45GB, we
prune the data every few weeks so it never gets much above 35M docs per
node.
On reading I've seen a recommendation that we should be using MMapDirectory,
currently it's set to NRTCachingDirectoryFactory. However currently the JVM
is configured with -Xmx131072m, and for MMapDirectory I've read you should
use less memory for the JVM so there is more available for the OS caching.
Looking at the dashboard in the JVM memory usage I see:
Not sure I understand the 3 bands, assume 127.81 is Max, dark grey is in use
at the moment and the light grey is allocated as it was used previously but
not been cleaned up yet?
I'm trying to understand if this will help me know how much would be a good
value to change Xmx to, i.e. say 64GB based on light grey?
Additionally once I've changed the max heap size is it a simple case of
changing the config to use MMapDirectory or are there things i need to watch
out for?
Thanks
Si
