Hello I am trying to find the right max and min settings for Java 1.6 on 20GB
index with 8 million docs, running 1.6_018 JVM with solr 1.4, and am currently
have java set to an even 4GB (export JAVA_OPTS="-Xmx4096m -Xms4096m") for both
min and max which is doing pretty well but occasionally still getting the below
OOM errors. We're running on dual quad core xeons with 16GB memory installed.
I've been getting the below OOM exceptions still though.
Is the memsize mentioned in the INFO for the uninvert in bytes? Ie is
memSize=29604020 mean 29MB? We have a few hundred of these fields and they
contain ints used as IDs, and so I guess could they eat all the memory to
uninvert them all after we apply load and enough queries are performed. Does
the field type matter, would int be better than string if these are lookup ids
sparsely populated across the index? BTW these are used for faceting and
filtering only.
<dynamicField name="*_contentAttributeToken" type="string"
indexed="true" multiValued="true" stored="true" required="false"/>
Jun 29, 2010 3:54:50 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field
{field=768_contentAttributeToken,memSize=29604014,tindexSize=50,time=1841,phase1=1824,nTerms=1,bigTerms=0,termInstances=18,uses=0}
Jun 29, 2010 3:54:52 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field
{field=749_contentAttributeToken,memSize=29604020,tindexSize=56,time=1847,phase1=1829,nTerms=143,bigTerms=0,termInstances=951,uses=0}
Jun 29, 2010 3:54:59 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: Java heap space
at
org.apache.solr.request.UnInvertedField.uninvert(UnInvertedField.java:191)
at
org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:178)
at
org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:839)
at
org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:250)
at
org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:283)
at
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:166)
