On 4/2/07, Yonik Seeley <[EMAIL PROTECTED]> wrote:
On 4/1/07, Jeff Rodenburg <[EMAIL PROTECTED]> wrote: > Our scenario: 150MB index, 140000 documents, read/write servers in place > using standard replication. Running Tomcat 5.5.17 on Redhat Enterprise > Linux 4. Java configured to start with -Xmx1024m. We encounter java heap > out-of-memory issues on the read server at staggered times, but usually once > every 48 hours. Could you do a grep through your server logs for "WARNING", to eliminate the possibility of multiple overlapping searchers causing the OOM issue?
We're not seeing warnings for overlapping searchers prior to the oom events. Only "SEVERE" -- java.lang.OutOfMemoryError: Java heap space. Are you doing incremental updates? If so, try lowering your
mergeFactor for the index, or optimize more frequently. As an index is incrementally updated, old docs are marked as deleted and new docs are added. This leaves "holes" in the document id space which can increase memory usage. Both BitSet filters and FieldCache entry sizes are proportionally related to maxDoc (the maximum internal docid in the index). You can see maxDoc from the statistics page... there might be a correlation.
We are doing incremental updates, and we optimize quite a bit. mergeFactor presently set to 10. maxDoc count = 144156 numDocs count = 144145
