One major factor that may result in heap space problems is if you are
doing any form of sorting when searching. Do you have any form of default
sort in your application? Also, the type of field used for sorting is
important with regard to memory consumption.
This issue has been discussed before on the list. (You can search the
archive for sorting and memory consumption.)
- Aleksander
On Sun, 16 Nov 2008 14:36:36 +0100, Zhibin Mai <[EMAIL PROTECTED]> wrote:
Hello,
I
am a beginner on using lucene. We developed an application to
create and search index using lucene 2.3.1. We would like to know how
to estimate how much memory is required to support
the index search given an index.
Recently,
the size of the index has reached to about 200GB with 197M of documents
and 223M of terms. Our application starts having intermittent
"OutOfMemoryError: Java heap space" when we use
it to search the index. We use JProfiler to get the following memory
allocation when we do one keyword search:
char[] 332MB
org.apache.lucene.index.TermInfo 194MB
java.lang.String 146MB
org.apache.lucene.index.Term 99,823KB
org.apache.lucene.index.Term 24,956KB
org.apache.lucene.index.TermInfo[] 24,956KB
byte[] 188MB
long[] 49,912KB
The memory allocation for the first 6 types of objects does not change
when we change the search criteria. Could you please give me some advice
what major factors will affect the memory allocation
and how those factors will affect the memory usage precisely on search?
Is it possible to reduce the memory usage on search?
Thank you,
Zhibin
--
Aleksander M. Stensby
Senior software developer
Integrasco A/S
www.integrasco.no
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
