Hi Mindaugas,
it is - as you sort of point out - the readers associated with your
searcher that consumes the memory, and not so much the searcher it
self. Thing that consume the most memory is probably field norms (8
bits per field and document unless omitted) and flyweighted terms
(String#interal), things you can't really do that much about.
But it is hard to say for sure when you've said so little about what
your index really looks like. How many fields, do you use norms, how
many documents do you have, how many unique terms are there, etc?
Another thing that can consume a lot of memory is sorting. Not sure if
that data is bound to the searcher or not though.
karl
21 sep 2009 kl. 16.32 skrev Mindaugas Žakšauskas:
Hi,
I was wondering what would be sensible amount of memory IndexSearcher
can consume? In my application we do retain reference to it for
quicker searches; however I have become a bit worried for it being a
memory hog. We are using Lucene 2.4.0 on 8 CPU Linux SMP box; JVM is
Sun's 1.6.0_14 64-Bit Server VM.
I am asking because I have ended up with IndexSearcher having Retained
size [1] of 145M. All of this memory is being eaten by
IndexSearcher::reader::subReaders[]. The reader is MultiSegmentReader
and all subReaders are SegmentReader. My memory dump showed subReaders
array having size of 37 SegmentReaders, 2 to 5 M each. I can send
YouKit screenshot if anyone's interested.
All of that should be viewed in the light of index size on the disk,
which is only 22M.
I appreciate that all of this memory can be used for legitimate
purposes; however is there a way to know when does it go over sensible
limit? Can there be a "sensible" limit at all? Also, is it possible to
set the physical boundary the IndexSearcher would never go over?
Thanks in advance for all answers.
Regards,
Mindaugas
[1] http://www.yourkit.com/docs/80/help/sizes.jsp
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
