You will also want to reuse the Token using Lucene 2.3-dev (i.e the
latest from Subversion.)
-Grant
On Dec 27, 2007, at 10:26 AM, tgospodinov wrote:
Reusing documents and invoking garbage collection slowed
performance, but
improved heap usage big time (From an average of 20 megs+ to about 12
megs+). I am wondering if there's anything else that can be done?
Grant Ingersoll-6 wrote:
This question is better for java-user, but some questions for you:
What version of Lucene are you using? The trunk now has the ability
to reuse Tokens/Documents, etc. to save on allocations that may help
reduce the amount of heap needed. The other thing to do is try
profiling to see where the allocations are coming from and then
report
back to us. Also, info about your JVM, etc. would be helpful.
-Grant
On Dec 27, 2007, at 9:01 AM, tgospodinov wrote:
I am indexing a collection of 100,000+ sentences in memory, as part
of a
client app. I tested the jvm heap consumption and it increases by
about 40
megs. I tried indexing on disk (which I can't do in a production
environment) just to test the heap usage, and I came up with about
20-25
megs. I have another search alternative, which is much simpler, but
doesn't
come close to Lucene, and it only consumes about 6-8 megs while
indexing the
same number of entries. Is there a way to improve heap usage while
indexing
in-memory?
I appreciate your help.
--
View this message in context:
http://www.nabble.com/Improving-jvm-heap-consumption-when-indexing-tp14513435p14513435.html
Sent from the Lucene - Java Developer mailing list archive at
Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------
Grant Ingersoll
http://lucene.grantingersoll.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
View this message in context:
http://www.nabble.com/Improving-jvm-heap-consumption-when-indexing-tp14513435p14514440.html
Sent from the Lucene - Java Developer mailing list archive at
Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------
Grant Ingersoll
http://lucene.grantingersoll.com
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
