Kevin A. Burton wrote:
I started a thread about a week ago about the memory footprint of opening up a lucene index.
Ug... you know I'm sorry. Doug responded to this but I didn't see his followup. I'll try this change this week and see what happens.
You can increase TermInfosWriter.indexInterval. You'll need to re-write the .tii file for this to take effect. The simplest way to do this is to use IndexWriter.addIndexes(), adding your index to a new, empty, directory. This will of course take a while for a 60GB index...
Doubling TermInfosWriter.indexInterval should half the Term memory usage and double the time required to look up terms in the dictionary. With an index this large the the latter is probably not an issue, since processing term frequency and proximity data probably overwhelmingly dominate search performance.
Perhaps we should make this public by adding an IndexWriter method?
Also, you can list the size of your .tii file by using the main() from CompoundFileReader.
Kevin
--
Use Rojo (RSS/Atom aggregator). Visit http://rojo.com. Ask me for an invite! Also see irc.freenode.net #rojo if you want to chat.
Rojo is Hiring! - http://www.rojonetworks.com/JobsAtRojo.html
If you're interested in RSS, Weblogs, Social Networking, etc... then you should work for Rojo! If you recommend someone and we hire them you'll get a free iPod!
Kevin A. Burton, Location - San Francisco, CA
AIM/YIM - sfburtonator, Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
