I finally had some time to take Doug's advice and reburn our indexes with a larger TermInfosWriter.INDEX_INTERVAL value.

The default is 128 but I increased it to 256 and then burned our indexes again and was lucky enough to notice that our memory usage dropped in 1/2.

This introduced a bug however where when we try to load our pages before and after we're missing 99% of documents from our index. What happens is that we have a term -> key mapping so that we can pull out documents based on essentially a primary key. The key is just hte URL of the document. With the default value it works fine but when I change it to 256 it cant' find the majority of the documents. In fact its only able to find one.

Is this setting incompatible with older indexes burned with the lower value?

Kevin

--

Use Rojo (RSS/Atom aggregator). Visit http://rojo.com. Ask me for an invite! Also see irc.freenode.net #rojo if you want to chat.

Rojo is Hiring! - http://www.rojonetworks.com/JobsAtRojo.html

If you're interested in RSS, Weblogs, Social Networking, etc... then you should work for Rojo! If you recommend someone and we hire them you'll get a free iPod!
Kevin A. Burton, Location - San Francisco, CA
AIM/YIM - sfburtonator, Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412



--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]



Reply via email to