On Wed, Oct 29, 2003 at 07:56:53AM -0500, Scott Ganyo wrote: > Offhand, I would say that using 2 directories and merging them is > exactly what you waht. It really shouldn't be all that complicated and > Lucene should handle the synchronization for you...
Will it do that if we build the indexes in two different JVMs, and let one of them handle the merge. I realize that Lucene is thread safe, and handles the locking, but I thought that that's only true if you do all the work in a single JVM. Regards, Dror > > Scott > > Dror Matalon wrote: > > >Hi folks, > > > >We're in the process of adding search to our online RSS aggregator. You > >can see it in action at www.fastbuzz.com. > > > >Currently we have more than five million items in the systems and it's > >growing at the rate of more than 100,00 a day. So we need to take into > >account is that the index is constantly growing. > > > >One of the things we want to build into the system is the ability to > >rebuild the index on the fly while still inserting the items that are > >coming in. > > > >We've looked at having things go into different directories and then > >merge them, but it seems complicated and we'd need to worry about race > >conditions and locking issues. > > > >Anyone's done this before? Any suggestions? > > > > > >Regards, > > > >Dror > > > > > > > > -- > ...there is nothing more difficult to execute, nor more dubious of success, > nor more dangerous to administer than to introduce a new order to things; > for he who introduces it has all those who profit from the old order as his > enemies; and he has only lukewarm allies in all those who might profit from > the new. - Niccolo Machiavelli > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.zapatec.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
