here is a pretty good and efficient way to do it, from the author of lucene himself http://www.mail-archive.com/[email protected]/msg12709.html
---------------------------------------- > Date: Wed, 17 Nov 2010 15:58:53 +0800 > Subject: How to Transmit and Append Indexes > From: [email protected] > To: [email protected] > > Hi, all, > > I am working on a distributed searching system. Now I have one server only. > It has to crawl pages from the Web, generate indexes locally and respond > users' queries. I think this is too busy for it to work smoothly. > > I plan to use two servers at at least. The jobs to crawl pages and generate > indexes are done by one of them. After that, the new available indexes > should be transmitted to anther one which is responsible for responding > users' queries. From users' point of view, this system must be fast. > However, I don't know how I can get the additional indexes which I can > transmit. After transmission, how to append them to the old indexes? Does > the appending block searching? > > Thanks so much for your help! > > Bing Li
