http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr This article is a great read.
To your other question about a 10 second delay, that seems inordinately large to me. I would suggest going through this http://wiki.apache.org/lucene-java/BasicsOfPerformance Should help you narrow down the issue. On Tue, Dec 29, 2009 at 5:11 AM, Li Bing <[email protected]> wrote: > Dear Shashi Kant, > > Could you let me know how to shard the index? Any existing APIs support this? > > Thanks, > LB > > > On Thu, Dec 17, 2009 at 1:45 AM, Shashi Kant <[email protected]> wrote: >> Have you considered sharding the index using some logic? such as >> alphabetically or perhaps by document id etc. >> That way you have ||el indexing and searching. >> >> >> >> On Wed, Dec 16, 2009 at 12:39 PM, Li Bing <[email protected]> wrote: >>> >>> Dear Didy, >>> >>> My solution is as follows. >>> >>> 1) Only one thread can index (write); >>> >>> 2) Multiple threads can retrieve (read). >>> >>> Now the retrieval performance is fine. Do you think my solution is >>> correct? Any better solutions? >>> >>> Best, >>> LB >>> >>> On Thu, Dec 17, 2009 at 1:36 AM, Digy <[email protected]> wrote: >>> > I remember that this was discussed in thread "Exceptions When Indexing >>> > with >>> > Multi-threading" Oct 2009 >>> > DIGY >>> > >>> > -----Original Message----- >>> > From: Li Bing [mailto:[email protected]] >>> > Sent: Wednesday, December 16, 2009 11:46 AM >>> > To: [email protected] >>> > Subject: What About the Performance When Retrieving at Indexing? >>> > >>> > Dear all, >>> > >>> > I just want to know what about the performance when retrieving while >>> > new data is being indexed if I do not manage the threads myself. It >>> > must be lower than retrieving an index which is not being updated, >>> > right? In my case, the retrieving can be processed after the indexing >>> > is done. It is really slow. Does Lucene solve the issue itself? >>> > >>> > Thanks again! >>> > LB >>> > >>> > ---------- Forwarded message ---------- >>> > From: Li Bing <[email protected]> >>> > Date: Wed, Dec 16, 2009 at 5:26 PM >>> > Subject: How To Append and Synchronize Lucene Index? >>> > To: [email protected] >>> > >>> > >>> > Hi, all, >>> > >>> > I am using Lucene.NET to index data. Meanwhile, more new data is >>> > frequently added to the index. However, if so, the retrieval >>> > performance becomes much lower when new data is being added and >>> > indexed because the indexing (writing) thread must synchronize the >>> > retrieving (reading) threads. >>> > >>> > I use .NET locks to manage the threads myself. Is it necessary? Or >>> > Lucene can handle the synchronize itself? >>> > >>> > I would like to know if it is possible for me to append one portion of >>> > Lucene indexes to another one. If so, it is easy to replicate Lucene >>> > indexes and the writing and reading do not affect each other. Or any >>> > other solutions in this case? >>> > >>> > Thanks so much! >>> > LB >>> > >>> > >> >> >
