RE: 100,000 indexes and what to do

2006-03-13 Thread John Powers
How does the information change in each of these customer's documents? I would think if they were very dynamic then updates to the single index would not be great for you. But if the updates were just now and then, then given the performance of lucene that the single index would be just fine.

Re: 100,000 indexes and what to do

2006-03-12 Thread Otis Gospodnetic
Lawrence, Thanks for the LIA compliments. In addition to what Paul and Chris already mentioned, keep in mind open files (also covered in LIA). If you have 100K separate indices, that means a lot of open file descriptors. One common index doesn't have this problem. Separate indices are still

Re: 100,000 indexes and what to do

2006-03-11 Thread Paul Elschot
On Saturday 11 March 2006 08:07, Lawrence wrote: Hi all, I was reading one of the posting on concurrency and I reread section 9.1 in Lucene in Action which lead me to this question. I have 100,000 customers and I want to provide them with personal searching for their documents and

Re: 100,000 indexes and what to do

2006-03-11 Thread Chris Lu
I think it's best to have one small index for each customer, and one large index for company's index. Merging customers' contents with the main index will cost a lot of resources, slowing down systems, while actually not necessary. If indexing is done by batch job, there'll be a delay between