8 apr 2010 kl. 20.05 skrev Ivan Provalov:
We are using Lucene for searching of 200+ mln documents (periodical
publications). Is there any limitation on the size of the Lucene
index (file size, number of docs, etc...)?
The only such limitation in Lucene I'm aware of is Integer.MAX_VALUE
documents. This might also be true for number of terms.
We are partitioning the indexes at about 10 mln documents per
partition (each partition is on a separate box, some mounted volumes
are shared among three-four partitions). We have index size around
10% of the content size. The total content is around 4Tb and the
index around 400Gb. This content is planned to be split into 20
partitions (10mln docs, 200Gb content size, 20Gb index size). We
are using a memory mapped index directory implementation. Our
testing is done with 600 concurrent users.
We are seeing consistently high response times from the partitions
(4-5 seconds). Is there a number of documents per partition
limitation in Lucene for this particular scenario?
I'm not sure if I got this right but it sounds like your index is
mounted over network? Can you tell us some more details about that?
What speeds do you see if you put the index on local disc?
karl
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
