We are using Lucene for searching of 200+ mln documents (periodical
publications). Is there any limitation on the size of the Lucene index (file
size, number of docs, etc...)?
We are partitioning the indexes at about 10 mln documents per partition (each
partition is on a separate box, some mounted volumes are shared among
three-four partitions). We have index size around 10% of the content size.
The total content is around 4Tb and the index around 400Gb. This content is
planned to be split into 20 partitions (10mln docs, 200Gb content size, 20Gb
index size). We are using a memory mapped index directory implementation. Our
testing is done with 600 concurrent users.
We are seeing consistently high response times from the partitions (4-5
seconds). Is there a number of documents per partition limitation in Lucene
for this particular scenario?
Thanks,
Ivan
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
