Hi Nitin, * I haven't heard about that 100GB limit but I tried Lucene.Net once with a 300GB index. The first searches (with a fresh IndexSearcher) took ~20sec(because of caching) but next searches performed quite well(varying from ~50msec to 3sec).
* If you deal with such large indexes, it is better to group the indexes according to some criteria(for ex., index of December, index of November etc.) and not to use an index when it is not needed in the search. Of course, keeping smaller indexes on multiple machines and making a parallel search on them and then merging the results would be a good solution too, but it would require more complex coding You may also want to see some tricks about search speed optimizations ( http://wiki.apache.org/jakarta-lucene/ImproveSearchingSpeed ) and the project Solr ( http://lucene.apache.org/solr/features.html ). * You can get the official releases of Lucene.Net from https://svn.apache.org/repos/asf/incubator/lucene.net/site/download and the current version from svn trunk https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/C%23/src/Lucene. Net/ DIGY. -----Original Message----- From: Nitin Shiralkar [mailto:nit...@coreobjects.com] Sent: Saturday, December 27, 2008 6:41 AM To: lucene-net-user@incubator.apache.org Subject: Lucene Scalability Options Hi All, We are using Lucene.NET v2.0 library in our project. Our index has grown to ~80 GB in last one year. We expect our index to grow beyond 100 GB in next six months. I have read somewhere long back about Lucene performance issues after crossing 100 GB mark. - Is there any specific issues that we might run into after 100 GB? - Is there any known impact on search performance? - Do we have any scalability features that we can consider for implementation? Clustering etc? Any inputs would be valuable. Also I would like to know the latest stable Lucene.NET release which we can migrate to, any download link would be useful. Thanks & regards, Nitin Shiralkar
