Hello Wilton, MergeFactor indicates when Segments must be merged. With a value of 10, the segments of your index will be merged into a single one when they reach this number. That's why MergeFactor has a direct impact on the number of files in your index.
However this has consequences on the performance if you are using FSDirectory. When you're adding a new Document, it is stored in RAM and then merged according to the MergeFactor specified. If this MergeFactor is low, this will produce Segments in FSDirectory more often and your indexing takes more time. If you need a faster indexation there are 3 possibilities : - indexing in several RAMDirectories and then merging the indexes in FSDirectory (see the excellent article by Otis "Advanced Text Indexing" ) - see the FileHandle solution proposed by Dmitry Serebrennikov in September (which gives the possibility to use a bigger MergeFactor but aggregates the Segments into a single file) - and at last use the patch I submitted 9/20/2003 that modifies IndexWriter by adding a new parameter minMergeDocs which specifies the number of Documents merged in RAM independently of the number of Segments in FSDirectory. You still have a fast indexation by using memory and control the number of files on the disk. Hope it helps Julien Nioche http://www.lingway.com ----- Original Message ----- From: "Wilton, Reece" <[EMAIL PROTECTED]> To: "Lucene Users List" <[EMAIL PROTECTED]> Sent: Thursday, October 09, 2003 12:21 AM Subject: RE: Too Many Open Files Ok, I lowered my MergeFactor down to 10 and am re-indexing (takes several hours). Will lowering the MergeFactor reduce the total number of files in the index directory? -Reece -----Original Message----- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: Tuesday, October 07, 2003 8:20 PM To: Lucene Users List Subject: Re: Too Many Open Files Wilton, Reece wrote: > The index directory that Lucene created has 2,322 files in it. When I > try to open it I get the dreaded "Too Many Open Files" problem: > java.io.FileNotFoundException: C:\Index\_1lvq.f107 (Too many open > files) > > The index has about 50,000 docs in it. It was created with a merge > factor of 5,000. Is there a way that I can reduce the number of files > or increase the number of files that windows can open? 5000 is way too large for the merge factor. Please read the FAQ and other messages on this list for guidelines. I've personally never found use for a merge factor larger than 50. Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
