Thanx Terry Well today again I have made a few changes in the archietecture of my component where am using Lucene, and changed the way I am using the IndexReader and as u said made sure that all readers are closed, mergefactor is back to default (10).
The test run is on and its working pretty well for now, have managed to have about 1000 documents, index of about 10MB, n counting :) .. hope this time things are better, well thanx for your word, it made me feel that this has been rightly done before and can be done even now. I appreciate the way you replied. Thanx would get back to you later ... Cheers!! Amit ----- Original Message ----- From: Terry Steichen To: Lucene Users Group Sent: Thursday, April 03, 2003 7:38 PM Subject: Re: Problem while indexing Amit, I don't exactly know what your problem is, but I'm using a configuration not too different from yours with no problems - so at least you know it's possible. I have an index of about 125MB which I use on various machines, including an old Windows98/SE 400MHz notebook. I used the default MergeFactor (10, I think) and do a daily merge (the daily addition represents about 200 documents added to a total of over 58,000). Each document (XML format) has about 15 fields of various types. I'm using release 1.3 dev 1. At one point I too had a problem of too many open files - turned out that I wasn't closing the IndexReader. Fixed that, and the number of open files usually stays below 500 (without Lucene, there are typically about 300-400 open files just for the system). HTH, Terry ----- Original Message ----- From: "Amit Kapur" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Cc: <[EMAIL PROTECTED]> Sent: Thursday, April 03, 2003 12:13 AM Subject: Problem while indexing > > hi all > > I m facing problems like mentioned below while indexing, If anyone has any > help to offer i would to obliged.... > **** couldn't rename segments.new to segments **** > **** F:\Program Files\OmniDocs Server\ftstest\_3cf.fnm (Too many open > files)**** > > I am trying to index documents using Lucene generating about 30 MB of index > (Optimized) which can be raised to about 100 MB or More ( but that would be > on a high end server machine). > > Description of Current Case: > #---Each Document has four fields (One Text field, and 3 other Keyword > Fields). > #---The analyzer is based on a StopFilter and a PorterStemFilter. > #---I am using a Compaq PIII, 128 MB RAM, 650 MHz. > #---mergeFactor is set to 25, and I am optimizing the index after adding > about 20 Documents. > #---Using Lucene Release 1.2 > > Problem Faced > After adding about 4000 Documents generating an index of 30 MB, I initially > got an error saying, **** couldn't rename segments.new to segments **** > after which the IndexReader or the IndexWriter to the current index couldnot > be opened. > > Then I changed a couple of settings, > #---mergeFactor=20 and Optimize was called after ever 10 documents. > #---Using Lucene Release 1.3 > > Problem Faced > After adding about 1500 Documents generating an index of 10 MB, I initially > got an error saying, **** F:\Program Files\OmniDocs Server\ftstest\_3cf.fnm > (Too many open files)**** after which the IndexWriter to the current index > couldnot be opened. > > Now my requirement needs to have a much much larger index (practically) and > I am actually at the point where these errors are coming unpredictably. > > Please if anyone could guide me on this ASAP. > Thanx in advance > > Regards > Amit > > PS: I have already read articles in the mail archieve > http://www.mail-archive.com/[EMAIL PROTECTED]/msg02815.html. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
