I put Lock in IndexReader.indexExists function, and testes for a few days It worked fine. I never had that mistery problem.
How can put the patch in a JIRA issue? Thanks, Wenjie On 5/9/06, Otis Gospodnetic <[EMAIL PROTECTED]> wrote:
Somebody (ah, Andy) already mentioned that you may be getting unlucky and calling that IndexReader.indexExists method right when the 'segments' file is being renamed. It looks like there is no lock in that method, but it looks like we may have to add it. Take a look at the IndexReader.open(....) method to see how the Lock class is used. Add it to IndexReader.indexExists method(s) and see if you still get the error. If you don't, please put the patch with your changes in a new JIRA issues, so we can fix the problem. Thanks, Otis ----- Original Message ---- From: wenjie zheng <[EMAIL PROTECTED]> To: java-dev@lucene.apache.org Sent: Tuesday, May 9, 2006 10:57:05 AM Subject: Re: weird behavior of IndexReader.indexExists() I am a little bit confused here. Since I didn't change the lock directory or disable locking and it is on the single server sharing same fs, does it mean I am not supposed to get the error? But I did get the error. What should I do? Wenjie On 5/9/06, Otis Gospodnetic <[EMAIL PROTECTED]> wrote: > > If you don't explicitly change the lock directory and do not disable > locking, the same directory should be used. I'm assuming this is all done > on a single server sharing the same file system. The locks are stored in > the system's default temporary directory. That's typically /tmp under > UNIX/OSX and C:\WINDOWS\TEMP (I think) on Winblows. > > Otis > > ----- Original Message ---- > From: wenjie zheng <[EMAIL PROTECTED]> > To: java-dev@lucene.apache.org > Sent: Tuesday, May 9, 2006 9:09:47 AM > Subject: Re: weird behavior of IndexReader.indexExists() > > Thanks, > I guess my question is how to make sure both instances are using the same > lock directory. > > Wenjie > > On 5/9/06, Vanlerberghe, Luc <[EMAIL PROTECTED]> wrote: > > > > Make sure both instances are using the same lock directory. > > The segments file should only be read or written while holding the > > commit lock. > > > > If the lock directories don't match, you'll get more 'strange' errors... > > > > In Lucene 1.4.2 some methods did not use the lock, this has been patched > > a couple of months ago. > > > > Luc > > > > -----Original Message----- > > From: Andy Hind [mailto:[EMAIL PROTECTED] > > Sent: dinsdag 9 mei 2006 13:47 > > To: java-dev@lucene.apache.org > > Subject: RE: weird behavior of IndexReader.indexExists() > > > > Hi > > > > I think I have discovered this too. > > It is on my list of issues to raise .... > > > > The index exist test looks for the segment file. > > When the index is committing, and you are unlucky, this file may not be > > found as the new segments file replaces the old one. The result is the > > index appears not to exist. > > > > Regards > > > > Andy > > > > -----Original Message----- > > From: wenjie zheng [mailto:[EMAIL PROTECTED] > > Sent: 08 May 2006 18:57 > > To: java-dev@lucene.apache.org > > Subject: Re: weird behavior of IndexReader.indexExists() > > > > This happens sometimes when number of docs is over 2000. So it's kinda > > of > > random. > > > > Wenjie > > > > On 5/8/06, wenjie zheng <[EMAIL PROTECTED]> wrote: > > > > > > I created an index with more than 30,000 text files. > > > I used indexExists() to determine either to create a new index or to > > add > > > docs to the existing index. > > > > > > But when the num of docs in the index was over 3,000 (sometimes 3,400, > > > sometimes 3,200), the indexExists function returns false, so I ended > > up > > > recreating a new index. > > > > > > Here is my code: > > > if index exists, we will add files to it, otherwise, create a new > > index. > > > In either case, an IndexingThread will be spawn to do that. > > > if(IndexReader.indexExists(indexDir)){ > > > logger.info("Working on existing index ..."); > > > IndexingThread.startIndexingThread(Username, new > > > File(propsFile), new File(indexDir), docs, > > > new StandardAnalyzer(), false); > > > }else{ > > > logger.info("Create a new index ..."); > > > IndexingThread.startIndexingThread(Username, new > > > File(propsFile), new File(indexDir), docs, > > > new StandardAnalyzer(), true); > > > } > > > > > > > > > inside the startIndexingThread function, I am calling the following > > > function to add files to the index: > > > /** > > > * Add an array of Files to an index > > > * > > > * @param propsFile the properties file > > > * @param indexDir the folder where index files will be created > > in > > > * @param docs an array of Files to be add to the index > > > * @param analyzer any Analyzer object > > > */ > > > public void addFiles(File propsFile, File indexDir, File[] docs, > > > Analyzer analyzer, boolean overwrite) throws Exception { > > > Properties props = new Properties(new > > FileInputStream(propsFile)); > > > > > > if(overwrite || IndexReader.indexExists(indexDir)){ //either > > > overwrite or working on an existing index > > > Directory index = FSDirectory.getDirectory(indexDir, > > > overwrite); > > > IndexWriter writer = new IndexWriter(index, analyzer, > > > overwrite); > > > > > > FileIndexer indexer = new FileIndexer(props); > > > > > > long start = new Date().getTime(); > > > indexer.index(writer, docs); > > > writer.optimize(); > > > writer.close (); // close the writer > > > index.close(); // close the index Directory > > > long end = new Date().getTime(); > > > > > > logger.info("Total time: " + (end - start) + " ms"); > > > > > > }else{ > > > logger.error("Index files are not found: " + > > > indexDir.getAbsolutePath() + ", overwrite = false"); > > > } > > > } > > > > > > Thanks, > > > Wenjie > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
