Juan Pablo,
it looks like there is platform-specific behavior. I ran the ant clean
tests a couple of times, and the SearchManagerTests all succeed.
So it's hard to debug if the problem does not occur on my laptop.
I also turned on the writer.setInfoStream( System.out ); and uploaded the
~1000 lines output here:
http://www.computerhok.nl/JSPWiki/attach/Lucene.log/lucene-log.txt
Maybe you can find some differences ?
regards,
Harry
2012/6/21 Juan Pablo Santos Rodríguez <juanpablo.san...@gmail.com>
> Ok, so finally, I think I got it.
>
> Updates to lucene Index are done in a separate thread, via the
> LuceneSearchProvider.LuceneUpdater inner class, which is executed on a
> different thread. This may lead to situations where there are more than one
> IndexWriter instances working on the same directory, provoking the
> exception noted on the first email.
>
> The proposed fix is relatively easy, on
> LuceneSearchProvider.luceneIndexPage( WikiPage page, String text,
> IndexWriter writer ) method, if an update index operation fails, simply
> re-schedule the index operation:
>
> protected synchronized void updateLuceneIndex( WikiPage page, String text
> )
> {
> [...]
> try
> {
> [...]
> luceneIndexPage( page, text, writer );
> }
> catch ( IOException e )
> {
> log.error("Unable to update page '" + page.getName() + "' from
> Lucene index", e);
> reindexPage( page );
> }
> catch( Exception e )
> {
> log.error("Unexpected Lucene exception - please check
> configuration!",e);
> reindexPage( page );
> }
> finally
> [...]
>
> This approach could cause some sort of infinite loop if there's something
> severe happening (i.e.: disk full, no permission to write on the work dir,
> etc.), but I think that those kind of situations would be noted throughout
> the whole application, so there would be little to do on those cases
> anyway.
>
> WDYT about the suggested change?
>
>
> br,
> juan pablo
>
>
> On Thu, Jun 21, 2012 at 1:00 AM, Juan Pablo Santos Rodríguez <
> juanpablo.san...@gmail.com> wrote:
>
> > Hello Harry,
> >
> > I've increased the time up to 100s and still the same tests failing /
> > exceptions. I think it has to be something related to index updating, as
> > the IOExceptions noted by enabling writer.setInfoStream( System.out );
> > occur every time a page is updated.
> >
> > Also, the exceptions are caught always at
> > LuceneSearchProvider.LuceneUpdater threads so maybe they are trying to
> open
> > an IndexWriter while the JUnit test has his own IndexWriter opened.
> Anyway,
> > I'll continue doing more tests to see if I can isolate the cause of the
> > exception.
> >
> >
> > regards,
> > juan pablo
> >
> >
> >
> > On Wed, Jun 20, 2012 at 6:21 AM, Harry Metske <harry.met...@gmail.com
> >wrote:
> >
> >> Wat happens of you increase the waittime specified in the
> >> SearchManagerTest?
> >>
> >> I think I had these ones when this time was too short.
> >>
> >> regards,
> >> Harry
> >> Op 20 jun. 2012 00:45 schreef "Juan Pablo Santos Rodríguez" <
> >> juanpablo.san...@gmail.com> het volgende:
> >>
> >> > Hi,
> >> >
> >> > I'm stuck running the tests from SearchManagerTest. If I run them
> >> > individually, then all is OK.
> >> >
> >> > However, if I run them all together (via Ant task or via Eclipse),
> then
> >> the
> >> > following tests fail: testSimplesearch3, testTitleSearch,
> >> testTitleSearch2,
> >> > at the assertEquals() regarding res.size()
> >> >
> >> > jspwikitests.log yields several exceptions like this one:
> >> >
> >> > java.io.IOException: Cannot overwrite:
> E:\tmp\testworkdir\lucene\_0.fdt
> >> > at
> >> >
> org.apache.lucene.store.FSDirectory.ensureCanWrite(FSDirectory.java:316)
> >> > at
> >> > org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:305)
> >> > at
> org.apache.lucene.index.FieldsWriter.<init>(FieldsWriter.java:83)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:64)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:107)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:151)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1404)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1424)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1043)
> >> > at
> >> >
> >> >
> >>
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:772)
> >> > at
> >> > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2060)
> >> > at
> >> > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:2034)
> >> > at
> >> >
> >> >
> >>
> org.apache.wiki.search.LuceneSearchProvider.luceneIndexPage(LuceneSearchProvider.java:485)
> >> > at
> >> >
> >> >
> >>
> org.apache.wiki.search.LuceneSearchProvider.updateLuceneIndex(LuceneSearchProvider.java:377)
> >> > [...]
> >> >
> >> > Also, if I place writer.setInfoStream( System.out ); just after the
> >> writer
> >> > is instantiated, I get several messages similar to this one:
> >> > IFD [Wed Jun 20 00:20:07 CEST 2012; JSPWiki Lucene Indexer]: unable to
> >> > remove file "_0.fdt": java.io.IOException: Cannot delete
> >> > E:\tmp\testworkdir\lucene\_0.fdt; Will re-try later.
> >> >
> >> > Done a little googling and found that may be related to opening an
> >> > IndexWriter on the index while there's another one open [1] or
> removing
> >> > files while the index is open at that directory [2]. I'll continue
> >> looking
> >> > at this tomorrow, but I was wondering if anyone else is experiencing
> >> this
> >> > problem too.
> >> >
> >> >
> >> > br,
> >> > juan pablo
> >> >
> >> > [1]: http://www.gossamer-threads.com/lists/lucene/java-user/92262
> >> > [2]:
> http://www.gossamer-threads.com/lists/lucene/java-user/62201#62201
> >> >
> >>
> >
> >
>
