[ 
https://issues.apache.org/jira/browse/LUCENE-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12493656
 ] 

João Fonseca commented on LUCENE-874:
-------------------------------------

AutofreshedSearcher seems to be something like what I was proposing, but it 
seems to rely on the NotifiableIndex mechanism. Will this work if the task that 
updates the index is on another process/JVM?

I struggled with this problem on my own project, and came with a solution with 
points similar to Lucene-550 - a LuceneFactory, with factory methods for 
creating everything related with Lucene (IndexSearcher, IndexWriter, Analyzer, 
etc). The factory method for creating an IndexSearcher always returns the same 
instance, as advised by the Lucene javadocs.

The problem is when the index is modified (which, in my case, is done by an 
external process, from time to time). The IndexSearcher must be reopened. There 
are several issues to solve:

-How to test if the index was modified? That's easy: !IndexReader.isCurrent && 
!IndexReader.locked
-When to test if the index was modified? I test it on my 
LuceneFactory.getIndexSearcher() method, but only from time to time - it would 
be costly to test for every search that was made.
-The index was modified; how to close the current IndexSearcher? Other 
processes may be still using it, or using Hits generated by it. This is the 
hardest part to solve.
    * A reference count to the IndexSearcher must be maintained by the 
LuceneFactory, to know when all parties have finished searching.
    * To maintain a reference count, these parties must have a way to notify 
the factory that the search is finished.
    * Also, to maintain the reference count in a thread-safe manner, some 
locking must be used when getting and releasing the searcher (slow!)
    * How to wait for the reference count to reach 0? On another thread? 
Polling from time to time (on each LuceneFactory.getIndexSearcher call)?

As you can see, this is not trivial at all to do correctly - and so it should 
be implemented and given out-of-the-box with Lucene. Note that the above 
description uses Lucene as a black box, maybe it's easier to implement this 
inside the IndexSearcher class, by updating its internal structure when the 
index is changed.

Another way is to maintain one IndexSearcher per thread (with a ThreadLocal). 
The reopening of the IndexSearcher would be easier, but there would be several 
IndexSearchers on memory...






> Automatic reopen of IndexSearcher/IndexReader
> ---------------------------------------------
>
>                 Key: LUCENE-874
>                 URL: https://issues.apache.org/jira/browse/LUCENE-874
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>            Reporter: João Fonseca
>            Priority: Minor
>
> To improve performance, a single instance of IndexSearcher should be used. 
> However, if the index is updated, it's hard to close/reopen it, because 
> multiple threads may be accessing it at the same time.
> Lucene should include an out-of-the-box solution to this problem. Either a 
> new class should be implemented to manage this behaviour (singleton 
> IndexSearcher, plus detection of a modified index, plus safely closing and 
> reopening the IndexSearcher) or this could be behind the scenes by the 
> IndexSearcher class.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to