I'm not sure I'm following what you're doing with those locks
You can safely access one index searcher from multiple threads, but
IndexWriter only from one writing thread. The thread accessing IW can work
in parallel to searching threads.
On Mon, Aug 1, 2011 at 10:29 PM, Andrew Victor <avictor...@gmail.com> wrote:
> hi,
>
> We have quite a big index, but each document is pretty small. (So +-
> 30 million documents in 3.2 Gb),
>
> In our software, all updates are via a one IndexWriter, and we have
> multiple threads using a single IndexSearcher.
> What I'm noticing that when a 2nd IndexSearcher is opened, the
> existing IndexSearcher occasionally blocks for a couple of seconds (+-
> 20-40).
>
> We pull updated and new entries (10000 - 40000) into the index every 5
> minutes.
> For each:
> indexWriter->deleteDocuments( new Term( "a stored / untokenized key
> field" ) )
> indexWriter->addDocument()
>
> Once the update is complete:
> indexWriter->flush()
>
> /* re-open the IndexSearcher to make the changes visible */
> newSearcher = new IndexSearcher("index directory") /* *** */
>
> acquire_write_lock...
> oldSearcher = indexSearcher;
> indexSearcher = newSearcher;
> release_write_lock..
>
> /* cleanup */
> oldSearcher->close();
> delete oldSearcher;
>
> The searching threads that are "blocked" out of the index are just
> performing:
> acquire_read_lock....
> indexSearcher->search(BooleanQuery)
> /* process hits .... */
> release_read_lock
>
>
> Is there a way to stop a "new IndexSearcher()" locking the index?
> Or any suggestions to limit the amount of time the index is locked, or
> better ways of implementing this?
>
>
> Regards,
> Andrew Victor
>
>
> ------------------------------------------------------------------------------
> BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
> The must-attend event for mobile developers. Connect with experts.
> Get tools for creating Super Apps. See the latest technologies.
> Sessions, hands-on labs, demos & much more. Register early & save!
> http://p.sf.net/sfu/rim-blackberry-1
> _______________________________________________
> CLucene-developers mailing list
> CLucene-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/clucene-developers
>
>
------------------------------------------------------------------------------
BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts.
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
CLucene-developers mailing list
CLucene-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/clucene-developers
