[
https://issues.apache.org/jira/browse/LUCENE-3786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13626519#comment-13626519
]
Michael McCandless commented on LUCENE-3786:
--------------------------------------------
OK I discussed these tricky issues with Shai ... with the non-NRT case
(app commits and then calls .maybeRefresh) there are some big
challenges.
First off, the app must always commit IW first then TW. But second
off, even if it does that, there is at least this multi-threaded case
where .maybeRefresh can screw up:
* Thread 1 (indexer) commits IW1
* Thread 1 (indexer) commits TW1
* Thread 2 (indexer) commits IW2
* Thread 3 (searcher) maybeRefresh opens IW2
* Thread 3 (searcher) maybeRefresh opens TW1
* Thread 1 (indexer) commits TW2
That will then lead to confusing AIOOBEs during facet counting...
Net/net I think there's too much hair around supporting the non-NRT
case, and I think for starters we should just support NRT, ie you must
pass IW and TW to STM's ctor. Then STM is agnostic to what commits
are being done ... commit is only for durability purposes.
We must still document that you cannot do IW.deleteAll /
TW.replaceTaxonomy (I'll add it).
bq. Why does the test uses newFSDirectory?
Just because it's using the LineFileDocs, which have biggish docs in
them. Add in -Dtests.nightly, -Dtests.multiplier=3, and it could
maybe be we are pushing the 512 MB RAM limit...
bq. Manager.decRef()-- I think you should searcher.reader.incRef() if
taxoReader.decRef() failed?
Hmm this isn't so simple: that decRef could have closed the reader. I
suppose I could do a "best effort" tryIncRef so that if the app
somehow catches the exception and retries the decRef we don't
prematurely close the reader ...
bq. It's odd that acquire() throws IOE ... I realize it's because the decRef
call in tryIncRef. I don't know if it's critical, but if it is, you may want to
throw RuntimeEx?
I think it's OK to add IOE to the signature?
> Create SearcherTaxoManager
> --------------------------
>
> Key: LUCENE-3786
> URL: https://issues.apache.org/jira/browse/LUCENE-3786
> Project: Lucene - Core
> Issue Type: New Feature
> Components: modules/facet
> Reporter: Shai Erera
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 5.0, 4.3
>
> Attachments: LUCENE-3786-3x-nocommit.patch, LUCENE-3786.patch
>
>
> If an application wants to use an IndexSearcher and TaxonomyReader in a
> SearcherManager-like fashion, it cannot use a separate SearcherManager, and
> say a TaxonomyReaderManager, because the IndexSearcher and TaxoReader
> instances need to be in sync. That is, the IS-TR pair must match, or
> otherwise the category ordinals that are encoded in the search index might
> not match the ones in the taxonomy index.
> This can happen if someone reopens the IndexSearcher's IndexReader, but does
> not refresh the TaxonomyReader, and the category ordinals that exist in the
> reopened IndexReader are not yet visible to the TaxonomyReader instance.
> I'd like to create a SearcherTaxoManager (which is a ReferenceManager) which
> manages an IndexSearcher and TaxonomyReader pair. Then an application will
> call:
> {code}
> SearcherTaxoPair pair = manager.acquire();
> try {
> IndexSearcher searcher = pair.searcher;
> TaxonomyReader taxoReader = pair.taxoReader;
> // do something with them
> } finally {
> manager.release(pair);
> pair = null;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
