Hello Rohit, We had a similar query time bottleneck when attempting to map lucene's internal id's to the uniqueKey, especially as we generally return only the uniqueKey to the user we had no other use of the stored field. As you noted, every internal id --> uniqueKey id requires a disk seek and as in our use case the documents did not get cached most of these reads were faulting (random read miss). There are two patches (for Solr) at SOLR-5478, both make use of docValues for this need. First one caches the above mapping and a second is more general and retrieves that mapping (or any other field value) if exists in fieldCache. Since implementing the first patch (although not efficiently) we see a significant performance improvement.
I don't know if these patches can be easily ported to Lucene but I hope this use case helps you, Manuel On Sat, Mar 22, 2014 at 11:45 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > On Sat, Mar 22, 2014 at 5:18 AM, Rohit Banga <iamrohitba...@gmail.com> > wrote: > > > Awesome BinaryDocValues sounds nice! > > I saw that NumericDocValues did not inherit from a base class hence I > > thought there is no StringDocValues :). > > > > Can I expect that a searcher manager will invoke > > searcherfactory.newSearcher at most once between searcher manager > > refreshes? I believe IndexSearcher is threadsafe. Is my assumption that > > newSearcher is invoked only once correct? > > It's invoked once, for each refresh. > > > If BinaryDocValues didn't exist I was thinking of using a custom searcher > > factory which would return an instance of a custom subclass of > > IndexSearcher.This subclass could encapsulate a map from numeric doc > value > > to string. I was thinking SearcherManager.acquire could then be used to > > fetch the instance of this subclass while permitting concurrent updates > and > > reads to index and HashMap. > > Is using SearcherManager in this way appropriate? Just want to make sure > my > > understanding of how SearcherManager works is correct. > > I think in theory that would be fine (custom IndexSearcher subclass, > custom map), but hopefully BDV solves this in a simpler way! > > Mike McCandless > > http://blog.mikemccandless.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
