Re: Question about Payloads in Lucene 4.5

2014-03-27 Thread Rohit Banga
Awesome works well for me! Thanks Rohit Banga http://iamrohitbanga.com/ On Sun, Mar 23, 2014 at 10:06 PM, Manuel Le Normand < manuel.lenorm...@gmail.com> wrote: > Hello Rohit, > We had a similar query time bottleneck when attempting to map lucene's > internal id's to the uniqueKey, especially a

Re: Question about Payloads in Lucene 4.5

2014-03-23 Thread Manuel Le Normand
Hello Rohit, We had a similar query time bottleneck when attempting to map lucene's internal id's to the uniqueKey, especially as we generally return only the uniqueKey to the user we had no other use of the stored field. As you noted, every internal id --> uniqueKey id requires a disk seek and as

Re: Question about Payloads in Lucene 4.5

2014-03-22 Thread Michael McCandless
On Sat, Mar 22, 2014 at 5:18 AM, Rohit Banga wrote: > Awesome BinaryDocValues sounds nice! > I saw that NumericDocValues did not inherit from a base class hence I > thought there is no StringDocValues :). > > Can I expect that a searcher manager will invoke > searcherfactory.newSearcher at most o

Re: Question about Payloads in Lucene 4.5

2014-03-22 Thread Rohit Banga
Awesome BinaryDocValues sounds nice! I saw that NumericDocValues did not inherit from a base class hence I thought there is no StringDocValues :). Can I expect that a searcher manager will invoke searcherfactory.newSearcher at most once between searcher manager refreshes? I believe IndexSearcher i

Re: Question about Payloads in Lucene 4.5

2014-03-22 Thread Michael McCandless
On Fri, Mar 21, 2014 at 10:25 PM, Rohit Banga wrote: > Thanks Michael for your response. You're welcome! > Few questions: > > 1. Can I expect better performance when retrieving a single NumericDocValue > for all hits vs when I retrieve documents for all hits to fetch the field > value? As far as

Re: Question about Payloads in Lucene 4.5

2014-03-21 Thread Rohit Banga
​Just saw the implementation of MultiDocValues.getNumericValues(). It uses sort of returns an anonymous inner classes to get the doc value from the appropriate index reader. Very cool impleentation! I guess that answers my question on how to get docVal from multiple​ ​ atomic readers. It would be

Re: Question about Payloads in Lucene 4.5

2014-03-21 Thread Rohit Banga
​Thanks Michael for your response. Few questions: 1. Can I expect better performance when retrieving a single NumericDocValue for all hits vs when I retrieve documents for all hits to fetch the field value? As far as I understand retrieving n documents from the index requires n disk reads. How ma

Re: Question about Payloads in Lucene 4.5

2014-03-21 Thread Michael McCandless
DocValues are better than payloads. E.g. index a NumericDocValuesField with each doc, holding your id. Then at search time you can use MultiDocValues.getNumericValues. Mike McCandless http://blog.mikemccandless.com On Fri, Mar 21, 2014 at 4:35 PM, Rohit Banga wrote: > Hi everyone > > When I

Question about Payloads in Lucene 4.5

2014-03-21 Thread Rohit Banga
Hi everyone When I query a lucene index, I get back a list of document ids. This index search is fast. Now for all documents matching the result I need a unique String field called "id" which is stored in the document. From the documentation I gather that document ids are internal and I should not