Re: [EXTERNAL] - Re: Is docvalue sorted by value?

2018-03-07 Thread Adrien Grand
Giving some guidance with Lucene APIs since the original question was about Lucene: the APIs that you need to use are `IndexWriterConfig.setIndexSort` to configure the index sort at index-time and then TopFieldCollector takes a `boolean trackTotalHits` which you can set to `false` to tell Lucene

Re: [EXTERNAL] - Re: Is docvalue sorted by value?

2018-03-06 Thread Tony Ma
Thanks Erick!! Index sorting and early termination is what I am looking for. On 3/6/18, 11:33 PM, "Erick Erickson" wrote: OK, you're asking a different question I think. See SOLR-5730 and SOLR-8621, particularly SOLR-5730. This will work only a single

Re: [EXTERNAL] - Re: Is docvalue sorted by value?

2018-03-06 Thread Erick Erickson
OK, you're asking a different question I think. See SOLR-5730 and SOLR-8621, particularly SOLR-5730. This will work only a single field which you decide at index time. You can still sort by any field at the same expense as now, but since your docs are ordered by one field the early termination

Re: [EXTERNAL] - Re: Is docvalue sorted by value?

2018-03-05 Thread Tony Ma
Hi Erick, I raise this question is about the sorting scenario as you mentioned in #2. If the hit docs are about 100, and my query just want top 2. If the values are not sorted, it has to iterate all 100 docs and find top2 in a priority queue. If the values are already sorted, it just need to

Re: Is docvalue sorted by value?

2018-03-05 Thread Erick Erickson
I think there are two issues here that are being conflated 1> _within_ a document, i.e. for a multi-valued field the values are stored as Dominik says as a SORTED_SET. Not only will they be returned (if you return from docValues rather than stored) in lexical order, but identical values will be

Re: Is docvalue sorted by value?

2018-03-05 Thread Dominik Safaric
> So, can doc values be persisted with order of values, not document id? This > should be fast in sort scenario that the values are pre-ordered instead of > scan/sort at runtime. No, unfortunately doc values cannot be persisted in order. Lucene stores this values internally as a

Is docvalue sorted by value?

2018-03-05 Thread Tony Ma
Per my understanding, doc values (binary doc values / numeric doc values) are stored with sequence of document id. Sorted numeric doc values just means if a document has multiple values, the values will be sorted for same document, but for different documents, the value is still ordered by