Re: Using setIndexSort on a binary field

2021-10-18 Thread Alex K
Thanks Michael. Totally agree this is a contrived setup. It's mostly for benchmarking purposes right now. I was actually able to rephrase my problem in a way that made more sense for the existing setIndexSort API using float doc values and saw an appreciable speedup in searches. The

Re: Using setIndexSort on a binary field

2021-10-17 Thread Michael Sokolov
Yeah, index sorting doesn't do that -- it sorts *within* each segment so that when documents are iterated (within that segment) by any of the many DocIdSetIterators that underlie the Lucene search API, they are retrieved in the order specified (which is then also docid order). To achieve what you

Re: Using setIndexSort on a binary field

2021-10-15 Thread Alex K
Thanks Adrien. This makes me think I might not be understanding the use case for index sorting correctly. I basically want to make it so that my terms are sorted across segments. For example, let's say I have integer terms 1 to 100 and 10 segments. I'd like terms 1 to 10 to occur in segment 1,

Re: Using setIndexSort on a binary field

2021-10-15 Thread Adrien Grand
Hi Alex, You need to use a BinaryDocValuesField so that the field is indexed with doc values. `Field` is not going to work because it only indexes the data while index sorting requires doc values. On Fri, Oct 15, 2021 at 6:40 PM Alex K wrote: > Hi all, > > Could someone point me to an example

Using setIndexSort on a binary field

2021-10-15 Thread Alex K
Hi all, Could someone point me to an example of using the IndexWriterConfig.setIndexSort for a field containing binary values? To be specific, the fields are constructed using the Field(String name, byte[] value, IndexableFieldType type) constructor, and I'd like to try using the