Re: Storing and retrieving Java objects in Lucene

[ganesh m]

Hi Santosh

>>Furthermore converting the Lucene Documents to Java object and vice- 
versa is a tedious task.


This should not be tedious, how big your document is?

One suggestion is to convert your Java object to JSON and store it in 
Lucene. You need to retrieve one field and you can easily convert back 
to object.


Regards
Ganesh

On 20-02-2018 08:34, Kumar, Santosh wrote:

Hi,

I have a requirement to store a Java object with multiple fields into the 
Lucene index. Basically, at the application startup I run a  select query on 
entities ( there are 5 of them as of now and may increase in future) and then 
create an index for each of these entities (5) i.e. five different indexes as 
of now(cannot have a common index. Need separation of entity data).   Ideally I 
would have liked to store only primary key field, but I need rest of the fields 
upon fetch.
I use this index(basically only the primary key field) to prevent users from 
creating duplicate entities or suggest them like a Did you mean(Google) ? 
feature . For this purpose, I’m using SpellChecker module to suggest entities 
or identify duplicates. Since, Spell checker only returns a String array, I 
again have to run a select separate search on the index(QueryParser search) or 
run select on the DB to fetch the entire object. Furthermore converting the 
Lucene Documents to Java object and vice- versa is a tedious task. Is there any 
API or library that can simplify this task ?  I have heard of Compass API, but 
not sure if it is still recommended. Any examples of the same or APIs will be 
appreciated. Thank you !!!


Thank you and Regards,
Santosh



-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Storing and retrieving Java objects in Lucene

[Kumar, Santosh]

Hi,

I have a requirement to store a Java object with multiple fields into the 
Lucene index. Basically, at the application startup I run a  select query on 
entities ( there are 5 of them as of now and may increase in future) and then 
create an index for each of these entities (5) i.e. five different indexes as 
of now(cannot have a common index. Need separation of entity data).   Ideally I 
would have liked to store only primary key field, but I need rest of the fields 
upon fetch.
I use this index(basically only the primary key field) to prevent users from 
creating duplicate entities or suggest them like a Did you mean(Google) ? 
feature . For this purpose, I’m using SpellChecker module to suggest entities 
or identify duplicates. Since, Spell checker only returns a String array, I 
again have to run a select separate search on the index(QueryParser search) or 
run select on the DB to fetch the entire object. Furthermore converting the 
Lucene Documents to Java object and vice- versa is a tedious task. Is there any 
API or library that can simplify this task ?  I have heard of Compass API, but 
not sure if it is still recommended. Any examples of the same or APIs will be 
appreciated. Thank you !!!


Thank you and Regards,
Santosh

Re: Storing and retrieving Java objects in Lucene

[Kumar, Santosh]

Hi Ganesh,  Thank you for quick response.
Most of these documents will have more than 10 fields. In some cases there is 
also a foreign key relationship. Will consider JSON. Also need to consider 
performance factor.

Thank you and Regards,
Santosh


On 20/02/18, 10:00 AM, "ganesh m"  wrote:

Hi Santosh

 >>Furthermore converting the Lucene Documents to Java object and vice- 
versa is a tedious task.

This should not be tedious, how big your document is?

One suggestion is to convert your Java object to JSON and store it in 
Lucene. You need to retrieve one field and you can easily convert back 
to object.

Regards
Ganesh

On 20-02-2018 08:34, Kumar, Santosh wrote:
> Hi,
>
> I have a requirement to store a Java object with multiple fields into the 
Lucene index. Basically, at the application startup I run a  select query on 
entities ( there are 5 of them as of now and may increase in future) and then 
create an index for each of these entities (5) i.e. five different indexes as 
of now(cannot have a common index. Need separation of entity data).   Ideally I 
would have liked to store only primary key field, but I need rest of the fields 
upon fetch.
> I use this index(basically only the primary key field) to prevent users 
from creating duplicate entities or suggest them like a Did you mean(Google) ? 
feature . For this purpose, I’m using SpellChecker module to suggest entities 
or identify duplicates. Since, Spell checker only returns a String array, I 
again have to run a select separate search on the index(QueryParser search) or 
run select on the DB to fetch the entire object. Furthermore converting the 
Lucene Documents to Java object and vice- versa is a tedious task. Is there any 
API or library that can simplify this task ?  I have heard of Compass API, but 
not sure if it is still recommended. Any examples of the same or APIs will be 
appreciated. Thank you !!!
>
>
> Thank you and Regards,
> Santosh

Read DocValue twice

[Vadim Gindin]

Hi all

I use DocValue for scoring function. I.e. I have some column with integers,
that are used in scoring formula. So I have a scorer that calculates
scoring function twice:
- in score()
- in explain()

I got the following error in explain:

Caused by: java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:540) ~[?:1.8.0_161]
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
~[?:1.8.0_161]
at
org.apache.lucene.store.ByteBufferGuard.getByte(ByteBufferGuard.java:118)
~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
ubuntu - 2017-10-13 16:12:42]
at
org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readByte(ByteBufferIndexInput.java:385)
~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
ubuntu - 2017-10-13 16:12:42]
at
org.apache.lucene.util.packed.DirectReader$DirectPackedReader8.get(DirectReader.java:145)
~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
ubuntu - 2017-10-13 16:12:42]
at
org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$3.longValue(Lucene70DocValuesProducer.java:481)
~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
ubuntu - 2017-10-13 16:12:42]
at
org.apache.lucene.index.SingletonSortedNumericDocValues.nextValue(SingletonSortedNumericDocValues.java:73)
~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
ubuntu - 2017-10-13 16:12:42]

I've found the following comment in the source code of
SortedNumericDocValues.java:

/**
 * Iterates to the next value in the current document.  Do not call
this more than {@link #docValueCount} times
 * for the document.
 */

public abstract long nextValue() throws IOException;


Questions:
1) Why I can't read the values twice?
2) How can I manage this situation?
3) Can it work for NumericDocValues?

Regards,
Vadim Gindin

Re: Read DocValue twice

[Adrien Grand]

Can you add some debug logging to see what the values of topList.doc and
reader.maxDoc() are before before you call advanceExact?

What do you mean by "I reuse the same DisiPriorityQueue of scorers in
score() and explain()". This shouldn't be possible.

Le lun. 19 févr. 2018 à 15:23, Vadim Gindin  a écrit :

> I use these calls in both cases. In score() and explain() I have the
> following code:
>
> SortedNumericDocValues numDocVal = DocValues.getSortedNumeric(reader,
> fieldName);
> if (numDocVal != null && numDocVal.advanceExact(topList.doc)) {
> long val = numDocVal.nextValue();
>
> ..
> }
>
> I reuse the same DisiPriorityQueue of scorers in score() and explain().
>
> On Mon, Feb 19, 2018 at 6:54 PM, Adrien Grand  wrote:
>
> > If you want to read the values again, you need to call setDocument
> (Lucene
> > < 7.0) or advanceExact (Lucene >= 7.0) before calling nextValue().
> >
> > Le lun. 19 févr. 2018 à 14:41, Vadim Gindin  a
> > écrit :
> >
> > > Hi all
> > >
> > > I use DocValue for scoring function. I.e. I have some column with
> > integers,
> > > that are used in scoring formula. So I have a scorer that calculates
> > > scoring function twice:
> > > - in score()
> > > - in explain()
> > >
> > > I got the following error in explain:
> > >
> > > Caused by: java.lang.IndexOutOfBoundsException
> > > at java.nio.Buffer.checkIndex(Buffer.java:540) ~[?:1.8.0_161]
> > > at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
> > > ~[?:1.8.0_161]
> > > at
> > > org.apache.lucene.store.ByteBufferGuard.getByte(
> > ByteBufferGuard.java:118)
> > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659
> -
> > > ubuntu - 2017-10-13 16:12:42]
> > > at
> > >
> > > org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readByte(
> > ByteBufferIndexInput.java:385)
> > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659
> -
> > > ubuntu - 2017-10-13 16:12:42]
> > > at
> > >
> > > org.apache.lucene.util.packed.DirectReader$DirectPackedReader8.get(
> > DirectReader.java:145)
> > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659
> -
> > > ubuntu - 2017-10-13 16:12:42]
> > > at
> > >
> > >
> org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$3.longValue(
> > Lucene70DocValuesProducer.java:481)
> > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659
> -
> > > ubuntu - 2017-10-13 16:12:42]
> > > at
> > >
> > > org.apache.lucene.index.SingletonSortedNumericDocValues.nextValue(
> > SingletonSortedNumericDocValues.java:73)
> > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659
> -
> > > ubuntu - 2017-10-13 16:12:42]
> > >
> > > I've found the following comment in the source code of
> > > SortedNumericDocValues.java:
> > >
> > > /**
> > >  * Iterates to the next value in the current document.  Do not call
> > > this more than {@link #docValueCount} times
> > >  * for the document.
> > >  */
> > >
> > > public abstract long nextValue() throws IOException;
> > >
> > >
> > > Questions:
> > > 1) Why I can't read the values twice?
> > > 2) How can I manage this situation?
> > > 3) Can it work for NumericDocValues?
> > >
> > > Regards,
> > > Vadim Gindin
> > >
> >
>

Re: Read DocValue twice

[Vadim Gindin]

I have the scorer that is similar to DisjunctionScorer.java with

private final DisiPriorityQueue subScorers;
private final DisjunctionDISIApproximation approximation;

They are initialized in a constructor like that:

   this.subScorers = new DisiPriorityQueue(subScorers.size());
   for (Scorer scorer : subScorers) {
   final DisiWrapper w = new DisiWrapper(scorer);
   this.subScorers.add(w);
   }
   this.approximation = new DisjunctionDISIApproximation(this.subScorers);



I use them in score() and in explain(). In explain() I do

   this.approximation.advance(doc);

And further the same code as in score(). I've also added logging. And
here is the one string:

explain: doc=2147483647, field=params, maxDoc=67649

doc looks not so good..


On Mon, Feb 19, 2018 at 7:32 PM, Adrien Grand  wrote:

> Can you add some debug logging to see what the values of topList.doc and
> reader.maxDoc() are before before you call advanceExact?
>
> What do you mean by "I reuse the same DisiPriorityQueue of scorers in
> score() and explain()". This shouldn't be possible.
>
> Le lun. 19 févr. 2018 à 15:23, Vadim Gindin  a
> écrit :
>
> > I use these calls in both cases. In score() and explain() I have the
> > following code:
> >
> > SortedNumericDocValues numDocVal = DocValues.getSortedNumeric(reader,
> > fieldName);
> > if (numDocVal != null && numDocVal.advanceExact(topList.doc)) {
> > long val = numDocVal.nextValue();
> >
> > ..
> > }
> >
> > I reuse the same DisiPriorityQueue of scorers in score() and explain().
> >
> > On Mon, Feb 19, 2018 at 6:54 PM, Adrien Grand  wrote:
> >
> > > If you want to read the values again, you need to call setDocument
> > (Lucene
> > > < 7.0) or advanceExact (Lucene >= 7.0) before calling nextValue().
> > >
> > > Le lun. 19 févr. 2018 à 14:41, Vadim Gindin  a
> > > écrit :
> > >
> > > > Hi all
> > > >
> > > > I use DocValue for scoring function. I.e. I have some column with
> > > integers,
> > > > that are used in scoring formula. So I have a scorer that calculates
> > > > scoring function twice:
> > > > - in score()
> > > > - in explain()
> > > >
> > > > I got the following error in explain:
> > > >
> > > > Caused by: java.lang.IndexOutOfBoundsException
> > > > at java.nio.Buffer.checkIndex(Buffer.java:540)
> ~[?:1.8.0_161]
> > > > at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
> > > > ~[?:1.8.0_161]
> > > > at
> > > > org.apache.lucene.store.ByteBufferGuard.getByte(
> > > ByteBufferGuard.java:118)
> > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> 8a38550659
> > -
> > > > ubuntu - 2017-10-13 16:12:42]
> > > > at
> > > >
> > > > org.apache.lucene.store.ByteBufferIndexInput$
> SingleBufferImpl.readByte(
> > > ByteBufferIndexInput.java:385)
> > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> 8a38550659
> > -
> > > > ubuntu - 2017-10-13 16:12:42]
> > > > at
> > > >
> > > > org.apache.lucene.util.packed.DirectReader$DirectPackedReader8.get(
> > > DirectReader.java:145)
> > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> 8a38550659
> > -
> > > > ubuntu - 2017-10-13 16:12:42]
> > > > at
> > > >
> > > >
> > org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$3.longValue(
> > > Lucene70DocValuesProducer.java:481)
> > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> 8a38550659
> > -
> > > > ubuntu - 2017-10-13 16:12:42]
> > > > at
> > > >
> > > > org.apache.lucene.index.SingletonSortedNumericDocValues.nextValue(
> > > SingletonSortedNumericDocValues.java:73)
> > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> 8a38550659
> > -
> > > > ubuntu - 2017-10-13 16:12:42]
> > > >
> > > > I've found the following comment in the source code of
> > > > SortedNumericDocValues.java:
> > > >
> > > > /**
> > > >  * Iterates to the next value in the current document.  Do not call
> > > > this more than {@link #docValueCount} times
> > > >  * for the document.
> > > >  */
> > > >
> > > > public abstract long nextValue() throws IOException;
> > > >
> > > >
> > > > Questions:
> > > > 1) Why I can't read the values twice?
> > > > 2) How can I manage this situation?
> > > > 3) Can it work for NumericDocValues?
> > > >
> > > > Regards,
> > > > Vadim Gindin
> > > >
> > >
> >
>

Re: Read DocValue twice

[Vadim Gindin]

I use these calls in both cases. In score() and explain() I have the
following code:

SortedNumericDocValues numDocVal = DocValues.getSortedNumeric(reader,
fieldName);
if (numDocVal != null && numDocVal.advanceExact(topList.doc)) {
long val = numDocVal.nextValue();

..
}

I reuse the same DisiPriorityQueue of scorers in score() and explain().

On Mon, Feb 19, 2018 at 6:54 PM, Adrien Grand  wrote:

> If you want to read the values again, you need to call setDocument (Lucene
> < 7.0) or advanceExact (Lucene >= 7.0) before calling nextValue().
>
> Le lun. 19 févr. 2018 à 14:41, Vadim Gindin  a
> écrit :
>
> > Hi all
> >
> > I use DocValue for scoring function. I.e. I have some column with
> integers,
> > that are used in scoring formula. So I have a scorer that calculates
> > scoring function twice:
> > - in score()
> > - in explain()
> >
> > I got the following error in explain:
> >
> > Caused by: java.lang.IndexOutOfBoundsException
> > at java.nio.Buffer.checkIndex(Buffer.java:540) ~[?:1.8.0_161]
> > at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
> > ~[?:1.8.0_161]
> > at
> > org.apache.lucene.store.ByteBufferGuard.getByte(
> ByteBufferGuard.java:118)
> > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> > ubuntu - 2017-10-13 16:12:42]
> > at
> >
> > org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readByte(
> ByteBufferIndexInput.java:385)
> > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> > ubuntu - 2017-10-13 16:12:42]
> > at
> >
> > org.apache.lucene.util.packed.DirectReader$DirectPackedReader8.get(
> DirectReader.java:145)
> > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> > ubuntu - 2017-10-13 16:12:42]
> > at
> >
> > org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$3.longValue(
> Lucene70DocValuesProducer.java:481)
> > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> > ubuntu - 2017-10-13 16:12:42]
> > at
> >
> > org.apache.lucene.index.SingletonSortedNumericDocValues.nextValue(
> SingletonSortedNumericDocValues.java:73)
> > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> > ubuntu - 2017-10-13 16:12:42]
> >
> > I've found the following comment in the source code of
> > SortedNumericDocValues.java:
> >
> > /**
> >  * Iterates to the next value in the current document.  Do not call
> > this more than {@link #docValueCount} times
> >  * for the document.
> >  */
> >
> > public abstract long nextValue() throws IOException;
> >
> >
> > Questions:
> > 1) Why I can't read the values twice?
> > 2) How can I manage this situation?
> > 3) Can it work for NumericDocValues?
> >
> > Regards,
> > Vadim Gindin
> >
>

Re: Read DocValue twice

[Adrien Grand]

If you want to read the values again, you need to call setDocument (Lucene
< 7.0) or advanceExact (Lucene >= 7.0) before calling nextValue().

Le lun. 19 févr. 2018 à 14:41, Vadim Gindin  a écrit :

> Hi all
>
> I use DocValue for scoring function. I.e. I have some column with integers,
> that are used in scoring formula. So I have a scorer that calculates
> scoring function twice:
> - in score()
> - in explain()
>
> I got the following error in explain:
>
> Caused by: java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkIndex(Buffer.java:540) ~[?:1.8.0_161]
> at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
> ~[?:1.8.0_161]
> at
> org.apache.lucene.store.ByteBufferGuard.getByte(ByteBufferGuard.java:118)
> ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> ubuntu - 2017-10-13 16:12:42]
> at
>
> org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readByte(ByteBufferIndexInput.java:385)
> ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> ubuntu - 2017-10-13 16:12:42]
> at
>
> org.apache.lucene.util.packed.DirectReader$DirectPackedReader8.get(DirectReader.java:145)
> ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> ubuntu - 2017-10-13 16:12:42]
> at
>
> org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$3.longValue(Lucene70DocValuesProducer.java:481)
> ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> ubuntu - 2017-10-13 16:12:42]
> at
>
> org.apache.lucene.index.SingletonSortedNumericDocValues.nextValue(SingletonSortedNumericDocValues.java:73)
> ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b8a38550659 -
> ubuntu - 2017-10-13 16:12:42]
>
> I've found the following comment in the source code of
> SortedNumericDocValues.java:
>
> /**
>  * Iterates to the next value in the current document.  Do not call
> this more than {@link #docValueCount} times
>  * for the document.
>  */
>
> public abstract long nextValue() throws IOException;
>
>
> Questions:
> 1) Why I can't read the values twice?
> 2) How can I manage this situation?
> 3) Can it work for NumericDocValues?
>
> Regards,
> Vadim Gindin
>

Re: Read DocValue twice

[Adrien Grand]

Yes, this is the problem. This doc ID is a special sentinel value that
means that the iterator is exhausted. I don't have enough context to know
what the exact problem is but there is a bug in your custom query.

Le lun. 19 févr. 2018 à 16:07, Vadim Gindin  a écrit :

> I have the scorer that is similar to DisjunctionScorer.java with
>
> private final DisiPriorityQueue subScorers;
> private final DisjunctionDISIApproximation approximation;
>
> They are initialized in a constructor like that:
>
>this.subScorers = new DisiPriorityQueue(subScorers.size());
>for (Scorer scorer : subScorers) {
>final DisiWrapper w = new DisiWrapper(scorer);
>this.subScorers.add(w);
>}
>this.approximation = new DisjunctionDISIApproximation(this.subScorers);
>
>
>
> I use them in score() and in explain(). In explain() I do
>
>this.approximation.advance(doc);
>
> And further the same code as in score(). I've also added logging. And
> here is the one string:
>
> explain: doc=2147483647 <(214)%20748-3647>, field=params, maxDoc=67649
>
> doc looks not so good..
>
>
> On Mon, Feb 19, 2018 at 7:32 PM, Adrien Grand  wrote:
>
> > Can you add some debug logging to see what the values of topList.doc and
> > reader.maxDoc() are before before you call advanceExact?
> >
> > What do you mean by "I reuse the same DisiPriorityQueue of scorers in
> > score() and explain()". This shouldn't be possible.
> >
> > Le lun. 19 févr. 2018 à 15:23, Vadim Gindin  a
> > écrit :
> >
> > > I use these calls in both cases. In score() and explain() I have the
> > > following code:
> > >
> > > SortedNumericDocValues numDocVal = DocValues.getSortedNumeric(reader,
> > > fieldName);
> > > if (numDocVal != null && numDocVal.advanceExact(topList.doc)) {
> > > long val = numDocVal.nextValue();
> > >
> > > ..
> > > }
> > >
> > > I reuse the same DisiPriorityQueue of scorers in score() and explain().
> > >
> > > On Mon, Feb 19, 2018 at 6:54 PM, Adrien Grand 
> wrote:
> > >
> > > > If you want to read the values again, you need to call setDocument
> > > (Lucene
> > > > < 7.0) or advanceExact (Lucene >= 7.0) before calling nextValue().
> > > >
> > > > Le lun. 19 févr. 2018 à 14:41, Vadim Gindin  a
> > > > écrit :
> > > >
> > > > > Hi all
> > > > >
> > > > > I use DocValue for scoring function. I.e. I have some column with
> > > > integers,
> > > > > that are used in scoring formula. So I have a scorer that
> calculates
> > > > > scoring function twice:
> > > > > - in score()
> > > > > - in explain()
> > > > >
> > > > > I got the following error in explain:
> > > > >
> > > > > Caused by: java.lang.IndexOutOfBoundsException
> > > > > at java.nio.Buffer.checkIndex(Buffer.java:540)
> > ~[?:1.8.0_161]
> > > > > at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:253)
> > > > > ~[?:1.8.0_161]
> > > > > at
> > > > > org.apache.lucene.store.ByteBufferGuard.getByte(
> > > > ByteBufferGuard.java:118)
> > > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> > 8a38550659
> > > -
> > > > > ubuntu - 2017-10-13 16:12:42]
> > > > > at
> > > > >
> > > > > org.apache.lucene.store.ByteBufferIndexInput$
> > SingleBufferImpl.readByte(
> > > > ByteBufferIndexInput.java:385)
> > > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> > 8a38550659
> > > -
> > > > > ubuntu - 2017-10-13 16:12:42]
> > > > > at
> > > > >
> > > > > org.apache.lucene.util.packed.DirectReader$DirectPackedReader8.get(
> > > > DirectReader.java:145)
> > > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> > 8a38550659
> > > -
> > > > > ubuntu - 2017-10-13 16:12:42]
> > > > > at
> > > > >
> > > > >
> > >
> org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$3.longValue(
> > > > Lucene70DocValuesProducer.java:481)
> > > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> > 8a38550659
> > > -
> > > > > ubuntu - 2017-10-13 16:12:42]
> > > > > at
> > > > >
> > > > > org.apache.lucene.index.SingletonSortedNumericDocValues.nextValue(
> > > > SingletonSortedNumericDocValues.java:73)
> > > > > ~[lucene-core-7.1.0.jar:7.1.0 84c90ad2c0218156c840e19a64d72b
> > 8a38550659
> > > -
> > > > > ubuntu - 2017-10-13 16:12:42]
> > > > >
> > > > > I've found the following comment in the source code of
> > > > > SortedNumericDocValues.java:
> > > > >
> > > > > /**
> > > > >  * Iterates to the next value in the current document.  Do not call
> > > > > this more than {@link #docValueCount} times
> > > > >  * for the document.
> > > > >  */
> > > > >
> > > > > public abstract long nextValue() throws IOException;
> > > > >
> > > > >
> > > > > Questions:
> > > > > 1) Why I can't read the values twice?
> > > > > 2) How can I manage this situation?
> > > > > 3) Can it work for NumericDocValues?
> > > > >
> > > > > Regards,
> > > > > Vadim Gindin
> > > > >
> > > >
> > >
> >
>