Michael Sokolov created LUCENE-9652:
---------------------------------------

             Summary: DataInput.readFloats to be used by Lucene90VectorReader
                 Key: LUCENE-9652
                 URL: https://issues.apache.org/jira/browse/LUCENE-9652
             Project: Lucene - Core
          Issue Type: Improvement
            Reporter: Michael Sokolov


Benchmarking shows a substantial performance gain can be realized by avoiding 
the additional memory copy we must do today when converting from `byte[]` read 
using IndexInput into `float[]` returned by `Lucene90VectorReader`. We have a 
model for how to handle the various alignments, and buffer underflow when a 
value spans buffers, in `readLELongs`.

I think we should only support little-endian floats from the beginning here. 
We're planning to move towards switching the whole IndexInput to that 
endianness, right?

Lucene90VectorWriter relies on {VectorValues.binaryValue()} to return bytes in 
the format expected by the reader, and its javadocs don't currently specify 
their endianness. In fact the order has been the default supplied by 
{ByteBuffer.allocate(int)}, which I now realize is big-endian, so this issue 
also proposes to change the index format. That would mean a 
backwards-incompatible index change, but I think if we're still unreleased and 
in an experimental class that should be OK?

Also, we don't need a corresponding {DataOutput.writeFloats} to support the 
current usage for vectors, since there we rely on {VectorValues} to do the 
conversion, so I don't plan to implement that.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to