jpountz commented on code in PR #987:
URL: https://github.com/apache/lucene/pull/987#discussion_r924959065
##########
lucene/core/src/java/org/apache/lucene/store/ByteBuffersDataInput.java:
##########
@@ -165,6 +165,36 @@ public void readBytes(byte[] arr, int off, int len) throws
EOFException {
}
}
}
+ /**
+ * ReadBytes from position with length, if [pos, pos + len] stay in one
ByteBuffer can ignore
+ * memory copy, otherwise return a new ByteBuffer with continuous byte array
+ *
+ * @param length from position to length
+ * @return ByteBuffer which bytes read from [pos, pos + length]
+ */
+ public ByteBuffer readBytes(int length) throws EOFException {
Review Comment:
I wonder if it should be called `readNBytes` for consistency with
`InputStream#readNBytes`. Maybe @uschindler has thoughts on this and on the
implementation.
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene90/DeflateWithPresetDictCompressionMode.java:
##########
@@ -163,20 +165,22 @@ private static class DeflateWithPresetDictCompressor
extends Compressor {
final Deflater compressor;
final BugfixDeflater_JDK8252739 deflaterBugfix;
byte[] compressed;
+ byte[] bufferDict;
boolean closed;
DeflateWithPresetDictCompressor(int level) {
compressor = new Deflater(level, true);
deflaterBugfix = BugfixDeflater_JDK8252739.createBugfix(compressor);
compressed = new byte[64];
+ bufferDict = BytesRef.EMPTY_BYTES;
}
- private void doCompress(byte[] bytes, int off, int len, DataOutput out)
throws IOException {
+ private void doCompress(ByteBuffer bytes, int len, DataOutput out) throws
IOException {
Review Comment:
Can we remove the `len` parameter?
##########
lucene/core/src/java/org/apache/lucene/codecs/lucene90/DeflateWithPresetDictCompressionMode.java:
##########
@@ -198,22 +200,25 @@ private void doCompress(byte[] bytes, int off, int len,
DataOutput out) throws I
}
@Override
- public void compress(byte[] bytes, int off, int len, DataOutput out)
throws IOException {
+ public void compress(ByteBuffersDataInput buffersInput, DataOutput out)
throws IOException {
+ final int len = (int) (buffersInput.size() - buffersInput.position());
Review Comment:
Should position() always be 0?
##########
lucene/core/src/test/org/apache/lucene/codecs/lucene90/compressing/TestCompressingStoredFieldsFormat.java:
##########
@@ -325,4 +341,160 @@ public void testChunkCleanup() throws IOException {
iw.close();
dir.close();
}
+
+ public void testSortedSetVariableLengthBigStoredFields() throws Exception {
Review Comment:
did you add this because the doc values test case found bugs in stored
fields?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
