easyice commented on PR #12842:
URL: https://github.com/apache/lucene/pull/12842#issuecomment-1858130375
> i'll test it with #12841 optimized code tomorrow.
emmm... there's still no significant performance improvement, possibly the
write path `writeGroupVInts` is a bit slower than `writeVInt`, because
group-varint needs write to a buffer per group at first(since it needs to
compute the length for every integer, and finally combined into flag), then
copy it to output. so it does an extra memory copy. secondly, the
`BytesRefBuilder#append` method may also have some overhead. thinking about
other approach...
JMH output:
```
Benchmark (size) Mode
Cnt Score Error Units
GroupVIntBenchmark.benchByteBuffersIndexInput_writeGroupVInt 64 thrpt
5 1.108 ± 0.624 ops/us
GroupVIntBenchmark.benchByteBuffersIndexInput_writeVInt 64 thrpt
5 1.603 ± 0.312 ops/us
```
<details>
<summary >JMH benchmark code for write</summary>
```java
public class GroupVIntBenchmark {
// Cumulative frequency for each number of bits per value used by doc
deltas of tail postings on
// wikibigall.
private static final float[] CUMULATIVE_FREQUENCY_BY_BITS_REQUIRED =
new float[] {
0.0f,
0.01026574f,
0.021453038f,
0.03342156f,
0.046476692f,
0.060890317f,
0.07644147f,
0.093718216f,
0.11424741f,
0.13989712f,
0.17366524f,
0.22071244f,
0.2815692f,
0.3537585f,
0.43655503f,
0.52308f,
0.6104675f,
0.7047371f,
0.78155357f,
0.8671179f,
0.9740598f,
1.0f
};
final int maxSize = 256;
final long[] docs = new long[maxSize];
// benchmark for write
final ByteBuffersDataOutput byteBuffersDataOutput = new
ByteBuffersDataOutput();
@Param({"64"})
public int size;
@Setup(Level.Trial)
public void init() throws Exception {
Random r = new Random(0);
for (int i = 0; i < maxSize; ++i) {
float randomFloat = r.nextFloat();
// Reproduce the distribution of the number of bits per values that
we're observing for tail
// postings on wikibigall.
int numBits = 1 +
Arrays.binarySearch(CUMULATIVE_FREQUENCY_BY_BITS_REQUIRED, randomFloat);
if (numBits < 0) {
numBits = -numBits;
}
docs[i] = r.nextInt(1 << (numBits - 1), 1 << numBits);
}
}
@Benchmark
public void benchByteBuffersIndexInput_writeGroupVInt(Blackhole bh) throws
IOException {
byteBuffersDataOutput.reset();
byteBuffersDataOutput.writeGroupVInts(docs, size);
}
@Benchmark
public void benchByteBuffersIndexInput_writeVInt(Blackhole bh) throws
IOException {
byteBuffersDataOutput.reset();
for (int i = 0; i < size; i++) {
byteBuffersDataOutput.writeVInt((int)docs[i]);
}
}
}
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
