clintropolis commented on a change in pull request #11004:
URL: https://github.com/apache/druid/pull/11004#discussion_r596567999
##########
File path:
processing/src/main/java/org/apache/druid/segment/data/VSizeLongSerde.java
##########
@@ -413,9 +412,81 @@ public void close() throws IOException
}
}
+ /**
+ * Unpack bitpacked long values from an underlying contiguous memory block
+ */
public interface LongDeserializer
{
+ /**
+ * Unpack long value at the specified row index
+ */
long get(int index);
+
+ /**
+ * Unpack a contiguous vector of long values at the specified start index
of length and adjust them by the supplied
+ * delta base value.
+ */
+ default void getDelta(long[] out, int outPosition, int startIndex, int
length, long base)
+ {
+ for (int i = 0; i < length; i++) {
+ out[outPosition + i] = base + get(startIndex + i);
+ }
+ }
+
+ /**
+ * Unpack a non-contiguous vector of long values at the specified indexes
and adjust them by the supplied delta base
+ * value.
+ */
+ default int getDelta(long[] out, int outPosition, int[] indexes, int
length, int indexOffset, int limit, long base)
Review comment:
without pushdown:
```
Benchmark
(distribution) (encoding) (filteredRowCountPercentage) (rows)
(zeroProbability) Mode Cnt Score Error Units
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-12 lz4-auto 1.0 5000000
0.0 avgt 5 20574.724 ± 3674.509 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-16 lz4-auto 1.0 5000000
0.0 avgt 5 20893.434 ± 2894.104 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-20 lz4-auto 1.0 5000000
0.0 avgt 5 19857.499 ± 1926.944 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-24 lz4-auto 1.0 5000000
0.0 avgt 5 22194.340 ± 2624.983 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uinform-32 lz4-auto 1.0 5000000
0.0 avgt 5 18321.336 ± 465.633 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-40 lz4-auto 1.0 5000000
0.0 avgt 5 23252.329 ± 341.846 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-48 lz4-auto 1.0 5000000
0.0 avgt 5 25273.632 ± 1414.751 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-56 lz4-auto 1.0 5000000
0.0 avgt 5 26429.779 ± 2649.011 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-64 lz4-auto 1.0 5000000
0.0 avgt 5 21943.211 ± 2124.446 us/op
```
compare with pushdown:
```
Benchmark
(distribution) (encoding) (filteredRowCountPercentage) (rows)
(zeroProbability) Mode Cnt Score Error Units
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-12 lz4-auto 1.0 5000000
0.0 avgt 5 18272.440 ± 71.751 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-16 lz4-auto 1.0 5000000
0.0 avgt 5 19042.292 ± 595.685 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-20 lz4-auto 1.0 5000000
0.0 avgt 5 18782.746 ± 248.738 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-24 lz4-auto 1.0 5000000
0.0 avgt 5 19048.354 ± 160.025 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uinform-32 lz4-auto 1.0 5000000
0.0 avgt 5 17984.778 ± 633.691 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-40 lz4-auto 1.0 5000000
0.0 avgt 5 22070.007 ± 166.035 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-48 lz4-auto 1.0 5000000
0.0 avgt 5 22052.517 ± 2168.763 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-56 lz4-auto 1.0 5000000
0.0 avgt 5 25001.739 ± 259.184 us/op
ColumnarLongsSelectRowsFromGeneratorBenchmark.selectRowsVectorized
uniform-64 lz4-auto 1.0 5000000
0.0 avgt 5 20325.369 ± 60.724 us/op
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
