GGraziadei opened a new pull request, #17774:
URL: https://github.com/apache/iceberg/pull/17774
interleaveBits moves one bit per loop iteration, and each iteration searches
for the next source column, so the cost is 8 * interleavedSize iterations of a
loop the hardware cannot pipeline. With four Z-ordered columns and an 8 byte
contribution each, that is 256 iterations per row.
When every column contributes the same number of bytes, which is what all
callers produce, the interleaving is a fixed permutation of the source bits
that depends only on the source byte and the column count. Tabulate it:
SPREAD[n][b] holds the bits of byte b spread n positions apart, so OR-ing the
spread bytes of n columns, each shifted right by its column index, yields the n
output bytes for that source offset.
Columns of differing lengths, which no caller produces but the method
accepts, keep using the existing bit at a time loop. The output is unchanged
for every input.
The tables cost 16680 bytes of heap once per JVM, and only the 2 KB row for
the column count in use is read. Nothing is allocated per row, by either
implementation.
ZOrderByteUtilsBenchmark also gained warmup iterations, without which it was
timing largely uncompiled code.
JMH, 10M rows per op, JDK 17, i7-13700H:
fourColumns 7.379 ± 0.563 -> 0.787 ± 0.277 s/op
fourColumns8ByteOutput 1.828 ± 0.459 -> 0.259 ± 0.038 s/op
threeColumns 5.601 ± 0.915 -> 0.685 ± 0.373 s/op
twoColumns 3.668 ± 0.530 -> 0.456 ± 0.033 s/op
gc.alloc.rate.norm is unchanged to the byte (5848, 5824, 5840, 5832 B/op,
all of it the benchmark's own buffer and JMH infrastructure) and gc.count is 0
in every case, before and after.
Closes #17758
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
