Le 12/11/2026 à 13:42, Gang Wu a écrit :
I'm trying to improve the ALP implementations on the Parquet C++ side:
https://github.com/apache/arrow/pull/55449. So I'm really interested
if you have any document on performance comparison among different
implementations.
When reviewing this PR, the question came up of why use such weird and
ad-hoc sampling schemes instead of actual pseudo-random sampling using a
good and fast PRNG.
I would be curious if this has been experimented with already
(pseudo-random sampling).
Regards
Antoine.
Thanks!
Gang
On Thu, Nov 12, 2026 at 6:50 PM Gunnar Morling via dev <
[email protected]> wrote:
Hey all,
We've added ALP encoding and decoding support to Hardwood too [1], [2].
Interop with parquet-java's implementation looks good: Hardwood decodes
pages written by parquet-java's ALP writer bit-exactly and vice versa.
Perhaps interesting: on the encoder side, we build the preset of a column
chunk from eight sample vectors distributed across the entire chunk (easy
in our case, as Hardwood's writer buffers whole chunks), whereas, IIUC,
parquet-java only samples within the first ~110K values of a chunk.
Depending on the data distribution, this can yield a more efficient
encoding, for instance when a high-cardinality column changes character
midway, e.g. moving from two to four decimal places (about 2.8× smaller in
a synthetic test with 1M doubles).
Best,
--Gunnar