Hi Prateek!
Le 13/11/2026 à 07:44, PRATEEK GAUR a écrit :
I would be curious if this has been experimented with already
(pseudo-random sampling).
Thanks, Antoine. Your comment prompted me to revisit the Arrow C++ sampler
and experiment
with a simpler approach using fixed-seed pseudo-random sampling.
- Pick evenly spaced value vectors across the page. This ensures values are
picked from different
parts of the input.
- From each vector, pick 32 (fixed) pseudo-random values using a fixed seed
so the result remains
deterministic. This ensures a statistically meaningful sample while
further reducing the sample
size (to reduce CPU compute).
Trying this on the dataset I didn't see any meaningful regression in
compression ratio or
decoding speed. So we can simplify the implementation.
Thank you, but my proposal goes a bit further: how about dropping the
"evenly spaced value vectors" and instead just sample the desired number
of values (for example 256) from the page?
Some PRNGs are extremely fast and I doubt the cost of sampling would be
significant compared to the actual cost of deriving ALP parameters from
the samples?
Regards
Antoine.