Hi Prateek!

Le 13/11/2026 à 07:44, PRATEEK GAUR a écrit :

I would be curious if this has been experimented with already
(pseudo-random sampling).

Thanks, Antoine. Your comment prompted me to revisit the Arrow C++ sampler
and experiment
with a simpler approach using fixed-seed pseudo-random sampling.
- Pick evenly spaced value vectors across the page. This ensures values are
picked from different
   parts of the input.
- From each vector, pick 32 (fixed) pseudo-random values using a fixed seed
so the result remains
   deterministic. This ensures a statistically meaningful sample while
further reducing the sample
   size (to reduce CPU compute).

Trying this on the dataset I didn't see any meaningful regression in
compression ratio or
decoding speed. So we can simplify the implementation.

Thank you, but my proposal goes a bit further: how about dropping the "evenly spaced value vectors" and instead just sample the desired number of values (for example 256) from the page?

Some PRNGs are extremely fast and I doubt the cost of sampling would be significant compared to the actual cost of deriving ALP parameters from the samples?

Regards

Antoine.


Reply via email to