dtenedor opened a new pull request, #12274:
URL: https://github.com/apache/arrow-rs/pull/12274
# Which issue does this PR close?
None.
# Rationale for this change
x86 has no vector `f64`<->`i64` conversion before AVX-512DQ. The ALP
encoder's per-value round trip
(`fast_round`'s `as i64`, `decode_value`'s `as f64`) therefore keeps the f64
encode loop scalar,
with `is_impossible_to_encode` called as a function per value. The decoder's
inverse FOR loop is
scalar for the same reason, and the FOR min/max over `i64` is emulated on
SSE2.
# What changes are included in this PR?
For an integer `i` with `|i| <= 2^51`, the bits of `MAGIC_NUMBER + i` are
`MAGIC_NUMBER.to_bits() + i`. This turns the rounding and both conversions
into float and integer
adds:
- `AlpFloat` gains batch methods: `encode_values`, `decode_values` and
`min_max_encoded`. Their
default implementations are the existing scalar loops, and f64 overrides
them.
- **Encode** goes 64 values at a time. Values that the magic path can't
represent are redone with
the existing scalar code while still in cache: a scaled magnitude of at
least `2^51`, NaN, ±inf,
and -0.0. Once a whole chunk misses, the rest of the vector uses the
scalar path directly, so
high-precision data doesn't pay twice. Output is byte-identical.
- **FOR min/max**, when every value fits, runs in the float domain with
compare-select. The values
are exact integers, never NaN.
- **Decode** checks once per vector, from `frame_of_reference` and
`bit_width`, that every value is
within `(-2^51, 2^51)` before using the magic path.
- `#[inline]` on the per-value helpers lets the f32 loops vectorize too.
# Are these changes tested?
Covered by the existing ALP tests. Encoded output was also checked to be
byte-identical, and decoded
output bit-identical, to main's. The datasets covered:
- decimal and high-precision values
- NaN, ±inf, -0.0 and subnormals
- values at the `2^51` boundary, large integers, mixed scales and constants
They used lengths around vector boundaries, over 3 pages with uneven `get`
sizes.
`alp` bench (128K values), Intel Xeon Platinum 8559C, default x86-64 target,
median of 5 alternating
rounds:
| `alp` bench, f64 | main | PR | Speedup |
| :--- | ---: | ---: | ---: |
| decode/page/decimal_with_exceptions | 98.44 µs | 58.66 µs | 1.68x |
| decode/page/high_precision | 186.17 µs | 186.01 µs | 1.00x |
| decode/random_100/decimal_with_exceptions | 14.83 µs | 14.81 µs | 1.00x |
| decode/random_100/high_precision | 55.33 µs | 45.36 µs | 1.22x |
| encode/first_page/decimal_with_exceptions | 2.196 ms | 1.801 ms | 1.22x |
| encode/first_page/high_precision | 3.354 ms | 3.029 ms | 1.11x |
| encode/next_page/decimal_with_exceptions | 742.76 µs | 539.60 µs | 1.38x |
| encode/next_page/high_precision | 2.097 ms | 1.979 ms | 1.06x |
f32 keeps the scalar default methods, which vectorize once the per-value
helpers are `#[inline]`:
encode 1.17-1.27x, decode 1.06-1.12x.
An earlier version did a full scalar fix-up pass after the vector pass. It
slowed f64
high-precision encoding to 0.95x, which is why the fix-up works per 64-value
chunk. None of the
measured inputs are slower in this version. Not measured on aarch64: NEON
converts between f64 and
i64 natively, so the f64 gain there is likely smaller.
# Are there any significant user-facing changes?
No. The new methods are on `AlpFloat`, which is crate-private.
# AI usage disclosure
An AI coding agent (Claude Code) generated the code and this description,
and ran the benchmarks and
the output checks. This PR is a draft until the submitter has reviewed it.
This pull request and its description were written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]