comphead commented on code in PR #12342:
URL: https://github.com/apache/arrow-rs/pull/12342#discussion_r4635495723
##########
arrow-cast/src/cast/decimal.rs:
##########
@@ -226,11 +257,12 @@ where
// div is >= 10 and so this cannot overflow
let (d, r) = x.div_rem_wrapping(div);
- // Round result
- let adjusted = match x >= I::Native::ZERO {
- true if r >= half => d.add_wrapping(I::Native::ONE),
- false if r <= half_neg => d.sub_wrapping(I::Native::ONE),
- _ => d,
+ // Round half away from zero. Whether a value rounds depends on the
+ // data, so select the result instead of branching on it.
+ let adjusted = if x >= I::Native::ZERO {
+ std::hint::select_unpredictable(r >= half,
d.add_wrapping(I::Native::ONE), d)
+ } else {
+ std::hint::select_unpredictable(r <= half_neg,
d.sub_wrapping(I::Native::ONE), d)
};
Review Comment:
Thanks, I checked each point.
The data and the branch are as you say. Both generators draw from
`0..1_000_000_000`, and the sign test is a branch on `main` as well, a `tbnz`
on bit 63 after `div_rem_wrapping` on aarch64 and a `testq` and `js` pair on
x86-64-v3. This PR keeps it on aarch64, x86-64 and x86-64-v3, and the remainder
select compiles to `csel` or `cmov` on all three.
I added the two cases as the first commit, c48302010, for both the
`Decimal256` and the `Decimal128` downscale: random signs in ±1e9, and positive
multiples of 10^delta_scale. I measured your suggestion and also a version
without the sign test. The remainder has the sign of `x`, so `r >= half` and `r
<= half_neg` already exclude each other:
```rust
let adjusted = std::hint::select_unpredictable(
r >= half,
d.add_wrapping(I::Native::ONE),
std::hint::select_unpredictable(r <= half_neg,
d.sub_wrapping(I::Native::ONE), d),
);
```
Criterion medians on an M5 Max, three rounds in rotating order, change
against `main`:
| Benchmark | `main` | This PR | Suggestion | No sign test |
|---|---:|---:|---:|---:|
| `cast decimal256 to decimal128 512` | 48.6 µs | -6% | +10% | +11% |
| ... `mixed sign` | 88.8 µs | -20% | -10% | -12% |
| ... `zero remainder` | 48.0 µs | -4% | +14% | +13% |
| `cast decimal128 to decimal128 512 with lower scale (infallible)` | 19.1
µs | -1% | +3% | -3% |
| ... `mixed sign` | 19.5 µs | -1% | +2% | -3% |
| ... `zero remainder` | 18.9 µs | +1% | +3% | -2% |
None of the six is slower with this PR beyond noise. Mixed signs do cost a
lot, 88.8 µs against 48.6 µs on `main`, but samply puts that time in
`i256::div_rem_wrapping` rather than in the closure. Per iteration of the
`Decimal256` case with this PR:
| | All positive | Mixed sign |
|---|---:|---:|
| `i256::div_rem_wrapping` | 17.7 µs | 44.6 µs |
| Per-row closure, with its sign branch | 20.0 µs | 18.7 µs |
The hot spots in `div_rem_wrapping` are its own sign branches (`tbz x20,
#63`, and the `self.is_negative() == other.is_negative()` test). On x86-64, 3
of its 11 conditional branches are `jns` sign tests. The sign branch in the
closure goes the same way right after them, so it appears to be predicted from
that history. The suggestion keeps the 44 µs in `div_rem_wrapping` and makes
the closure cost 27 µs to 28 µs on both inputs, because both comparisons run on
every row.
So I'd keep the sign branch here. A sign-branch-free `i256::div_rem` looks
like the fix for mixed-sign `Decimal256` input, and it affects `main` the same
way, so I'd do it in a separate PR. These numbers are from Apple Silicon. I'll
run the bot with c48302010 as the baseline to check x86.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]