1fanwang opened a new pull request, #51251:
URL: https://github.com/apache/arrow/pull/51251
### Rationale for this change
`winsorize` on a sliced array returns the wrong null pattern: values come
back where the input had nulls, and nulls where it had values. Limits of 0.0
and 1.0, which should return the input unchanged, still corrupt it.
```python
flat = pa.array([1.0, 2.0, None, 4.0, None, 6.0, 7.0, 8.0], pa.float64())
sliced = flat.slice(2, 5) # [None, 4.0, None, 6.0, 7.0]
pc.winsorize(sliced, lower_limit=0.0, upper_limit=1.0).to_pylist()
# before: [0.0, 4.0, None, 6.0, None]
# after: [None, 4.0, None, 6.0, 7.0]
```
`[valid, valid, null, valid, null]` is the parent's bitmap read from bit 0,
not from bit 2.
### What changes are included in this PR?
`ClipValues` built a zero-offset output but assigned the input's validity
buffer to it unchanged, so a non-zero `data.offset` was ignored when the bitmap
was read back. The values loop was already correct, since `GetValues` applies
the offset. The bitmap is now copied out of the slice with `CopyBitmap`, and a
null-free input still shares the no-bitmap fast path.
### Are these changes tested?
Yes. `TestWinsorize.SlicedInput` covers floating point and integer slices
whose parent nulls sit at different positions, plus a null-free slice.
<details>
<summary>Raw logs</summary>
```console
# before: restore the kernel to its pre-change state, keeping the new test
$ git checkout HEAD~1 -- cpp/src/arrow/compute/kernels/vector_statistics.cc
$ ninja arrow-compute-vector-test && ./debug/arrow-compute-vector-test
--gtest_filter='TestWinsorize.SlicedInput'
Actual:
[
0,
4,
null,
6,
null
]
[ FAILED ] TestWinsorize.SlicedInput (6 ms)
1 FAILED TEST
# after: restore the change
$ git checkout HEAD -- cpp/src/arrow/compute/kernels/vector_statistics.cc
$ ninja arrow-compute-vector-test && ./debug/arrow-compute-vector-test
--gtest_filter='TestWinsorize.*'
[ PASSED ] 5 tests.
$ ./debug/arrow-compute-vector-test
[==========] 1146 tests from 152 test suites ran. (2656 ms total)
[ PASSED ] 1146 tests.
```
</details>
### Are there any user-facing changes?
Yes, `winsorize` returns the correct nulls for a sliced input.
Closes: https://github.com/apache/arrow/issues/51224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
