dannygoldstein opened a new issue, #38139:
URL: https://github.com/apache/arrow/issues/38139
### Describe the bug, including details regarding any error messages,
version, and platform.
Issue: `pc.case_when` gives incorrect results when the input `cond` contains
null struct values, and the length of `cond` is greater than ~70.
Example:
When the length of `cond` is small (in this case 2), I get what I would
expect:
```python
from pyarrow import compute as pc
print(pc.case_when([{'0': True, '1': False}, {'0': None, '1': None}], [0,0],
[1,1], [-1,-1]).to_pylist())
# prints [0, -1]
```
But if cond gets longer than about ~70, it starts replacing all values after
a certain point with the default value. Here's the same example as above, but
50x longer (100 elements),
```python
from pyarrow import compute as pc
pc.case_when([{'0': True, '1': False}, {'0': None, '1': None}] * 50, [0,0] *
50, [1,1] *50, [-1,-1] * 50).to_pylist()
```
Expected behavior:
Should print
[0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0,
-1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0,
-1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0,
-1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0,
-1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1]
Actual behavior:
Prints (note all the -1s at the end).
[0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0,
-1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0,
-1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, 0, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
Pyarrow version: 10.0.1
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
