hcrosse opened a new issue, #1453:
URL: https://github.com/apache/arrow-go/issues/1453
### Describe the bug, including details regarding any error messages,
version, and platform.
`is_not_null` ignores a sliced array's offset, so its mask comes from the
start of the parent array instead of the slice. A filter using that mask can
select null rows and drop non-null rows, with no error.
I reproduced this on v18.8.0 and main at
`22c9f26df0abbc2066f5d9ae45f31720d29a4300`, on macOS with Go 1.27.3. Slicing
`[null, 1, null, 2]` to `[1, null, 2]` gives `[false true false]` from
`is_not_null`, instead of `[true false true]`.
**Reproduction**
Save this as `main.go` in a module that requires
`github.com/apache/arrow-go/v18 v18.8.0`, or in a new subdirectory of a
checkout at `22c9f26d`, then run `go run .`:
```go
package main
import (
"context"
"fmt"
"strings"
"github.com/apache/arrow-go/v18/arrow"
"github.com/apache/arrow-go/v18/arrow/array"
"github.com/apache/arrow-go/v18/arrow/compute"
"github.com/apache/arrow-go/v18/arrow/memory"
)
func main() {
arr, _, err := array.FromJSON(memory.DefaultAllocator,
arrow.PrimitiveTypes.Int32,
strings.NewReader(`[null, 1, null, 2]`))
if err != nil {
panic(err)
}
sliced := array.NewSlice(arr, 1, 4)
out, err := compute.CallFunction(context.Background(), "is_not_null",
nil, compute.NewDatum(sliced))
if err != nil {
panic(err)
}
fmt.Println("input: ", sliced)
fmt.Println("is_not_null:", out.(*compute.ArrayDatum).MakeArray())
// Expected: is_not_null: [true false true]
}
```
Output on both versions:
```text
input: [1 (null) 2]
is_not_null: [false true false]
```
**Expected behavior**
`is_not_null` should return `[true false true]` for the sliced array. Its
result should describe the slice's rows, not the first three rows of the parent.
**Cause and impact**
`isNotNullExec` returns the input validity buffer as its result without
carrying the input offset. The output starts at bit 0, so it reads the parent's
bits rather than the slice's:
https://github.com/apache/arrow-go/blob/22c9f26df0abbc2066f5d9ae45f31720d29a4300/arrow/compute/internal/kernels/scalar_comparisons.go#L732-L745
`isNullExec`, immediately above it, already uses `input.Offset`:
https://github.com/apache/arrow-go/blob/22c9f26df0abbc2066f5d9ae45f31720d29a4300/arrow/compute/internal/kernels/scalar_comparisons.go#L718-L730
Callers that slice before evaluating `is_not_null` can select the wrong
rows. The same happens through `exprs.ExecuteScalarExpression`: on v18.8.0 and
`22c9f26d`, `is_not_null(a)` on `rec.NewSlice(1, 5)` of `a = [null, 1, null, 2,
3]` returns `[false true false true]` instead of `[true false true true]`.
https://github.com/apache/arrow-go/issues/1452 is a separate bug in the same
kernel: it treats a valid index to a null dictionary value as not null.
I have a patch with tests and can open a PR.
### Component(s)
Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
