hcrosse opened a new pull request, #1448: URL: https://github.com/apache/arrow-go/pull/1448
### Rationale for this change Executing a Substrait expression that calls `functions_string` `starts_with` fails with `not implemented: starts_with`, because `arrow/compute` has no such function. I added a `starts_with` function and mapped the Substrait call to it. iceberg-go translates prefix row filters to this call, so they fail at read time (https://github.com/apache/iceberg-go/issues/2416). With this change, those filters returned the expected rows when I ran iceberg-go against it locally. ### What changes are included in this PR? `starts_with` accepts `binary`, `large_binary`, `string` and `large_string`. Dictionary inputs are decoded and extension inputs use their storage type. Null inputs produce null. An empty pattern matches every non-null value. Substrait calls require a non-null literal pattern and accept only `CASE_SENSITIVE`, which is also the default. `ExprBuilder.CallScalar` and `NewScalarCall` can build a `starts_with` call by name. I left out: - View types. View arrays carry a variable number of data buffers, but `ArraySpan` holds at most three, and the `scalar` package has no view scalars. - `IgnoreCase`. Arrow C++ implements it with RE2, which folds case as UTF-8 for strings and as Latin-1 for binary. Matching that needs its own change. ### Are these changes tested? Yes. I added two tests and a serialization case. - `TestStartsWith` covers the four input types as arrays and scalars, nulls, empty inputs and patterns, non-UTF-8 binary, slices, chunks, dictionaries and invalid options. - `TestStartsWithExpr` runs Substrait calls on `large_string` and `varchar` columns. It checks that a non-literal pattern, a null pattern and `CASE_INSENSITIVE` return `arrow.ErrNotImplemented`. - `TestExpressionSerializationRoundTrip` gains a `starts_with` case. On Linux amd64, `go test -count=1 ./arrow/compute/...` passed, along with `gofmt`, `go vet` on the three changed packages, `go build ./arrow/compute/...` and `golangci-lint` on `arrow/compute/...`. The `prek` hooks, including the Release Audit Tool, also passed. I did not run `noasm`, arm64 or 32-bit builds. ### Are there any user-facing changes? Yes. `arrow/compute` adds `StartsWith`, `MatchSubstringOptions` and `RegisterScalarString`, and the default registry includes `starts_with`. `arrow/compute/exprs` adds `SubstraitStringFuncsURI` and maps Substrait `starts_with` in both directions. Setting `IgnoreCase` returns `arrow.ErrNotImplemented`. Existing functions are unchanged. Part of https://github.com/apache/arrow-go/issues/1329. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
