hcrosse opened a new issue, #1460: URL: https://github.com/apache/arrow-go/issues/1460
### Describe the enhancement requested Setting `MatchSubstringOptions.IgnoreCase` makes `starts_with` return `arrow.ErrNotImplemented` in https://github.com/apache/arrow-go/pull/1448. I would like to support it with Arrow C++'s matching semantics. C++ matches a literal prefix by building `^` plus `RE2::QuoteMeta(pattern)` with case sensitivity disabled. It uses UTF-8 for string input and Latin-1 for binary input, and returns `NotImplemented` without RE2. See its [prefix matcher](https://github.com/apache/arrow/blob/0ab78e2df489055b71a7d26a666a52823f87f01c/cpp/src/arrow/compute/kernels/scalar_string_ascii.cc#L1403-L1422) and [encoding options](https://github.com/apache/arrow/blob/0ab78e2df489055b71a7d26a666a52823f87f01c/cpp/src/arrow/compute/kernels/scalar_string_ascii.cc#L60-L74). For strings, I propose matching Unicode simple folds, without full folding or Turkish special casing. Arrow's pinned RE2 and Go 1.26 use Unicode 15.0 folding tables. Valid UTF-8 needs no special handling, but invalid UTF-8 needs care: - An invalid pattern must return `arrow.ErrInvalid`. - An invalid input byte must not match a pattern character. `strings.EqualFold` alone is unsuitable because it decodes invalid bytes as `utf8.RuneError`, so distinct invalid bytes can compare equal. - Compare characters rather than slicing the input by the pattern's byte length. The Kelvin sign and `k`, for example, fold together but occupy different numbers of bytes. For binary input, match ASCII and Latin-1 byte folds. The additional pairs are `0xC0` through `0xDE` with `0xE0` through `0xFE`, excluding `0xD7` and `0xF7`. Substrait `CASE_INSENSITIVE` can then map to `IgnoreCase`, as it does in C++. I can open a PR once https://github.com/apache/arrow-go/pull/1448 merges. ### Component(s) Other -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
