hcrosse commented on PR #1448: URL: https://github.com/apache/arrow-go/pull/1448#issuecomment-6638673771
@zeroshade Thanks for checking the C++ behavior. The docs now say dictionaries are decoded and extensions use their storage type. The Substrait comment now says `CASE_INSENSITIVE` returns `NotImplemented`, where C++ maps it to `ignore_case=true`. The cast would work: the `cast` meta function already coalesces views with multiple data buffers. I prefer a direct kernel that reads the 16 byte view headers and their 4 byte prefixes, avoiding a column conversion. It can reuse `coalesceMultiBufferViewDatum` when there is more than one data buffer. For valid UTF-8 I found no case where Go's simple folding differs from RE2's: both use Unicode 15.0 tables. Invalid UTF-8 needs care: RE2 rejects a pattern with invalid UTF-8 and never matches an invalid input byte to a character, while `strings.EqualFold` treats invalid bytes as `utf8.RuneError`. Folded characters can also have different byte lengths. I'd rather keep this PR to the case-sensitive kernel and handle views and `ignore_case` in separate small PRs after it merges, so each comes with its own tests. They're tracked in https://github.com/apache/arrow-go/issues/1459 and https://github.com/apache/arrow-go/issues/1460. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
