1fanwang opened a new pull request, #10672: URL: https://github.com/apache/arrow-rs/pull/10672
# Which issue does this PR close? - Closes #10671. # Rationale for this change `substring()` takes `Utf8`, `LargeUtf8`, `Binary`, `LargeBinary`, `FixedSizeBinary` and `Dictionary`, but rejects the view types, so a caller holding a `StringViewArray` has to cast to `Utf8` first and pay for the copy. The other string kernels in this crate already accept them: `length`, `bit_length`, `like` and `concat_elements` all have `Utf8View` arms. # What changes are included in this PR? `Utf8View` and `BinaryView` arms. Views carry their own offsets rather than sharing one value buffer, so the byte range is computed per element and the result is built through the view builder. The existing `Dictionary` arm recurses, so a dictionary of views now works too. The utf-8 boundary check applies to `Utf8View` the same way it does to `Utf8`. The offset quoted in that error is per element, because there is no shared buffer to index into. # Are these changes tested? Yes. The view result is compared against the `Utf8` and `Binary` result for the same inputs, so the two paths cannot drift: empty strings, nulls, values longer than the twelve-byte inline limit, zero length, lengths past the end, and starts past either end in both directions. Plus the boundary error and a dictionary of views. The cross-check earned its keep — it caught the error offsets differing between the two paths, which is what prompted the note above. <details><summary>Tests</summary> ``` $ cargo test -p arrow-string 186 passed; 0 failed ``` Removing the two new match arms fails exactly the four new tests, so they cannot pass vacuously: ``` test substring::tests::string_view_matches_utf8 ... FAILED test substring::tests::binary_view_matches_binary ... FAILED test substring::tests::string_view_rejects_an_invalid_char_boundary ... FAILED test substring::tests::dictionary_of_string_view ... FAILED ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
