[
https://issues.apache.org/jira/browse/ARROW-11693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307359#comment-17307359
]
Antoine Pitrou commented on ARROW-11693:
----------------------------------------
I don't think we need {{utf8proc}}. Counting UTF8 characters is simply a matter
of counting bytes with the upper bit cleared
(Python pseudocode: {{utf8_length(bytestring) = sum((b < 0x80) for b in
bytestring)}})
> [C++] Add string length kernel
> ------------------------------
>
> Key: ARROW-11693
> URL: https://issues.apache.org/jira/browse/ARROW-11693
> Project: Apache Arrow
> Issue Type: New Feature
> Components: C++
> Reporter: Neal Richardson
> Assignee: Eduardo Ponce
> Priority: Major
> Fix For: 4.0.0
>
>
> We have "binary_length" but that doesn't handle UTF-8 the way we need for
> this. Example (from R):
> {code}
> > string <- "áéíóú"
> > nchar(string)
> [1] 5
> > arrow:::call_function("binary_length", Scalar$create(string))
> Scalar
> 10
> {code}
> cc [~maartenbreddels] [~apitrou] [~jorisvandenbossche]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
