[
https://issues.apache.org/jira/browse/CALCITE-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648585#comment-16648585
]
Julian Hyde commented on CALCITE-2619:
--------------------------------------
I think we should focus on the cost of validating whether a string is valid in
UTF-8. (I probably have not seen this issue because I use Latin1 or something
similar.) Is there a cheap general check?
I did a Google search and the top hit was by my friend [~lemire]:
[https://lemire.me/blog/2018/05/09/how-quickly-can-you-check-that-a-string-is-valid-unicode-utf-8/]
> Reduce string literal creation cost by removing charset check
> -------------------------------------------------------------
>
> Key: CALCITE-2619
> URL: https://issues.apache.org/jira/browse/CALCITE-2619
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Ted Xu
> Assignee: Ted Xu
> Priority: Major
>
> The cost of creating NlsString is very high, due to its charset check. In
> some cases, e.g., expression evaluate because of Partition Prune, the
> NlsString creation costs 40%+ of total executor's overhead.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
