[
https://issues.apache.org/jira/browse/IMPALA-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17933514#comment-17933514
]
Csaba Ringhofer commented on IMPALA-12675:
------------------------------------------
I am not against this, but it should be noted that it is still easy to write
STRINGs in Impala that is not valid UTF-8. utf8_mode is still false by default
and even if it is true, UTF-8 validity is generally not checked (e.g. when
reading/writing Parquet and BINARY->STRING casts).
It could make sense to add some check for this when writing Parquet files and
return a warning/error when the STRING is not valid UTF-8. Probably this
wouldn't cause a major perf regression.
> Set PARQUET_ANNOTATE_STRINGS_UTF8 to true by default
> ----------------------------------------------------
>
> Key: IMPALA-12675
> URL: https://issues.apache.org/jira/browse/IMPALA-12675
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Priority: Major
>
> Now that we have the BINARY type for binary data, and also have UTF-8
> support, it makes sense to set PARQUET_ANNOTATE_STRINGS_UTF8 to true by
> default in a new major release
> *We should only resolve this issue if we are ready for a new major release.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
