alamb commented on code in PR #25802:
URL: https://github.com/apache/datafusion/pull/25802#discussion_r4324591458
##########
datafusion/common/src/config.rs:
##########
@@ -1380,6 +1380,18 @@ config_namespace! {
/// reading them as binary and then casting to string.
pub binary_as_string: bool, default = false
+ /// (reading) If true, a column the parquet file stores as binary is
+ /// read directly as a string when the table schema declares a string
+ /// type for it. The parquet reader only validates UTF-8 for columns
Review Comment:
What does "invalid string array" mean here? It seems like creating invalid
arrays with non utf8 data is a bad idea to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
