weiyuyilia commented on code in PR #39802:
URL: https://github.com/apache/spark/pull/39802#discussion_r1090460819
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala:
##########
@@ -145,6 +145,8 @@ class CSVFileFormat extends TextBasedFileFormat with
DataSourceRegister {
override def equals(other: Any): Boolean = other.isInstanceOf[CSVFileFormat]
override def supportDataType(dataType: DataType): Boolean = dataType match {
+ case _: BinaryType => false
Review Comment:
As description in
[SPARK-42237](https://issues.apache.org/jira/browse/SPARK-42237):
- when writing, it works, but content written in csv files is not the actual
value of the binary column.
- when reading, it doesn't work, throws unsupported exception in
UnivocityParser.
It causes that if users write csv tables with binary columns, like using udf
UnBase64, these tables can't be read follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
