Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/20004#discussion_r158576836
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala
---
@@ -148,6 +149,9 @@ class CSVOptions(
format.setDelimiter(delimiter)
format.setQuote(quote)
format.setQuoteEscape(escape)
+ if (charToEscapeQuoteEscaping != '\u0000') {
--- End diff --
Because we always call `format.setQuoteEscape(escape)`, this default value
becomes wrong. That means, if users set `\u0000`, the actual quoteEscape in
`univocity` is `\`.
Let us use `Option[Char]` as the type of `charToEscapeQuoteEscaping `? We
call `format.setCharToEscapeQuoteEscaping ` only if it is defined.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
