Github user holdenk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21247#discussion_r195151361
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala
---
@@ -94,32 +94,15 @@ private[sql] class JSONOptions(
sep
}
+ protected def checkedEncoding(enc: String): String = enc
+
/**
* Standard encoding (charset) name. For example UTF-8, UTF-16LE and
UTF-32BE.
- * If the encoding is not specified (None), it will be detected
automatically
+ * If the encoding is not specified (None) in read, it will be detected
automatically
--- End diff --
Since this comment mentions what happens in the read path, it would
probably be good to mention what happens in the write path (e.g. based on
`JsonFileFormat.scala` it looks like in the write path it defaults to UTF-8.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
