Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20877#discussion_r176324829
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala
---
@@ -251,5 +254,8 @@ private[sql] class JacksonGenerator(
mapType = dataType.asInstanceOf[MapType]))
}
- def writeLineEnding(): Unit = gen.writeRaw('\n')
+ def writeLineEnding(): Unit = {
+ // Note that JSON uses writer with UTF-8 charset. This string will be
written out as UTF-8.
--- End diff --
I meant here:
https://github.com/apache/spark/blob/de36f65d3a819c00d6bf6979deef46c824203669/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala#L88-L93
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
