Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20949#discussion_r203228243
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala
---
@@ -146,7 +148,12 @@ private[csv] class CsvOutputWriter(
context: TaskAttemptContext,
params: CSVOptions) extends OutputWriter with Logging {
- private val writer = CodecStreams.createOutputStreamWriter(context, new
Path(path))
+ private val charset = Charset.forName(params.charset)
+
+ private val writer = CodecStreams.createOutputStreamWriter(
+ context,
--- End diff --
tiny nit:
```scala
private val writer = CodecStreams.createOutputStreamWriter(
context, new Path(path), charset)
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
