[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

Tue, 24 Jul 2018 23:00:56 -0700 

Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20949#discussion_r204988574
  
    --- Diff: 
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
 ---
    @@ -514,6 +516,41 @@ class CSVSuite extends QueryTest with SharedSQLContext 
with SQLTestUtils with Te
         }
       }
     
    +  test("SPARK-19018: Save csv with custom charset") {
    +
    +    // scalastyle:off nonascii
    +    val content = "µß áâä ÁÂÄ"
    +    // scalastyle:on nonascii
    +
    +    Seq("iso-8859-1", "utf-8", "utf-16", "utf-32", "windows-1250").foreach 
{ encoding =>
    +      withTempPath { path =>
    +        val csvDir = new File(path, "csv")
    +        Seq(content).toDF().write
    --- End diff --
    
    nit: `.write.repartition(1)` to make sure we write only one file


---

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to