Github user koertkuipers commented on a diff in the pull request: https://github.com/apache/spark/pull/23173#discussion_r237663865 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1987,6 +1987,18 @@ class CSVSuite extends QueryTest with SharedSQLContext with SQLTestUtils with Te assert(errMsg2.contains("'lineSep' can contain only 1 character")) } + test("SPARK-26208: write and read empty data to csv file with header") { + withTempPath { path => + val df1 = Seq.empty[(String, String)].toDF("x", "y") --- End diff -- that doesnt seem to be what is happening. if i do a .repartition(4) on empty dataframe it still only writes one part file with header if i do a .repartition(4) on a dataframe with 2 elements then it writes 2 part files with headers so it seems empty partitions get pruned, except when all partitions are empty then it writes a single partition thanks to SPARK-23271
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org