Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20511#discussion_r168838556
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -160,6 +160,15 @@ abstract class OrcSuite extends OrcTest with
BeforeAndAfterAll {
}
}
}
+
+ test("SPARK-23340 Empty float/double array columns raise EOFException") {
+ Seq(Seq(Array.empty[Float]).toDF(),
Seq(Array.empty[Double]).toDF()).foreach { df =>
+ withTempPath { path =>
--- End diff --
Then, since the current case is already testing against vectorized
configuration *by default*.
We can add `Seq(false, true).foreach(...
spark.sql.orc.enableVectorizedReader <- value)`. we can add
`vectorization=false` for now. However, both test cases will test MR reader
only for now. If it sounds okay to you, then I'll add that.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
