Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20511#discussion_r168834997
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -160,6 +160,15 @@ abstract class OrcSuite extends OrcTest with
BeforeAndAfterAll {
}
}
}
+
+ test("SPARK-23340 Empty float/double array columns raise EOFException") {
+ Seq(Seq(Array.empty[Float]).toDF(),
Seq(Array.empty[Double]).toDF()).foreach { df =>
+ withTempPath { path =>
--- End diff --
It's not about ORC issue. Spark doesn't allow complex type vectorization.
Please see `def supportBatch`.
> When we write these test cases, we should do our best to cover all the
scenarios even if it is not supported now. This is like black-box testing.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
