Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20511#discussion_r167952000
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala
---
@@ -160,6 +160,16 @@ abstract class OrcSuite extends OrcTest with
BeforeAndAfterAll {
}
}
}
+
+ // This is a test case for ORC-285
--- End diff --
Thank you, @omalley . Yes. It is. There is one difference in Spark.
Currently, Spark uses the vectorized reader for simple schema only. For
`array<float>` or `array<double>` columns, we use ORC MR reader. However, MR
reader uses vectorized RowBatch internally and it fails here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
