Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18747#discussion_r145137543
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala
---
@@ -63,6 +64,23 @@ case class InMemoryRelation(
val batchStats: LongAccumulator =
child.sqlContext.sparkContext.longAccumulator)
extends logical.LeafNode with MultiInstanceRelation {
+ /**
+ * If true, get data from ColumnVector in ColumnarBatch, which are
generally faster.
+ * If false, get data from UnsafeRow build from ColumnVector
+ */
+ private[columnar] val useColumnarBatches: Boolean = {
--- End diff --
I think this should belong to the read path. Whether we use columnar batch
scan or not, the write path doesn't need to change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
