Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/20513#discussion_r166211814
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala
---
@@ -61,6 +61,9 @@ case class InMemoryTableScanExec(
}) && !WholeStageCodegenExec.isTooManyFields(conf, relation.schema)
}
+ // TODO: revisit this. Shall we always turn off whole stage codegen if
the output data are rows?
+ override def supportCodegen: Boolean = supportsBatch
--- End diff --
I think this is safe to keep the same behavior of 2.2.
I'm not sure if enabling whole stage codegen can hurt performance for scan
nodes, btw. We can revisit this, of course.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
