guixiaowen commented on code in PR #1393:
URL: https://github.com/apache/auron/pull/1393#discussion_r2403970865
##########
spark-extension/src/main/scala/org/apache/spark/sql/auron/AuronSparkSessionExtension.scala:
##########
@@ -68,12 +66,12 @@ case class AuronColumnarOverrides(sparkSession:
SparkSession) extends ColumnarRu
override def preColumnarTransitions: Rule[SparkPlan] = {
new Rule[SparkPlan] {
override def apply(sparkPlan: SparkPlan): SparkPlan = {
- if (!sparkPlan.conf.getConf(auronEnabledKey)) {
- return sparkPlan // performs no conversion if auron is not enabled
+ if (sparkPlan.isInstanceOf[LeafExecNode] &&
!sparkPlan.isInstanceOf[FileSourceScanExec]) {
Review Comment:
> > this is incorrect because a single FileSourceScanExec can still be
optimized to NativeParquetScanExec
>
> @richox The condition is not FileSourceScanExec. A unit test has been
written, and a standalone FileSourceScanExec can be converted.
@richox ri FileSourceScanExec is a subclass of LeafExecNode, and it is
currently the only execution plan being converted. FileSourceScanExec has been
excluded.
##########
spark-extension/src/main/scala/org/apache/spark/sql/auron/AuronSparkSessionExtension.scala:
##########
@@ -68,12 +66,12 @@ case class AuronColumnarOverrides(sparkSession:
SparkSession) extends ColumnarRu
override def preColumnarTransitions: Rule[SparkPlan] = {
new Rule[SparkPlan] {
override def apply(sparkPlan: SparkPlan): SparkPlan = {
- if (!sparkPlan.conf.getConf(auronEnabledKey)) {
- return sparkPlan // performs no conversion if auron is not enabled
+ if (sparkPlan.isInstanceOf[LeafExecNode] &&
!sparkPlan.isInstanceOf[FileSourceScanExec]) {
Review Comment:
> this is incorrect because a single FileSourceScanExec can still be
optimized to NativeParquetScanExec
@richox The condition is not FileSourceScanExec.
A unit test has been written, and a standalone FileSourceScanExec can be
converted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
