Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22778#discussion_r229178084
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -171,10 +171,13 @@ abstract class Optimizer(sessionCatalog:
SessionCatalog)
// "Extract PythonUDF From JoinCondition".
Batch("Check Cartesian Products", Once,
CheckCartesianProducts) :+
- Batch("RewriteSubquery", Once,
+ Batch("Rewrite Subquery", Once,
--- End diff --
I do not have a good answer for this PR. Ideally, we should run the whole
batch `operatorOptimizationBatch`. However, running the rules could be very
time consuming. I would suggest to add a new parameter for introducing the time
bound limit for each batch.
cc @maryannxue WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
