Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21668
One of refactoring ideas is to inject the functionality of
`ReorderJoin`(=`StarSchemaDetection`)
into `CostBasedJoinReorder`;
In [the batch rule `Join Reorder` (`Once`
strategy)](https://github.com/apache/spark/blob/7c08eb6d61d55ce45229f3302e6d463e7669183d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala#L139),
if `spark.sql.cbo.starSchemaDetection` enabled (false by default), the rule
applies star schema detection first. If a fact table found, [dimension tables
are
reordered](https://github.com/apache/spark/blob/7c08eb6d61d55ce45229f3302e6d463e7669183d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/StarSchemaDetection.scala#L342)
by the cost-based algorithm. If `spark.sql.cbo.starSchemaDetection` disabled,
the rule just uses `CostBasedJoinReorder`.
Currently, we have `ReorderJoin`(=`StarSchemaDetection`) in the batch rule
with `fixedPoint` strategy,
so, I thnk that, if we could remove this rule from there, we would skip
unnecessary checks caused by `ReorderJoin` per rule iteration.
@cloud-fan WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
