shrirangmhalgi commented on code in PR #55839:
URL: https://github.com/apache/spark/pull/55839#discussion_r3405054888
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEPropagateEmptyRelation.scala:
##########
@@ -52,19 +53,37 @@ object AQEPropagateEmptyRelation extends
PropagateEmptyRelationBase {
// - positive value means an estimated row count which can be
over-estimated
// - none means the plan has not materialized or the plan can not be
estimated
private def getEstimatedRowCount(plan: LogicalPlan): Option[BigInt] = plan
match {
- case LogicalQueryStage(_, stage: QueryStageExec) if stage.isMaterialized =>
+ case LogicalQueryStage(_, physicalPlan) =>
+ getEstimatedRowCount(physicalPlan)
+
+ case _: EmptyRelation => Some(0)
+
+ case _ => None
+ }
+
+ private def getEstimatedRowCount(plan: SparkPlan): Option[BigInt] = plan
match {
Review Comment:
Nit: The wrapper propagation covers `SortExec`, `ProjectExec`,
`ColumnarToRowExec`, and `AQEShuffleReadExec`. Is `LimitExec` intentionally
omitted? A `LocalLimitExec` / `GlobalLimitExec` wrapping a materialized empty
stage would also preserve the 0-row signal (limit of 0 rows is still 0), but
currently falls to case _ => None. Not critical since limits rarely wrap
materialized stages directly, but mentioning for completeness.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
