raghavyadav01 commented on code in PR #26640:
URL: https://github.com/apache/datafusion/pull/26640#discussion_r4426383634
##########
datafusion/optimizer/src/decorrelate.rs:
##########
@@ -897,6 +923,37 @@ impl PullUpCorrelatedExpr {
}
}
+/// Whether the emptiness of `plan`'s output follows from the emptiness of its
Review Comment:
Yes, that's the idea. The property is that whether the node returns any rows
depends only on whether its input has any, not on which ones. A `Limit` with no
skip and fetch > 0 has it too, so if every node above it does, the subquery
returns a row exactly when the `Limit`'s input does, with or without the
`Limit`. That's the whole argument.
User-defined plans aren't in the list, so for an `Extension` the `Limit`
stays and the subquery isn't decorrelated, the same as the `offset 1` case. I
rewrote the doc comment along these lines and renamed the function to
`depends_only_on_input_having_rows`.
##########
datafusion/optimizer/src/decorrelate.rs:
##########
@@ -897,6 +923,37 @@ impl PullUpCorrelatedExpr {
}
}
+/// Whether the emptiness of `plan`'s output follows from the emptiness of its
+/// input alone, so that `plan` cannot tell apart two non-empty inputs.
Removing
+/// a `Limit` below such a node changes which rows it is given, never whether
it
+/// is given any, and so never whether the subquery is empty.
+///
+/// A `Projection`, a `SubqueryAlias`, a `Sort`, a `Window` and a `Repartition`
+/// each output one row per input row. A `DISTINCT` and a grouped `Aggregate`
output rows
+/// exactly when their input has rows, and an `Aggregate` without a grouping
+/// always outputs its one row, so no `Aggregate` can be emptied by the rows a
+/// `Limit` held back.
+/// A materialized CTE outputs exactly its continuation's rows, and only the
+/// continuation is rewritten, so its emptiness follows that input.
+///
+/// A `Filter` is what this leaves out: it reads values, so the rows the
`Limit`
Review Comment:
Yes. A `Filter` keeps rows by their values, so which rows the `Limit` kept
decides whether any get through. Reworded that paragraph to say it directly.
##########
datafusion/optimizer/src/decorrelate_predicate_subquery.rs:
##########
@@ -2820,6 +2820,137 @@ mod tests {
)
}
+ /// A `Limit` whose row count nothing above it can observe is still
dropped,
Review Comment:
Not much. The SLT cases cover the same shapes, and a subquery that isn't
decorrelated fails to plan, so the passing SLT results already show the rewrite
happened. Removed them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
