Github user viirya commented on the issue:
https://github.com/apache/spark/pull/16998
> I do not get your points. What does this mean? Constraint propagation is
a bottom up mechanism for inferring the constraints. Can you elaborate your
idea in the more formal way.
We fully expand the constraints with aliased attributes now. For example,
if there is a constraint "a > b", and current query plan aliases "a" to "c" and
"d". The final constraints of this plan is "a > b", "c > b", "d > b".
The values of those constraints are all the same, either all true or all
false. So in case of inferring filters from the constraints, we only need "a >
b", other aliased constraints "c > b", "d > b" are not necessary.
> I did not read the code. Just wondering if we could miss the chance of
plan optimization after this PR? What is the negative impact, if exists?
The only one optimization I think would be affected is `PruneFilters`.
`PruneFilters` will prune a condition if its child's constraints already
contain the condition. Using above example to elaborate, if there is a `Filter`
above the query plan and its condition is "c > b". As we only have "a > b" in
the query plan's constraint, we can't prune the condition and the `Filter`.
However, this is not a big impact and it can be easily solved. We can use a
simple method to inquire if a given condition like "c > b" is contained in the
fully expanded constraints of a query plan, without really fully expanding the
constraints.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
