Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/15319
After more thoughts, I discovered various ways to create non-converge
constraint set(in the following cases, a represents `Alias(f(b, c), "a")`):
Condition 1. `a = b` will infer `f(b, c) = f(f(b, c), c)`;
Condition 2. `a = d, b = d` will infer `a = b` and therefore deducts `f(b,
c) = f(f(b, c), c)`;
Condition 3. `a = d, d = e, e = f, ..., x = b` will deduct `f(b, c) =
f(f(b, c), c)` too after certain iterations;
Condition 4. `a = d, b = d` will also infer `f(d, c) = f(f(d, c), c)` and
so on.
For the Cond. 1/2/3, we can avoid to create non-converge constraint set by
checking whether constraint contains `a.child`(e.g. f(b, c)), but for the Cond.
4, I haven't figure out a work around.
A new approach will be avoid replace an `Alias` into constraint, it will
ensure that `QueryPlan.inferAdditionalConstraints` and
`UnaryNode.getAliasedConstraints` won't both apply and therefore no
non-converge constraint set will be created. For this approach, we may miss
some constraints that could have been infered, but I think it won't be super
harmful because these constraints are commonly complex that rarely filters more
data.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
